The sound of shininess: Cross-modal influence of auditory pitch on the perception of gloss

When shopping for jewelry online, we typically just see pictures of items, and cannot hear or feel them. One might assume that this is not a big loss—at the very least, that removing auditory or tactile information should not influence our perception of intuitively visual properties, such as gloss. However, it is also possible that perceivers irresistibly integrate auditory information, such as pitch, into their perception of gloss. We investigated this in two experiments. In Experiment 1, subjects saw pairs of spheres, which were both rendered in the same material (metal, wood, or leather). The spheres differed slightly in glossiness, and subjects reported which was glossier, while ignoring concomitant sounds. In one condition, the glossier sphere was paired with a high-pitched sound and the less glossy sphere with a low-pitched sound; in the other condition, the pairings were reversed. Surprisingly, subjects could not ignore the sounds when discriminating gloss. Rather, they were more accurate when the glossier sphere was paired with the high-pitched sound, suggesting an automatic association between higher pitch and higher gloss. These objects were computer-generated; does the same association also hold when viewing pictures of real objects? In Experiment 2, subjects viewed pictures of jewelry from the Metropolitan Museum of Art’s digital archive. Each item was paired once with a high-pitched sound and once with a low-pitched sound. Subjects were instructed to ignore the sounds, and to rate the gloss of each item from “Not at all shiny” to “Very shiny”. They rated jewelry items as much shinier when paired with the high-pitched sound, indicating that the association between high pitch and high gloss also holds for pictures of real objects. We conclude that when displaying objects in digital spaces such as online stores and museum catalogs, auditory pitch can be used to drive impressions of gloss.