Translating Sounds into Visual Images, and Vice Versa

This article outlines the history of theoretical developments, techniques and technologies for translating audio track sounds into graphic elements or pictorial images, and vice versa. The presentation includes various attempts by philosophers (Pythagoras, Aristotle), artists (Guiseppe Arcimboldo, Olivier Messiaen, Alexander Scriabin, Arnold Schoenberg, Wassily Kandinsky, Valentin Afanasyev, Richard David James), natural scientists (Isaac Newton, Robert Hooke, Ernst Chladni, Hans Jenny), engineers (Louis Bertrand Castel, Evgeny Sholpo, Arseny Avraamov, Boris Yankovsky) to reveal the unity of color and sound and to translate colors as well as drawings into sound forms and vice versa. It features a whole variety of such translation types throughout the history of humankind, including mathematical, neuropsychological, physical, technical, and software translation. In the first type, translation is carried out by correlating the wavelength of a particular sound or musical tonality with a certain hue. Its adherents created a color and sound matching table based on their calculated mathematical formulas. Some of them then construct technical devices (like color-harpsichords) that allow this translation to be clearly demonstrated to the public. Within the framework of the neuropsychological approach, the phenomenon of synesthesia is being analyzed as well as the manifestations of this ability for a holistic perception of reality by artists and musicians who tried to convey their experience through works of art in which color and sound are constantly converted into each other. Representatives of physical translation focus on translating sounds into graphic forms and conduct experiments on the effect of sounds of different frequencies and amplitudes on physical substances such as sand, special powder or even liquids, classifying the resulting graphic forms. Adepts of technical translation are mainly engaged in sound recording, which is carried out by applying certain graphic patterns to film. A modern software approach allows for the translation of sound and image into each other in both directions using digital technologies.