Synaesthesia – an essay on the adjacency of the sense of hearing and the other senses

During my Master’s degree in musicology at Utrecht University, I took the course “A History of Modern Listening”. This blog post is based on an essay I wrote for that course, and published in response to a Tweet by Elise Cordaro.

Picture by David Matos via Unsplash.

“My mind to me a Kingdom is…” ­

– Sir Edward Dyer

For a physicist, concepts like “colour” or “pitch” might be a complete triviality, captured into a single number. But for me, they evoke not only numerous associations, but also each other. Monday is a red falling minor third. C major is bright, friendly and yellow. And my cat smells like freshly baked apple pie. Letters, numbers, persons… in my world, everything has colours, smells and sounds. This little confession -­ I have always been oddly embarrassed to talk about it ­- of sensory confusion is called synaesthesia (“synesthesia” in American spelling), from the Ancient Greek σύν “together” and αἴσθησις “sensation”.

Synaesthesia occurs when one of the five senses is aroused, yet more then one respond. Because of my blending of the traditional perceptions, it seems rather funny to me to write an essay on the adjacency of the sense of hearing and (one of) the other senses. To me they simply appear too close to each other to properly distract. Of course I know that most people strictly separate hearing from feeling and seeing, but I often think about myself as having just one sense, that can be aroused by different sorts of input.

In this short essay, I would like to explore various thoughts and associations connecting my personal experiences with various theories about the adjacency of the senses. Therefore, this essay is more a tool for thinking ­ and a jumble of ideas than a fixed and and scholarly presentation of facts. To describe the adjacency of the sense of hearing and the other senses, I will explain how I experience language – letters, words, spoken – and music. Along the way, concrete examples of experiences where others may have less senses responding will be given. Supposedly as many as 1 out of 200 people have synaesthesia, but apparently most of them, like me, are shy about it. Therefore, I hope that my perception can add to the discourse about synaesthesia. After these descriptions, I will try to establish a connection between my perception of the world and more general ­ non­synaesthetic ­ ideas about the adjacency of senses. But let’s start with a short introduction about the history of my sense(s).

Her-story
As long as I remember, I’ve experienced colours coming along with the sounds around me. This is completely involuntary, it just happens. Personally, I cannot easily image what it would be like to live without this – it is the only reality I know – but for many people, this makes be an inhabitant of a mysterious no-man’s-land between fantasy and reality. I even had to do three years of kindergarten, as the teachers said I was living in a fantasy world, babbling about flying colours and more nonsense, and therefore I was certainly not ready to go to primary school. (Although I could already read quite well with colour as a kind of visual building block.) Because of the amount of experiences each sound evokes, I often had a sensory overload as a child. I cried when I felt overwhelmed by having too much sensations at once, but when people asked me what was wrong, I could not describe my perceptions, knowing it would confuse them. Those whom I did tell about it often wrote it off as “foolishness” and/or “posturing”. Because of my behaviour, I had to visit a child’s psychiatrist when I was 13 years old. I got to talk to his assistant, a young woman, and was completely honest with her, trusting that she would explain everything to the doctor and he would be wise and understanding. But what happened was almost the opposite: the doctor accused me of “fakery”.

Looking back, it is inconceivable that no one thought of autism at the time. Sensory overload, a strange fixation about colours, use of metaphor meaningful only to the speaker (“salty sounds”), unable of making decisions, experiencing everything all at once, a too precise eye for unimportant details, intellectual gifted, yet poor at spelling (dyslexia)… it was all there – loud and clear. Decades later, after reading much literature about both autism and synaesthesia (and having my brain scanned) I can accept that I am just wired differently. Now I try to use this special perception to my benefit, instead of feeling like an anthropologist from a different planet (to paraphrase Temple Grandin). I will always remain bad at making decisions and remain liking to combine what to other seem very different things. Don’t blame me, the poor little brain is simply not capable of thinking ‘normal’ aka neurotypical. So, how does my brain work, then, in regards to sensory processing? Let’s explore some of the perceptions that I learned to differ from average.

Letters & colours
It has often been stated that synaesthetes have a fixed colour for each letter, but in my experience, that’s not completely true. For me, each letter does indeed have a different colour, but this colour can depend on the context. Then it seems to be coherent with the meaning of the letter. But often, it depends more on sound than on shape, I think (I am not quite sure about this, because I hear the shapes as well). This also goes across languages: the Dutch letter I is green, but the English word I has the colour of the person it refers to. Logically, the colour of the letter also depends on the alphabet it is used in. When I started with Russian, I learned most letters rather quickly, because I can see the и as green and therefore know it is similar to the I. This happened with most letters ­ except for the ones that have (almost) the same shape as the Latin letters I knew, but come with a different sound. For example the Cyrillic R, which looks like our P. It also took me a while before I could distinguish the Russian н (sounding like N) from the H that I knew, because I experienced the same colour. A few years later, the same thing happened with the Greek H. But after some time, the colours of the different H­-shaped letters started to change. Now they have different colours, depending on the alphabet and language in which they are used. But I still have problems with English words written in Russian letters or vice versa, because then the colours don’t match.

The same happened when I learned the Roman numerals, I didn’t recognise VIII as being blue until my Latin teacher trained us in Roman calendar counting. This suggests that at first it is not the numerical concept of a number ­ I knew VIII was eight ­ but the grapheme’s visual appearance – the symbol 8, which I think of while calendar counting ­ that drives the colour. There have been various reports on which colour is linked to which letter and it turned out that for example the A’s are often red – so is mine. A scientific explanation for this phenomena is that more common letters get primary colours, while infrequently letters are experienced with less ordinary colouring. To me, this makes sense, because my X is some sort of copper or bronze. Therefore, as a teenager, I liked to end my emails with xMM, which has caused me all sorts of problems, ranging from classmates who thought I had a crush on them to a teacher who thought I was intentionally rude.

Words
The same nuance has to be applied to the scholarly claim that for a synaesthete each word has a different and fixed colour. In my case it is true that generally, a word gets the same colour as the “strongest” (mostly the first) letter, except for words written in capitals. I am profoundly dyslexic, but not many people notice, because those colours give me a natural memory system, which helps me read, spell and remember words very well. I am also rather good at Swedish puzzles (in which you have to find words in a field of letters) and the colours help me to find passages in text quickly. (These experiences are scientifically confirmed by the tests by Vilayanur S. Ramachandran and Edward M. Hubbard.) But just like the colours of the letters and numbers, the colours of a whole word are not static; they can change depending on the context. The word “ballet” for example is often grey with pink, but in the combination “modern ballet” the grey is very dark – almost black – and the pink leans more towards purple. Moreover, names are also made up from different colours. To give you an example of my associations by a name, let’s return to my little friend with the smell of freshly ­baked apple pie: my cat. I named him Timon, which sounds like broken chords in G major, played fast on a vielle­-like instrument:

His name starts with T, the orange­ meets Ochre like, Judas­-colour his pelt has. (In Dutch, we call this colour “rossig”, but that word is so much darker because of the red brown R that I usually avoid it.) After this T comes the complementary colour spring green of I, the colour of the grass Timon likes so much. His M is what I call “double ­coloured” – two colours at the same time, as if you were wearing sunglasses with two different coloured lenses – because it is in the middle of his name. Narcissistic enough: M is my favourite letter. Most of the time, it is double ­coloured: purplebrown and magoon red at the same time ­ “a colour from Mars” as Robbert Dijkgraaf calls it. The O is a very light shade of blue (actually much more airy than in the image above). The N is greyish blue with silver sparks, that tinkle like little bells. After reading this, I think you can image why I chose the name Timon for my cat.

Let’s move on to words that are not names. As stated above and often confirmed in literature, most of the time, words have the colour of the first letter. I would nuance that by saying that they have the colour of the strongest letter, because that way, I have less exceptions myself. But there is something non­logical about words in different languages. I don’t know whether this is related to synaesthesia, because I have not been able to trace it in the literature, yet. But to me, the same word in another language can evoke another colour and another image. The English word cat is dark blue and I think about a cat with stripes, some sort of Cyprian tabby. But the French word le chat is black (while la chatte is black with pink) and evokes the image of a white cat, with long hair and a ribbon. I have different cats for all the words for cat I know and so it goes for all the other words. While
translating, I am often looking for words that have similar colours and images. Strangely enough, when I was working on my thesis about Vladimir Nabokov’s Lolita, I often felt that exactly the right match had been made for the Russian version. Likely, the colours of Nabokov and me are very different, but on some sort of higher level, we might make the same connections. This instinct for the colours of words can be also felt in works by other authors, like for example Anna Akhmatova, William Blake, Emily Dickinson (“With blue, uncertain, stumbling buzz / Between the light and me”) ­ and many more poets of whom I do not know whether they had synaesthesia or not.

Music
As explained above, letters and words have certain colours, that I experience while reading and writing. The same goes for sounds. Each type of sound has it’s own colour, shape and texture. But the shades vary when sound is moving, for example when people are talking. Sometimes this can be very unpleasant. For example, when a certain teacher (I won’t mention his name) is talking, I often see high thin red lines, angrily fuzzing out. Those can even become painful, like a migraine. But most voices are nice – fortunately.

This experience of sound is the same with music, but of course the musical colours and patterns are more interesting, complex and organised. My most beautiful and therefore most difficult to explain experiences are while being involved in music. When I perform music, I live in a whirl of colour. I “see” music as colours, which is an almost impossible experience to describe, but let’s say that when I hear a sound, the sound is a coloured shape. This “image of sound” is as real as the sound itself, because in fact they are the same thing. Just like ordinary sounds and people talking, the colours and shapes depend on the pitch and timbre – for example, the base notes are ubiquitous balls ­ but when I hear or play music, they all blend together. I always think of this as the reason I love music so much.

Music also has a sort of taste to me, therefore I find it more difficult to listen to music when the taste of the harmonical solutions is different from the taste usually produced by the tone interval. To be honest, as a professional musician, I dare to doubt whether this sound-colour synaesthesia gives me an advantage. As a child, I was tested for “perfect pitch” and was told I had it, but I don’t think I have it. Maybe I lost it because I started playing in so many different tunings (after a while, the low C on my baroque cello became as yellow as on my modern instrument). Still, I use my synaesthetic experiences of music artistically when I write liner notes, as then I look for words that have colours that fit the colours of the music. It makes me very happy when I find a ‘perfect match’.

Perfect Match
As explained above, I live in a lucky kaleidoscope. But sometimes, my experiences collide with the world outside my world. Intellectually I know we all are just wired differently, but when facing it, I feel like an outsider. For example: I often feel like the colours of letters as written in advertisements are “wrong”. Not even different, but just plain wrong. To me, this wrong colouring is as absurd as the purple cow on the Milka chocolate TV commercial, but I try to be tolerant. Sometimes the opposite happens: I experience a “perfect match”, when everything seems to make sense. This always makes me very, very happy. I took a photograph of my friend Ela when she was dressed in magenta purple, because I think of her as being a very purple person, with a love for the purple art of modern ballet and photography, and the reddish purple opera Carmen. Moreover, the strong colour of the middle L in her name is definitely purple too. If you ask me about the adjacency of the senses, these perfect matches are what I think about.

Another example was when I had to do a presentation about the Loreley song. Although I like the song and do think it fits the poetry (both are in various shades of green, which look rather nice next to the colours of the name Loreley), I did not much like the video I had of it, because the man on it was singing in such kitschy colours. But when after class the teacher started to sing along, I was happy as can be with the warm greenish clouds of his singing. By the way, I am not sure whether this is clear, but the word “warm” as used in the last sentence does not refer to shade but to a sense that lies somewhere between touch and sense of temperature. It is a bit like the feeling of blushing, while being in love, embarrassed or drinking alcohol. Often I feel this “warmth” in my mouth, as if I have just drunk a nicely warm cup of coco. I could not find this in literature, but I think it is the same connection as when a sound hurts, although I would locate that somewhere behind my eyes.

(I will leave this point for now, surpassing al kinds of sexually loaded senses, as these work in the same manner).

Adjacency of the senses
My senses are most closely connected while I am either involved in music or recalling memories. Especially those perfect matches described above have often made lasting impressions of vivid sensory memories and are therefore very close to me. This may seem a bit far­fetched, but rather than distinguishing sound from vision, I classify my experiences by place, especially the perfect matches, because they have such a strong sense of location. Just like we all do when making an appointment, we look in our agenda’s, pick a date and image the place where we will meet. Many people do this as well when they are remembering certain events from their past, they see time as having a shape and or a place. And to me, this spatially extension is the most wonderful aspect of
our senses. They span distance, cultures and time, by connecting us intimately to our own past, in a way that none of our cherished ideas ever could. In this classification, the exact nature and name of those experiences does matter less. Moreover because they all arrive from “personal coding”, a term Patricia Lynne Duffy uses to describe the unique way in which each person codes information and makes a one-of-a-kind “inner map” of the world around them.

Following the thought of various books on Buddhism I’ve read, I would dare to state that there is no such thing as objective reality, because ­ whether you have synaesthesia or not ­ we only have our individual perceptions. I realise that no one else could ever really sense what just I see and that makes me feel pretty lonely at times. But surprisingly, our experiences may be more similar than we think. We all perceive the outside world with our whole being — we experience it with our entire body sensorium. Consider the synchronisation between eyes and ears, that complement each other: while the eye sees only 180 degrees, the ear hears a panoramic 360. So, seeing and hearing are very close to each other, which is naturally, because both tell us about distant parts of our environment by receiving vibes and waves.

Numerous complementary synchronisations of two senses can be traced back in our language. Don’t we all say things like cheese tastes sharp, music is soft, the girl is hot, his voice is warm…? Yes, we do. And in Dutch we even talk about timbre as “klankkleur” ­ the colour of sound. These examples can easily be regarded to as “just” metaphors in a flowery language, but the fact that they work means that also non­-synaesthetes can imagine what it would be like to have two or more senses working closely together. The awful endurance of nails-­on-­chalkboard is an auditive phenomenon which many people can relate to as being physically felt. Probably you do not feel cold when you look at a picture of an ice cube, but after the encounters you may have had with ice and snow, you might at
least imagine a feeling of coldness. As Michel de Montaigne states: “A strong imagination creates its own reality”. Perhaps we can also agree that loud tones are brighter than soft tones? And that lower tones are darker than higher tones? At that point, you are entering my world.

Conclusions?
I hope that the reading of this descriptions of my personal experiences have brought your a bit closer to understanding a synaesthetic mind. Some examples may be funny, but I don’t mind embarrassing myself for the sake of knowledge. It is rather difficult to explain what I see/hear/taste/feel, because our languages do not provide proper words for the blending of senses and the experiences in between the senses. Moreover, it is not perfectly clear to me how a non-synaesthete experiences the world. For example: many people told me that they actually do see/imagine/feel/experience colours with music if they close their eyes. But many others claim they don’t, they only see the black behinds of their eyelids. So perhaps the Buddhists are right and should we get rid of the idea of such a thing as an objective reality? More research should bring us closer to answering this and other questions rising in this essay. What strikes me most while digging in the existing literature, is that most scholars seem to agree that colours are static and that the colour of words are just depending on the first letter. As I explained above, for me, the situation is often much more nuanced. The meaning and sound of a word also have an undeniable influence on its colour(s). One last example: the word “banana” is as yellow for me as I imagine it is for anyone else (as an experience or in imagination), despite the fact that its letters are predominantly red:

So the A can often be a perfect match, that sounds (vowel as well as musical key) and looks red. But is does not have to be so, like in my perception:

Therefore, I think that I might conclude that – for synaesthetes and non­-synaesthetes alike -the adjacency of the senses is much more complex than we use to regard it to in our ordinary languages.