A fascinating thing about Iris’ description of her synesthesia and out-of-body like fantasy life is the hints it offers about the workings of her brain, as well as of others. The great pattern matching machine, which researchers think the brain must be, has to spend most of its first years learning and storing patterns that it acquires through the senses. Patterns that it receives from the eyes obviously make optical sense and are perceived as such when acquired. However, in Iris’ case with her synesthesia, audio signals from the ears, and do not make any optical sense, get into her vision system where they are still perceived as images. But what kind of images can they be. In Iris’ case they tend to be light shows, like fireworks or shooting stars. The brain expects images from the vision system, but when they originate as sound signals, the higher layers don’t have any sensible visual patterns to recognize so it has to create some out of thin air so to speak, which are then perceived. I imagine the situation similar to a computerized pattern recognition system where the IO driver has a bug so that signals from the microphone go to the optical pattern recognizer software, which of course can’t make sense of it. However, it is programmed to treat stuff coming in as potential patterns so it proceeds to process them as if they were optical signals. Similarly, the higher layers in Iris’ brain treats these nonsensical signals in such a way that she perceives them as light shows, as he describes in the following excerpt from her book. What a tantalizing glimpse into the deepest mystery of the brain, i.e. how it generates experiences, that this story hints at. Is it possible that this, what brain researchers call the hard problem, may yield some of its mystery from the clues here. Thinking about how it works that Iris perceives fantastical light shows in response to sounds, could that maybe lead to some ideas about what is happening in the brain to produce that sensation.
….When I had done something that farmor thought was very sinful and wrong she would call me up to her. Then I had to stand before her as she scolded and admonished me. This I liked. I stood totally still and stared out in space. Her words were flying around and formed shapes, it was different lights that sparkled. It was beautiful, and you could never predict how it would look. I stood there and was entranced. I flowed into the colors, shapes, patterns, let myself be carried back and forth, up and down. Everything changed shape continuously and I flowed in with it.