How technology enabled us to ‘see’ with our skin
A brief history of Sensory Substitution Devices
WHAT ARE OUR SENSES?
We perceive the world around us by interpreting information from our five senses — we see the room around us, listen to the music, feel the seat beneath us, smell the rainy air and taste the warm mug of tea — in other words we see, hear, touch, smell and taste our reality. In addition to these five we also have numerous other less well-known senses, including: proprioception — the sense of our body’s position in space; nociception — the sense of pain; interoception — the sense of ones’ internal organs; and temporal perception — the sense of time (among others).
Our brains receive information from our sensory organs — e.g. the eyes, ears, skin, nose, tongue — interpreting the incoming electrical signals and conveying them to us as a sensory perception — for example the brain receives waves of light through the eyes, and conveys this to us as the vision of the room. However, our brains are unaware of which sense organ it is receiving information from, because all the sensory information reaches the brain as electrical signals. Therefore it is possible to send in one type of sensory information and have it interpreted as another sense (e.g. ‘hearing’ light), or have the brain interpret completely new information as a sensory perception (e.g. ‘feeling’ Twitter posts).
The brains’ ability to interpret new incoming information as a sensory perception is what allows us to create technology which can change our sensory perception of the world.
SENSORY SUBSTITUTION DEVICES
So how do we augment our senses? We have been using tools and technology to augment our senses throughout our history — the telephone extends our hearing across the globe, and telescopes allow us to see into the depths of space. For this article we are going to jump to a more recent history of technology and look at Sensory Substitution Devices (SSDs); these are generally medical devices which use one sense to compensate for another impaired sense.
For example, in 1969, pioneering neuroscientist Paul Bach-y-Rita created a Tactile Visual Sensory Substitution (TVSS) device which receives visual information through a camera — such as the shape, size and orientation of an object — and transforms the light waves into patterned vibrations. This is then transmitted to a grid of vibrating stimuli on a chair which stimulates the back of a subject who is visually impaired. The subject interprets the pattern of vibrations as the visual perception of the object in front of them, enabling them to ‘see’ through their skin.
Over time, the subject learns to understand the different patterns of vibrations as different visual objects in front of them — just as we learn to read by interpreting different shapes as letters. Though it cannot be claimed that the device is restoring the subjects’ vision:
‘if ‘seeing’ is defined as the ability to create a mental representation of the shape, surface properties, and location of surrounding objects and to interact with them in a manner comparable to a normally sighted person, then SSDs indeed enable the blind to “see”’ (Maidenbaum et al., 2014, p. 5).
HOW THE TECHNOLOGY HAS ADVANCED
Later, Bach-y-Rita created a similar, more accurate and much smaller, device which could be placed on the subjects’ tongue — called the Brain Port. The subjects closed mouth protects the device and their saliva creates a good electrolytic environment for the patterned stimulations (Bach-y-Rita et al., 1998).
The most widely used SSD is the vOICe app (capitalized letters representing “Oh, I see”) developed by Peter Meijer in 1992 which uses auditory-vision substitution for subjects with visual impairments. The app receives visual information through the phones’ camera, and converts the image into a soundscape. The vOICe app has the advantage of being portable and low cost (unlike Back-y-Rita’s TVSS) which has promoted its use among the visually impaired community.
WHERE ARE THEY NOW?
While these devices showed great promise for assisting people with sensory impairments, there are several key reasons why SSDs are not in widespread use now, 50years after they were created.
- Devices can be expensive and cumbersome
- The devices are not accurate enough for everyday use (they do not compensate well enough for the impaired sense)
- Some people are reluctant to try new devices
- The biggest restriction is the length of time it takes to train with a device to achieve adequate sensing capabilities
Though there has been limited success with widespread use of SSDs (Maidenbaum et al., 2014), the technology has shown great potential for enhancing sensory perceptions. The principles of Sensory Substitution (transforming one type of sensory information into another sense) can be used to create Sensory Augmentation technologies which can enhance your existing senses, convey novel information through your senses or give you new senses!
Follow our weekly blog to keep learn more about the senses and technology.
Discover more: cyborgnest.net | lesleyanndaly.com
