By Bryan Bergeron View In Digital Edition
A fundamental application of electronics is transformation — from power supplies that transform AC to DC to microprocessors that transform binary code to text, graphics, and sound. While transformations at many levels may be critical to the function of a device or application, it’s the high-level sensory transformations that hold the greatest potential to change the world. I’m talking about the high-level transformation of sensory experiences, illustrated at one level by Craig Lindley’s article on a digital color organ.
Transforming sensory data from one form to another has both entertainment and pragmatic value. For example, a color organ that transforms sound amplitude and/or frequency to colors can serve as an entertainment device and as an aid to configuring the graphic equalizer front-end of a stereo system. More significantly, sensory transformation may be used to augment normal senses and to supplement damaged sense organs.
Consider the matrix of potential sensory transformation in Table 1 as an aid to exploring how sensory transformation can be applied to correct a sensory deficient or to augment a person’s normal abilities. The matrix is an over-simplification to the extent that each sense may take on several variables or may be modulated. For example, hearing may involve the perception of pitch, amplitude, and direction, and vision can encompass color, intensity, movement, and size.
| Vibration | Pressure | Hearing | Vision |
| Pressure | Sound Localization | ||
| Proprioception | Flight Indicator | ||
| Balance | Flight Attitude Indicator | ||
| Hearing | Frequency Shift | Light Organ | |
| Taste | |||
| Smell | |||
| Vision | Haptics | Frequency Shift (IR Camera) | |
| Temperature | IR Camera, No-contact Thermometer | ||
| Pain | |||
| Additional headers not listed at the top also include: Proprioception, Taste, Smell, Temperature, and Pain. | |||
TABLE 1. Sensory Transformation Matrix showing common transformations.
A prominent feature of the transformation matrix is that it is sparse — with a few prominent exceptions, sensory transformation is largely unexploited. There is the familiar light organ that transforms sound to either fluctuations in LED bar graphs or dynamic displays in Apple iTunes or Microsoft Media Player. Computer-controlled haptics interfaces provide virtual screen objects with realistic mass, elasticity, and momentum. Flight
attitude indicators supplement a pilot’s vestibular system by showing attitude and an artificial horizon. As suggested by the matrix, useful transformations can be performed within a given sense, such as augmenting hearing by transforming high frequency audio signals into signals within the normal range of human hearing, or transforming infrared light to visible light.
One reason for the relatively few sensory transformations in everyday use is the lack of appropriate sensors and transducers. My latest focus in esoteric sensors is on the various light sensors from Texas Advanced Opto-Electronic Solutions (TAOS, www.taosinc.com) and Texas Instruments (www.ti.com), including light-to-voltage and light-to-frequency chips and a variety of color sensors. Light-to-frequency chips convert light intensity to a microcontroller-compatible pulse train with a frequency proportional to light intensity. With a microprocessor on the sensor output, light intensity can be easily converted to sound, color shift, smell, or pressure, given an appropriate transducer.
Another reason for the lack of sensory transformations is the difficulty in determining what is practical and useful in everyday life. One area of active research and development is medicine. Consider a diabetic with the inability to detect pain in his feet because of the loss of sensation (diabetic neuropathy). Because of this loss of sensation, diabetics often neglect sores, cuts, and bruises on their feet, resulting in infections and, in some cases, gangrene. However, pressure sensors in the shoes or shoe inserts of a diabetic could transform a sudden pressure gradient — ordinarily perceived as pain — into a prick or modest electric shock on the leg, an audible alarm, a flashing light, or even an odor.
Study the matrix in Table 1, devise your own applications, and then dive in with the appropriate sensors, transducers, and microcontrollers. And don’t be constrained by the matrix — many useful assistive devices add new senses. For example, a talking compass or GPS provides direction and location information that a normal person can’t determine from his innate senses.
If you’d like a ready source of inspiration, google “synesthesia.” People with this condition experience sensations seemingly unrelated to the initial stimulus. For example, a particular sound may evoke the visualization of a color — akin to a built-in light organ. NV
