I remember, I remember,
Where I was used to swing,
And thought the air must rush as fresh
To swallows on the wing;
My spirit flew in feathers then,
These lines from the famous poem ‘I Remember, I Remember’ by Thomas Hood, describing the experience of a child playing on the swing, capture the exhilaration that children derive from simple games. We have similar memories from our own childhood, of moments when our Barbies, Legos and HotWheels weren’t at hand, like during long rides on the school bus, and we were no less exuberant because we could thump our fist against the palm and play “Rock, Paper and Scissors”, or gesture with our fingers to play "Chopsticks", or choose to play any of the myriad other hand games. And before we noticed all those otherwise dull and boring hours would fly by, leaving behind a bunch of happy kids.
So, when asked to come up with an idea of a toy for the visually impaired, team Toy Story decided that we had to introduce them to the treasure trove of hand games - which they couldn’t normally play due to their impairment. For this, we partnered with Samarthanam Trust for the Disabled, an NGO based out of Bangalore, India, that provides skill training and empowerment to the visually impaired. We got to know about Samarthanam through a colleague - who discovered this trust three years ago, and has ever since been associated with it – and knew that we had found the right organization. What makes Samarthanam go hand in glove with the aspirations of Project Playtime, is that its trustees (a majority of whom are visually impaired) are fervid patrons of sport. In fact, the Cricket Association for the Blind in India (CABI) is the sporting wing of Samarthanam, and also the winners of the last two seasons of blind world cup! (How cool is that? Check out this video to learn how blind cricket is played.)
Figure 1 Brainstorming for the prototype
To enable the blind to play hand games, we need to find solutions to the following problems:
We went to the drawing table with these pointers in mind and brewed up a prototype. We would take a regular fabric glove and glue flex sensors on to the glove fingers – one piece per finger. When the flex sensor was bent, its resistance would change. So, a simple potential divider circuit could be used to detect if a finger was bent or straight. We would also mount an IMU sensor, which is a three channel accelerometer combined with a three channel gyro, at back of the glove. The accelerometer data would help in detecting motion of the hand while the gyro data would help in determining the orientation of the hand. And finally we would have to rig up a Bluetooth link between the glove and a smart phone, so that an application running on the phone could act as the arbitrator - enforcing the game rules and deciding the outcome of a match. The phone could also be used to provide auditory cues and feedback to the players.
With this infrastructure in place, went ahead and developed a simple hand game - rock, paper and scissors. See the video here. The video shows a player wearing the glove and competing against a computer. We designed a simple dashboard to display the moves of the player and the computer, and their respective scores. There are audio cues embedded into the application to inform the player of his moves (this is of course for demonstration purpose only, more detailed cues need to be implemented).
The scope of the glove stretches beyond being used as a toy. It could also be used as a controller for appliances that a visually impaired person would use on a daily basis. For instance, the visually impaired now rely on screen reading software to use their phones. So if they need to place a call or dial in a password, they need to raster scan their finger across the screen until they find the right number or character. But using a glove that can recognize gestures and communicate wirelessly, the person can use gestures to dial numbers and manage calls. Many other applications can be thought of, once we know the capabilities of the glove, and contrariwise, the capabilities of the glove can be tailored to meet the expectations of certain applications.
We will soon visit Samarthanam Trust with the prototype and get direct feedback from the children there.
Figure 5 3D rendering of "The Gauntlet"