WhiteCane: A Virtual Assistant for the Visually Impaired

Students of Indian Institute of Technology Kharagpur, Barnopriyo Barua, Kaustubh Mani, Divyansh Gupta won the award for research work of WhiteCane: A Virtual Assistant for the Visually Impaired

We are using various artificial intelligence techniques to simulate a digital assistant that uses the smartphone camera to help visually challenged users in various day to day tasks. The app will take a spoken query as an input from the user.

This query will be converted from speech to text and will then be categorized into one of the many functions like Screen Description, Currency Recognition, OCR, Facial Recognition, Visual Question Answering, Object Localization etc. using the latest state-of-the-art Computer Vision and Natural Language Understanding techniques. This query, along with an image captured by the camera will be passed as an input to the appropriate computer vision model based on the intent predicted in the previous step. The result will then be spoken back to the user by our application, by using android’s text-to-speech functionality. Until now, assistive technologies in this field were usually in the form of standalone hardware devices, which turned out to be prohibitively expensive for most users. The advent of cheap smartphones allows us to use the camera to help users in day to day tasks like currency recognition, face identification, reading, etc. all through an easy to use voice based interface.

Dr. Harsh Vardhan, Minister of Science and Technology, awarded the Gandhian Young Technological Innovation (GYTI) Appreciation to Barnopriyo Barua, Kaustubh Mani, Divyansh Gupta , Indian Institute of Technology Kharagpur at the GYTI 2018 Awards function held at Rashtrapati Bhawan on March 19, 2018.

Leave a Reply

Your email address will not be published. Required fields are marked *