VizLens: An Appliance Reader App for Users with Visual Impairment

OVERVIEW  

VizLens is a mobile app and backend system that enables blind and visually impaired users to independently operate everyday appliances using a combination of crowd-sourced human labeling and real-time computer vision.

• Quickly transforms any physical interface (like microwaves or kiosks) into an accessible, interactive experience for users with visual impairment.  
• The most valuable market opportunity is to provide scalable, low-cost accessibility solutions for both consumer appliances and public interfaces, improving independence for millions globally.

BACKGROUND 

As the world moves toward sleeker, touch-based appliances and public devices, interfaces are more frequently hidden behind smooth surfaces without tactile or braille cues—leaving many blind users dependent on sighted assistance or costly, specialized adaptations. This lack of accessibility represents not just a barrier to independence, but a mounting market opportunity: globally, hundreds of millions of individuals with visual impairments seek solutions for day-to-day access to home and public devices. Existing approaches, such as custom tactile markers, self-voicing appliances, or conventional vision-based tools, are often limited, expensive, or highly device-specific. The rapid adoption of smart devices and the growing awareness of digital inclusion trends underscore the need for robust, generalizable accessibility solutions like VizLens.

INNOVATION  

VizLens introduces a user workflow in which a blind individual takes a photo of an appliance's interface (such as a microwave or checkout terminal) with their smartphone. The image is sent to multiple crowd workers who quickly label and describe each visual element—like buttons and screens—without any prior knowledge of the device, making it adaptable to new or legacy devices. These labeled images serve as references; later, when the user hovers their finger near the interface, VizLens employs computer vision to match the live camera image to the labeled reference, dynamically identifying which control the user’s finger is near and providing real-time voice feedback and guidance. Compared to prior solutions, VizLens is device-agnostic, fast, cost-effective, and robust even for complex or changing interfaces. Its second version further adds adaptability for dynamic screens, efficient text reading (using OCR), and wearable camera support, overcoming problems like camera framing and evolving device states.

ADDITIONAL INFORMATION

REFERENCES: "VizLens: A Robust and Interactive Screen Reader for Interfaces in the Real World"