Seeing AI Advances & Updates
Since it’s initial launch in 2017, the Seeing AI app from Microsoft is the gift that keeps giving to visually impaired people. We embraced the intelligent camera for it’s ability to translate text, images, objects and environments into spoken words. Even as it inspired many competitors, Seeing AI has maintained it’s prominent position. As the software continued to improve, the app’s interface did not change much…until now.
The Seeing AI interface has been consolidated into 3 main tabs labeled: Read, Describe and More. The idea was to streamline what was previously a dedicated “channel” for each feature. The result does not seem simplified, at first, and it takes some getting used to. Here’s how the reshuffled tabs lay out.
READ Tab
The first tab combines three channels, from previous versions, for reading Short Text, Documents and Handwriting. Point the camera at any text, or handwritten words, and it will read instantly. Taking a picture of the text may improve accuracy and will allow you to ask for more information.
DESCRIBE Tab
In the second tab, take a photo of an environment and it will be described to you in detail, including recognition of objects and people. This feature offers option to have photos in your library described, and to ask for more information or explore image by touch.
MORE Tab
The third tab is home to all the remaining channels, seven in all, each standing on it’s own.
- PRODUCT – Reads barcodes and accessible QR codes. Audio guidance helps you find the code, then scans it. In the grocery store, for example, the code might identify the brand and flavor on the ice cream container, or the fat content of the milk, or if the bread is wheat or white.
- PERSON – Use the camera to scan and tell you about the people nearby. Facial descriptions include an estimate of age and expression. Set face recognition to announce the name of someone you know when they come into view.
- CURRENCY – Select the paper currency you wish to identify (choose from 17 currency options) and hold the camera over a single note to hear the value.
- WORLD — An experimental combination of artificial intelligence and augmented reality helps you navigate unfamiliar indoor spaces in 3D guided by spatial audio using headphones. Caution is advised and feedback is appreciated.
- FIND MY THINGS — Teach Seeing AI to recognize the items you often find yourself searching for, like keys, wallet, AirPods, sunglasses, etc. Follow the prompts to take videos of the item and save it to your list. Once set up, the camera can scan the environment to help you locate the selected item through audible beeps that quicken as you get closer to it.
- COLOR – Point the camera at an item and it will identify the color.
- LIGHT – For users with no light perception, this channel indicates intensity and direction of light through audible tone and pitch changes.
The Seeing AI app is available for iOS on the App Store, and for Android on Google Play.
Get quick help at the upper right corner on every channel.
Feedback helps improve the app, so do share your thoughts through the app menu or by email to [email protected].
Post originally published on Jan 19, 2018 and most recently updated Apr 16, 2025
