Since it’s initial launch in 2017, the Seeing AI app from Microsoft is the gift that keeps giving to visually impaired people. We embraced the intelligent camera for it’s ability to translate text, images, objects and environments into spoken words. Even as it inspired many competitors, Seeing AI has maintained it’s prominent position. As the software continued to improve, the app’s interface did not change much…until now.
The Seeing AI interface has been consolidated into 3 main tabs labeled: Read, Describe and More. The idea was to streamline what was previously a dedicated “channel” for each feature. The result does not seem simplified, at first, and it takes some getting used to. Here’s how the reshuffled tabs lay out.
Seeing AI graphic image with app icon overlay. Image credit: Microsoft
READ Tab
The first tab combines three channels, from previous versions, for reading Short Text, Documents and Handwriting. Point the camera at any text, or handwritten words, and it will read instantly. Taking a picture of the text may improve accuracy and will allow you to ask for more information.
DESCRIBE Tab
In the second tab, take a photo of an environment and it will be described to you in detail, including recognition of objects and people. This feature offers option to have photos in your library described, and to ask for more information or explore image by touch.
MORE Tab
The third tab is home to all the remaining channels, seven in all, each standing on it’s own.
PRODUCT – Reads barcodes and accessible QR codes. Audio guidance helps you find the code, then scans it. In the grocery store, for example, the code might identify the brand and flavor on the ice cream container, or the fat content of the milk, or if the bread is wheat or white.
PERSON – Use the camera to scan and tell you about the people nearby. Facial descriptions include an estimate of age and expression. Set face recognition to announce the name of someone you know when they come into view.
CURRENCY – Select the paper currency you wish to identify (choose from 17 currency options) and hold the camera over a single note to hear the value.
WORLD — An experimental combination of artificial intelligence and augmented reality helps you navigate unfamiliar indoor spaces in 3D guided by spatial audio using headphones. Caution is advised and feedback is appreciated.
FIND MY THINGS — Teach Seeing AI to recognize the items you often find yourself searching for, like keys, wallet, AirPods, sunglasses, etc. Follow the prompts to take videos of the item and save it to your list. Once set up, the camera can scan the environment to help you locate the selected item through audible beeps that quicken as you get closer to it.
COLOR – Point the camera at an item and it will identify the color.
LIGHT – For users with no light perception, this channel indicates intensity and direction of light through audible tone and pitch changes.
The Seeing AI app is available for iOS on the App Store, and for Android on Google Play.
Get quick help at the upper right corner on every channel.
Feedback helps improve the app, so do share your thoughts through the app menu or by email to [email protected].
Post originally published on Jan 19, 2018 and most recently updated Apr 16, 2025
This article was originally published April 1, 2017 and most recently updated on October 23, 2024.
With a welcomed emphasis on accessibility and inclusion, numerous companies now offer specialized support services for customers with vision loss. They are there to help you get the most out of their products and to ensure your devices are set right to accommodate your needs. This is an effort we greatly appreciate and are delighted to spread the word.
Call to speak with an accessibility specialist who can help you buy books and other products, or get you directly to tech support for Amazon devices and services including Alexa, Kindle & Fire Tablet. Hands raised triumphantly for accessible and inclusive smartphones.
Arrange the assistance you need for navigating the airport, security screening, location of gate, seat number, connecting flights, baggage claim and ground transportation.
For users of Apple’s iPhone, iPad, iPod Touch, Apple Watch, or Mac – this is a gift. It is a special support line reserved for customers with vision, hearing, motor, and learning impairments. Senior AppleCare specialists are available 24/7 to help you get your Accessibility Settings right, answer your questions, and resolve your technical issues. They’ll even teach you a thing or two.
AT&T
National Center for Citizens with Disabilities (NCCD)
Call Comcast’s Accessibility Support Center to speak with a specially trained representative about accessibility features. You can also communicate via email at: [email protected].
A special and all-inclusive service from Delta, for customers with accessibility needs, book your air travel and accessibility assistance, all together.
Google
To request a call back, from a Google Disability support specialist, complete this online form.
Call an accessibility specialist at Google through the BeMyEyes app on a smartphone or tablet. Click the ‘Service Directory tab’ button on the app’s home page, select Google in the Technical tab.
Google Technical Support: 855-971-9121 may (or may not) be able to help with accessibility questions, depends on the specialist you connect with.
Contact an accessibility specialist at the LinkedIn Disability Answer Desk through the Be My Eyes app, on a smartphone or tablet; click the Specialized Help button on the apps home page, then select Technical to get to LinkedIn.
The features you need to make your PC ‘vision friendly’ are built right into Windows – unfortunately, many people don’t know that. Microsoft recognized this as a significant problem and put in place a support team specialized in accessibility. So dial them up and get your computer set up to work for you (not against you). And, to make it even easier on yourself — allow them remote access to your PC and they can make the adjustments.
Peloton
The Accessibility Team at Peloton will answer your questions and give you a call, if you request it by email to: accessibility@onepeloton.com.
Contact the Accessibility Team at Samsung for answers to questions, large and small, about all Samsung products. Reach out by phone or email: [email protected] to request a call back.
For general support in the U.S. contact Samsung at: 800-726-7864 by text or voice.
Get help from your smartphone or tablet with any questions you may have about Spotify, the world’s largest music streaming service. Speak to a trained customer service specialist on BME; go to Specialized Help, Technical, then Spotify.
T-Mobile has an Accessibility Support web page. It says, “T-Mobile will provide accessible support for wireless network service and billing inquiries. Please contact our Accessibility Customer Care at 1-833-428-1785 if you have any questions or need special assistance. For additional support, please contact Customer Care.”
Arrange the assistance you need for navigating the airport, security screening, location of gate, seat number, connecting flights, baggage claim and ground transportation.
Free 411 for your home phone and for your mobile phone. Get your bills in the format of your choice…large print or digital. They can help you with anything else pertaining to your home phone service.
It can sometimes be challenging to get information about accessibility features that make your mobile phone, or tablet, work for you. Questions about enlarging text, using voice commands effectively, or using the screen reader, often remain unanswered when posed to your store representative. That should be the case no more – now there is a customer service and technical support center dedicated to helping people maximize the accessibility features built into all of Verizon’s mobile devices. Let them help you get your device set just right for you, from 8am – 9pm EST, Monday – Friday.
We will continue adding Accessibility Support services as we discover them. Please let us know if you know of any before we do, and we’ll share by posting to this list.
Dorrie Rush, Chief Content Editor of OEPatients.org
2019 Is A Year of Possibilities for People with Vision Loss
It is true, I do love a fresh new year brimming with promise. My optimistic outlook continues to center on the factor most impacting life with vision loss today, and that is accessible and inclusive technology. In terms of progress, last year was no slouch either, and it set the pace for more good things to come!
The momentum in everyday accessible technology has been building for more than a decade now. It started with Apple’s delivery of the iPhone in 2009, accessible out-of-the-box for people with low or no vision. All products that followed from Apple included accessibility by design. The paradigm was shifting. These developments significantly changed the direction of my life, I no longer felt technology was leaving me behind.
Last year we were the beneficiaries of a few big surprises coming from somewhat unexpected places.
Microsoft created two new important accessibility apps. Seeing AI is a smart camera with 9 apps in 1, and Soundscape is a “3D map with sound.” This delightful surprise extends even further, both apps are free and they were built exclusively for Apple’s iPhone. A magnanimous gesture by Microsoft, perhaps more will be revealed about this in the coming year.
I enjoyed every one of our accessibility events last year and delighted in the lively, sophisticated and always curious audiences. I’d have to say the most fun was our session on smart speakers. They are entertaining and incredibly useful in daily life. Smart speakers are being adopted at a faster rate than smartphones were, and they are especially popular with people visually impaired or blind. This is a category we will continue to pay close attention to.
At another event we applauded the proliferation of audio description, now available on demand for Broadway shows and at the movies. I love the voice that speaks quietly in my ear and fills in the details, I might have otherwise missed.
So, it’s pretty obvious why we should be enthusiastic about the coming year for inclusivity. Apple doesn’t like to say much about their developing technology, but there are reports and rumors that lead us to believe we may be seeing some smart glasses and even clothing designed for accessibility. Actually, I would not be surprised to find many variations of smart glasses showing up, and would expect they’ll begin shrinking to a more normal, wearable size, from the rather heavy visor like devices we saw last year.
We can look forward to new developments in navigating airports and shopping centers.
There is Aira to watch, a service that teams technology with the eyes of a live agent for reading, identification, travel, etc. IBM says it is close to making the NavCog, indoor voice activated navigation app, available to the public. Toyota has been working on Project BLAID, an indoor navigation device, for some time, maybe this is the year we’ll try it. Google’s app “Lookout” for object recognition, is anticipated any day now.
AI (Artificial Intelligence) is powering much of the progress. Our digital assistants and smart speakers will mature and become more intuitive and dependable.
As I write this letter, Google, Alexa, and Siri are all sitting on my desk, ready to assist whenever I ask. They’re excellent researchers, spellers, fact checkers, appointment schedulers, alarm setters, time tellers, and they have instant access to massive amounts of music. Like all assistants, these 3 were not created equal, they all possess their own individual strengths and weaknesses. We’ll definitely talk more about that as the year moves forward.
One thing is for sure, we have only just begun to tap the potential of what technology can do to empower people living and working with vision loss. Stay with us, as we report on an exciting year, packed with possibility!
Thanks to all who attended our accessibility session on Apps, coming out despite the Arctic blast that hit with a relentless mix of icy rain and snow.The discussion was lively and always interesting and informative, exactly as we hoped it would be. People sharing their experience and learning from each other.
We acknowledged, upfront, that the topic of apps is massive and could be discussed for days. Apps that help us compensate for vision loss are available in many categories. We have lots of choices, and selecting the apps we integrate into daily life is a very individual process.
It was also delightful to note, that accessibility for people with vision loss is a priority, emphasized by tech leaders like Apple, Microsoft, Google, and Amazon.Inclusive technology is shifting our expectations and bridging the gaps.Although it’s not perfect…it’s really good.
Finding the apps that work for you will make life a little easier.
The recap that follows is a look at some specific apps and new developments we talked about at our “App Happy” session.It’s not only the apps with the bells and whistles that can change your life. Access to the basic built-in iPhone apps like Contacts, Camera, Messages, News, Clock, and Maps, can make a huge difference in your everyday.
Here are the apps that kicked off our “App Happy” discussion:
Seeing AI App (iOS Only)
This app from Microsoft has been around for just over a year and has already expanded its channels and improved its performance noticeably. Built for iOS exclusively, but rumored to be adding Android soon. Read more about Seeing AI in our article here.
9 Channels:
Short Text – Reads instantly
Document – Aligns and reads full page
Product – Barcode reader
Person – Facial recognition; fun but not sure how functional
Currency Reader – US and Canadian
Scene Reader – take a photo and it describes the scene and reads text
Color Reader – identifies color
Handwriting Reader – much improved for handwritten print or cursive
Light Indicator – changes tone to the direction of light
My Location – figure out where you are, direction facing, points of interest
Nearby Marker – locations you have marked
Around Me – what’s in the 4 surrounding quadrants
Ahead of Me – street names, buildings, businesses coming up
Aira App
A new assisted navigation service provided by trained agents viewing your location by live video.
Log in as a guest to try in fee-free locations like Walgreens, AT&T, Wegmans and a growing list of airports and businesses
Also available for a monthly subscription starting at $89. for 100 minutes
Be My Eyes App
This app provides live video access to a network of 1 million volunteers helping 80 thousand people with vision loss in 180 countries.
Average wait for a volunteer: 30 seconds
Get help with appliances, electronics, instructions, identification, etc.
Not recommended to share private information
Newly partnered with Microsoft Disability Desk
GalaPro App
The app making New York’s Broadway shows accessible to audience members with vision and hearing loss, from the privacy of their own mobile phone. Read more about GalaPro in our article here.
Audio descriptions syncs with live performance
An accessibility kiosk in every theater helps get you started
Also, offers closed captioning and language translation
More App Happy audience favorites:
Amazon App
Now with access to live Disability Hotline 888-283-1678 for shopping assistance.
Alexa App
For Book Reading, Shopping, Calendar, Music, Recipes, Games, Translation and more.
Amazon Prime Video App
Watch Prime Video on numerous devices with audio description.
Apple News App
The standard iOS app that provides access to a large selection of curated news from major outlets including New York Times, Wall Street Journal, Washington Post, Vanity Fair, Bloomberg, CNN, Food Network and more.
NFB Newsline App (iOS Only)
The National Federation of the Blind now offers an app for iOS that extends their newspaper reading service to over 300 publications, for eligible users.
ShazamApp
An entertaining app that identifies music, movies, TV, and ads, after hearing just a short snippet. Apple recently bought the technology and now offers it on iOS, Mac OS, TV OS, and Watch OS.
Drop us a note by email, or post a comment on our Facebook.
Now that the technology is universal, the awareness should be as well. In the following article, accessibility expert Dorrie Rush shows how inclusive technology has been a game-changer for people with vision loss. Original version published in the September 2018 issue of Retina Specialist Magazine.
Imagine what it is like to be shut out of a major event, then suddenly the doors open. The opportunity to be like everyone else, in this respect, is life changing.
For people with vision loss, the ability to perform daily functions had been limited by their access to bulky overhead readers and thick magnifiers, but those days are over. Thanks to the evolution of technology, the doors are opening for these individuals. They can now perform visual tasks inconspicuously.
This technology evolution has been happening for more than a decade, driven by inclusive design, universal access and the needs of an aging demographic. So now that the technology is universal, the awareness should be as well.
Here is a brief overview to bring the retina specialist up to speed on where we are and how we got here.
Apple Accessibility
In 2008, the fully accessible iPhone 3s came to market.
This was the first smartphone designed to accommodate the entire spectrum of vision loss. The accessibility settings on the iPhone included large text, zoom, invert colors and VoiceOver, a full-function screen reader. It was a pivotal moment: a popular consumer product, with built-in accessibility ready to go right out of the box at no extra cost.
Hand holding iPhone
As the mobile operating system developed, the visual accessibility tools developed with it. In many ways the regular features that evolved in the iPhone were a boon to people who are visually impaired. Dictation allows everyone to avoid typing on the keyboard by turning speech to text. Siri was our first experience with a digital assistant driven by artificial intelligence, which initiated an era unto itself.
In 2018, the iPhone is the most common denominator among people with vision loss. Although all smartphones now come off the shelf with accessibility requirements, Apple’s attention to inclusive design, technical and customer support is unparalleled.
Extra apps can add even more function.
Convert the camera to a high-definition magnifier, document scanner, product identifier or sign reader. Get walking directions from the digital assistant using Maps, find the right way with Compass, have the newspaper read aloud or dictate a shopping list into reminders. There is a flashlight always on hand, a book reader, transit tracker, banking, contacts and much more. Think of it as 100 low-vision devices in one.
The iPad, iPod Touch, Apple Watch, Apple TV and the Mac also come with the same standard functions for visual and non-visual access.
Microsoft
Microsoft is also committed to accessibility more than ever.
The Windows operating system has vastly improved its built-in accessibility options for people with visual impairments. The adjustments users need to make in terms of magnification, contrast and speech can be accomplished with a little help from their IT department.
For low-vision users, this largely eliminates what once was an inevitably awkward conversation with an employer requesting complicated, expensive and often unsupported assistive software. Requiring some adjustments at work to personalize visual settings is not at all unusual today.
Workspace with bright, open windows
Recently, Microsoft made another move to progress mobile accessibility, but not for its own product. The company built two groundbreaking apps for the iPhone and put them on the Apple App Store for free.
They are:
Seeing AI, accessed more than 1 million times in its first six months, offers multiple channels that identify short text, documents, people, products, handwriting and more. Some of the features are in development and keep improving. For low-vision users, the short text feature is a dream come true; just point the phone at text and it instantly starts reading.
Soundscape is billed as a “map with 3D sound.” This technology incorporates GPS to give visually impaired and blind users enhanced information about their surroundings. It’s a bit like strolling along with a friend who is telling you about your environment and calling out streets and intersections on your rout.
Accessibility Support
Apple continued to remove barriers by implementing a phone support line dedicated to accessibility, further empowering customers with visual, hearing, motor and learning impairments. It soon went from a limited number of hours each day to 24/7.
In relatively short order, Microsoft launched the Disability Answer Desk. Verizon Wireless and Comcast opened their own accessibility support centers. Amazon’s technical support specialists are well versed in accessibility. Google is reportedly preparing to roll out its own accessibility support phone service soon.
CVAA Pushes the Envelope
It would only be fair to call out the influence of the 21st Century Communications and Video Accessibility Act, signed in 2010 to update federal laws to require advanced communications products and services, including digital, broadband and mobile technologies, be accessible to people with disabilities.
The requirements, which were phased in over a period of years, address the accessibility of mobile browsers, descriptive video, on-screen menus and TV program guides.
Although the rules of the CVAA were neither strict nor specific, the technology makers of note seem to have stepped up and continue to deliver.
Amazon
Not always the bastion of accessibility, Amazon is undeniably the most improved in this category. All Kindle e-readers and Fire tablets offer an array of options to adjust text and display settings, or to use VoiceView, the screen reader for non-visual access. Magnification is available in the tablets, as is Alexa, the digital assistant.
Smart Speakers and Digital Assistants
The advent of artificial intelligence is driving interactivity into an entirely new realm.
No reading or typing required, just talking. Digital assistants in smartphones have quickly gained traction, particularly for people living with vision loss. Just ask for the news, weather, audiobooks, podcasts, time and timers. Or you can play games, order an Uber, calculate math or ask an endless number of interesting questions.
Perhaps Amazon’s greatest contribution to accessibility is the Echo speaker with Alexa.
Amazon Echo speaker
Smart speakers are already being adopted at a rate faster than smartphones, and they are accessible to everyone who has a voice. For the moment, the smart speaker business is dominated by Amazon, with Google Home and Apple’s Home Pod and others vying for a bigger piece of the action. Amazon, Google and Apple all have phone support available to help in the setup and use of their smart speakers.
Android
Phones and tablets with Android operating systems can be counted on for advanced visual accessibility, although not universally as well supported or user friendly as Apple’s offerings.
Recently Google announced an accessibility app in development for Android called Lookout. It’s designed to provide auditory cues about your surroundings and to read text. It will be available by year’s end on Google Play.
Accessible TV and Movies
The CVAA now requires cable television providers to supply voice-enabled on-screen menus and television guides to visually impaired customers. This accommodation can be achieved in a number of ways, including the use of apps and smart speakers.
Comcast offers the most comprehensive services. They include talking menus, voice search and a dedicated accessibility support center. Descriptive audio is also now available for moviegoers who are unable to clearly see the big picture. An embedded track is woven into the quiet spaces, describing the visual details of the film that might otherwise be missed.
Theaters are now required to have listening devices available upon request. Descriptive audio can also be accessed on mobile devices and smart TVs with a selection in settings.
While technology to aid the visually impaired has come far in the past few years, this is just the beginning. There is much more to come. Stay tuned and be aware.
We love technology that empowers… especially when it gives us back something vision loss has taken. Microsoft’s Seeing AI app is all that. It is an intelligent camera that reads text on the go and tells you who and what is around you.
Seeing AI is a sweet surprise of a package filled with stuff you’ll want. Get back to reading signs, documents, and products on the go.And get more detail about the people and scenes you’re not seeing clearly.
The interface is clean and nicely designed. It’s relatively easy to acclimate to, but the more you use it…the better you get. There is a Quick Help Guide in the upper right of each channel, with both text and a video tutorial.
Here’s a rundown of the Seeing AI channels…
Short Text – hold the camera over text and it reads, just like that. Use it on signs, price tags, tickets, mail, packages, etc. Start reading everything in and out of sight.
Documents – the camera guides you to photograph the entire page and reads back. Navigates headings in documents with VoiceOver, particularly useful in reading restaurant menus. (Similar to the KNFB Reader app.)
Product – a bar code reader with audio guidance. In the grocery store, for example, you’ll be able to identify the brand and flavor on the ice cream container, or the fat content of the milk, or if the bread is wheat or white.
Person – use the camera to scan and tell you about the people around. Facial descriptions include an estimate of age and expression. Set face recognition to announce the name of someone you know when they come into view.
Microsoft Seeing AI app describing a person sitting outside.
Scene – take a photo and hear the description of the scene.
The accuracy is very good, but not perfect. Like you, it is still learning. Help the process by sharing your thoughts and experience with the developers. There’s a link to send feedback directly from the Menu.
The Seeing Ai app is free, available in the App Store. This is by no means a small token, it is a meaningful gift. Not long ago, technology designed for people with visual impairments was available only at very hefty prices.
The best is yet to come. Microsoft says this is just a glimpse into artificial intelligence.
Please note that this article was not paid for, affiliated with, or endorsed by any third-party companies. The views and opinions expressed in this article are solely those of the author’s.
Originally published on August 11, 2017 and updated on August 3, 2018.
