AI for accessibility: vision, hearing, and mobility tools
AI is doing some of its most genuinely meaningful work in accessibility. For people with vision, hearing, mobility, or cognitive challenges, AI has quietly made everyday tasks easier in ways that were science fiction five years ago. Most of these tools are free and built into devices you already own.
The most useful tools to know
- Vision: Be My Eyes (free), Seeing AI (free), iPhone Magnifier (built in)
- Hearing: Live Captions (iPhone/Android built in), Live Transcribe (free)
- Mobility: Voice Control (built in), Dictation (built in), Eye Tracking (iPhone)
- Cognitive: ChatGPT for explaining complex things, breaking tasks into steps
For people with vision impairment or blindness
Be My Eyes + Be My AI
Free app on iPhone and Android. Originally connected blind users with sighted volunteers via video call. Now also has "Be My AI" which uses AI to describe what your camera sees in detail.
- Point your camera at anything; AI describes what it sees.
- "Read my mail." "What's in front of me?" "What does this label say?"
- Free, no subscription.
- Built by Be My Eyes with OpenAI's vision model.
Microsoft Seeing AI
Free iPhone app. Identifies:
- Short text (read aloud as you point)
- Documents (full pages, structured)
- Products (scans barcodes)
- People (with their names if you've taught it)
- Currency (US dollars by denomination)
- Light levels (helps with photography)
- Colors
One of the most useful free accessibility apps on iPhone.
Google Lookout (Android equivalent)
Similar feature set for Android phones. Free from Google.
iPhone Magnifier app
Built into iPhone. Acts as a digital magnifying glass with:
- Zoom into anything with your camera
- Read text aloud
- People detection (for navigating sidewalks)
- Door detection (identifies signs and door handles)
Settings > Accessibility > Magnifier to enable.
VoiceOver and TalkBack
Screen readers built into iPhone (VoiceOver) and Android (TalkBack). Reads everything on screen aloud and lets you navigate by touch. Settings > Accessibility > VoiceOver.
iOS 18+ added Apple Intelligence-powered improvements making VoiceOver descriptions more natural and detailed.
For people with hearing impairment
Live Captions on iPhone
Captions any audio playing on your iPhone in real time:
- Settings > Accessibility > Live Captions.
- Turn on.
- A caption box appears whenever any audio plays. You can move and resize it.
Works for video calls, podcasts, voice memos, conversations through your phone's microphone.
Live Caption on Android
Similar built-in feature. Settings > Accessibility > Live Caption. Works on any audio playing on the device.
Google Live Transcribe
Free Android app that captions live conversation around you. Hold up your phone in a restaurant or meeting and the conversation appears as captions.
Otter.ai
Free tier transcribes meetings (Zoom, Google Meet, Teams) in real time and saves a searchable transcript. otter.ai.
AirPods Pro hearing features
AirPods Pro 2nd gen and later include Conversation Boost (amplifies the person in front of you) and a hearing aid feature for mild-to-moderate hearing loss. Not a replacement for medical-grade hearing aids, but a real help in many situations.
Captioned phone services
CapTel and similar services provide captioned phones (the phone displays text as the other person speaks). Often free for people with documented hearing loss; check your state's program.
For people with limited hand use or mobility
Voice Control on iPhone or Mac
Operates the entire device by voice. Settings > Accessibility > Voice Control.
- "Open Mail." "Tap Compose." "Type [text]."
- "Scroll down." "Go back." "Swipe right."
- "Show numbers" to see numbered overlays for tapping.
Setup is fast and the feature is dramatically more capable than people expect.
Voice Access on Android
Same idea on Android. Settings > Accessibility > Voice Access.
Dictation everywhere
Press the microphone button in any text field (iPhone, Android, Mac, Windows). Speak; text appears. Modern AI dictation handles punctuation, paragraphs, and corrections.
Eye tracking on iPhone
iPhones from 2024 onward support eye tracking. The phone tracks where you look and uses eye movements to control the interface. Settings > Accessibility > Eye Tracking. Useful for people with very limited hand or arm use.
ChatGPT and Claude voice mode
Hands-free, full-conversation AI. For anyone who finds typing slow, painful, or impossible. Open the ChatGPT app, tap headphones icon. See our ChatGPT on phone guide.
Smart home control
Alexa, Google Home, and Siri let you control lights, locks, thermostats, and TVs by voice. For someone with limited mobility, this can mean independence.
For people with cognitive or learning challenges
Break tasks into steps
"Help me make a step-by-step list for paying my electric bill online. I'm not tech-savvy. Each step should be very specific."
Explain complex things simply
"Explain my new lease in plain English. List anything I should pay extra attention to."
Reminders and structure
Voice assistants for medication reminders ("Alexa, remind me to take my blood pressure pill at 8am"). Combine with ChatGPT for the "what does this mean" half of things.
Reading help
Apps like Speechify and NaturalReader use AI voices to read books, web pages, PDFs aloud. Helpful for dyslexia or fatigue from reading.
iPhone accessibility features powered by AI (turn on in Settings)
Settings > Accessibility for all of these:
- Personal Voice: create a synthetic version of your own voice for people who may lose theirs.
- Live Speech: type and have the iPhone speak for you.
- Magnifier: visual aid (covered above).
- Sound Recognition: alerts you visually when doorbell rings, baby cries, smoke alarm sounds.
- Vehicle Motion Cues: reduces motion sickness when reading in cars.
- Assistive Access: simplified interface for people with cognitive challenges.
Android accessibility features
Settings > Accessibility:
- Live Caption
- TalkBack screen reader
- Sound Notifications
- Lookout (visual identification)
- Voice Access
- Magnification
- Color correction and inversion
For families and caregivers
If you're helping someone use accessibility features:
- Don't assume they want every tool turned on. Ask what would help.
- Practice the tool with them, not for them. Hands-on time matters.
- Build phone shortcuts: many accessibility features can be triggered with triple-click on the side button.
- Print a simple cheat sheet of how to use the features they care about.
- Ask your local Center for Independent Living for in-person help. They often know specific local resources.
The compound effect of AI accessibility
Five years ago, helping a parent with macular degeneration read mail required you being there. Now they pull out their phone, Be My AI describes the letter, and they handle it themselves. That's not just convenience; it's dignity and independence.
Same for someone with hearing loss who can now follow restaurant conversations with live captions, or someone with arthritis who runs their entire computer by voice. AI is removing barriers that used to define daily life.
Local resources in Santa Cruz County
- Central Coast Center for Independent Living: 831-462-8720
- Hearing Loss Association of America Santa Cruz chapter
- Vista Center for the Blind and Visually Impaired
- Santa Cruz Library has assistive technology for in-library use
Verify current phone numbers; resources change.
5 things to try today
- Turn on Live Captions on your phone (Accessibility > Live Captions). Try it with a video.
- Download Seeing AI or Be My Eyes. Try reading a piece of mail.
- Turn on Voice Control on iPhone. Try one task without touching the screen.
- Open ChatGPT voice mode. Have a 2-minute conversation hands-free.
- Ask AI: "What accessibility features should I be using on my iPhone for [your specific challenge]?"
Want help setting up accessibility tools?
Many of these features are powerful but hidden in Settings menus. Isaac can come out (or do screen share) to set them up properly for you or a family member.