16/11/2017 5:24 AM AEDT | Updated 18/11/2017 8:50 AM AEDT

Artificial Intelligence Is Helping Blind People To ‘See’

Microsoft has launched a new app that describes everything around you, in real time.

Fairfax Media
The free Seeing AI app will be life changing for vision impaired people as it can describe anything you point your smartphone at.

There are more than 350,000 vision impaired people in Australia and the exciting news is that advances in Artificial Intelligence (AI) are beginning to make their lives much richer.

The Seeing AI app, released this week by Microsoft, is poised to be a completely life-changing app for people who are vision impaired. It's a smartphone-based narrator that can be used anywhere: people can point their smartphone at the pantry to find the vegemite, or use it to narrate a child's homework questions.

Dave Heiner, from Microsoft, told HuffPost Australia Seeing AI started as a hack at a recent hackathon, where a group of people decided to build an app that combined a smartphone and AI vision technology.

"They realised they can hack those things together and ultimately try to give vision to the visually impaired. The Seeing AI means you can hold your phone up to a person and it can tell me what that person looks like, what their age is, the colour of their hair, whether they're wearing glasses, whether they look happy or sad," Heiner said.

"It does a whole set of AI capabilities around facial recognition. You can point it at a product and it will read the bar code and tell you what the product is, you can point it at a sign in a hallway or airport and it will read the text back instantly."

"It's just an unbelievably magical application for the visually impaired community and for anyone that wants to see the benefits of AI."

Kenny Johar Singh initially hoped to be a doctor until a degenerative retinal condition changed his career plans. But, a lack of vision wasn't going to limit his aspirations.

"I took a pivot into computers because they were the mechanisms I was using to access information. As I was losing my sight, computers were quite an empowering force, so I thought 'let's make a career out of this,'" Singh said.

Singh uses a range of technologies to allow him equal access to information as his fully sighted colleagues. And, as an early adopter of Seeing AI, Singh uses the app to give him access to the physical world.

"I use it a lot to identify objects products and to read documents if I get them in a physical printed format. And I'm experimenting with the person identification capability."

Fairfax Media
The Seeing AI app allows data to be read, simply by pointing your smartphone at a document.

Seeing AI can read documents, and make sense of structural elements such as headings, paragraphs and lists so that people can quickly navigate a document using voiceover. It can also recognise images in other apps.

Microsoft's work in AI means it now has solutions that can recognise speech with an error rate of just 5.9 percent and images with an error rate of only 3.5 percent. This means computers can recognise words in a conversation and provide real-time translation.

ANU law lecturer Cameron Roles is excited about the new apps to help vision impaired people lead rich, full lives.

Roles has been blind since birth; one of a set of triplets born three months early, the oxygen that saved his life caused his blindness. He said assistive technologies will empower people to more easily access important content, while expanding the workforce to include a larger base of talented people.

"Now is definitely, in my view, the most exciting time in human history to be blind," Roles said.

More On This Topic