Mobile Phone App for Recognising Places

GPS is a great way to work out your position outside, but it is not very reliable indoors. Nabeel Khan has developed a mobile phone app as part of his PhD at the University of Otago, that allows visually impaired users to take a photo of their current location and be told where in a particular building they are.

Currently, the app uses a database of about 5,000 photos which covers four or five buildings on campus. Each location has a signature, with prominent features being marked by things like stark contrasts in colour. The app then tries to match the photo taken with photos in the database, but there is a ranking system which means that less time is taken searching through photos which are definitely not a match. If the app can’t reliably tell where a location is from the photo, then it will say so.

Some other applications for the app could be in a museum or art gallery, so that signs would not need to be placed on the walls, instead visitors could take a photo and the app would give them information about the artwork. It could also be used in tourism spots, where GPS would still work, but information could be given about the specific site. Also, there may be health applications for people with dementia who may not remember why they were in a particular location, the app could provide suggestions based on a photo taken in a room. Ruth Beran meets Nabeel Khan’s PhD supervisor Brendan McCane (above right) and Steven Mills (above left) to see the app in action.