ARGUS: A smart shopping assistant for blind and visually impaired people

Non-accessible supermarkets pose a number of challenges for blind and visually impaired people: First of all, they have to orientate themselves in the aisles. Navigation is made even more difficult by the frequent changes in the sales floor, such as temporarily placed special offers. Once the right shelf is found, product recognition is often difficult because packages are too similar in material and shape. The hurdles for informing oneself and comparing products are accordingly high, and this applies all the more to independence in everyday life and the fun of shopping.

In the ARGUS project, Hochschule Bonn-Rhein-Sieg is working on a smart digital agent to support blind people in finding desired products and obtaining information about them. Integrated into everyday devices such as smartphones, this assistant is to be controlled via speech and an app and provide consumers with individual support in the shop. Artificial intelligence plays a key role here: as a learning assistant, the digital companion is to recognise products on the shelf and get to know its human and his or her preferences better and better in order to discuss taste and quality with him or her.
For the project team led by Professor Alexander Boden and Margarita Esau-Held, it is important to involve future users in the development of the digital shopping companion from the very beginning. For example, they conducted interviews with the target group to learn about their everyday practices and requirements. In order to experience problems described in everyday life, they accompanied blind people while shopping. One study participant describes her shopping experiences like this:

"For me, shopping is one of the biggest stressful situations. I would perhaps much rather go shopping because of the assistant, because it somehow takes the stress away from me and I don't have to make an effort to find anything. I could also help myself in a certain way. That's also important to me, that I can do as much as I can myself.

During our inception study, we became really aware again of the abundance of visual information and similar packaging that consumers are exposed to," says Margarita Esau-Held, project manager at H-BRS. "Therefore, the key challenge is to translate information such as notices, explanations, product features, colours, materials, which are all visually available at the same time, auditorily. And in doing so, to design a temporal sequence of information that meets people's diverse information needs."