Communications and Marketing

ARGUS: A smart shopping assistant for blind and visually impaired people

Supermarkt Symbolbild COLOURBOX34875466

Thursday 12 October 2023

Grocery shopping in the supermarket is anything but easy for blind and visually impaired people. Supermarkets are confusing, the products sometimes hard to find and difficult to distinguish from each other because of the packaging. Browsing, comparing prices and trying things out therefore often fall by the wayside. Hochschule Bonn-Rhein-Sieg (H-BRS) is working with project partners on a digital shopping companion to support blind and visually impaired people in their shopping.
Supermarkt Symbolbild COLOURBOX34875466
Orientation in the supermarket and the abundance of very similar packaging are a challenge for blind people. Photo (symbol image): Colourbox

Non-accessible supermarkets pose a number of challenges for blind and visually impaired people: First of all, they have to orientate themselves in the aisles. Navigation is made even more difficult by the frequent changes in the sales floor, such as temporarily placed special offers. Once the right shelf is found, product recognition is often difficult because packages are too similar in material and shape. The hurdles for informing oneself and comparing products are accordingly high, and this applies all the more to independence in everyday life and the fun of shopping.

In the ARGUS project, Hochschule Bonn-Rhein-Sieg is working on a smart digital agent to support blind people in finding desired products and obtaining information about them. Integrated into everyday devices such as smartphones, this assistant is to be controlled via speech and an app and provide consumers with individual support in the shop. Artificial intelligence plays a key role here: as a learning assistant, the digital companion is to recognise products on the shelf and get to know its human and his or her preferences better and better in order to discuss taste and quality with him or her.
For the project team led by Professor Alexander Boden and Margarita Esau-Held, it is important to involve future users in the development of the digital shopping companion from the very beginning. For example, they conducted interviews with the target group to learn about their everyday practices and requirements. In order to experience problems described in everyday life, they accompanied blind people while shopping. One study participant describes her shopping experiences like this:

"For me, shopping is one of the biggest stressful situations. I would perhaps much rather go shopping because of the assistant, because it somehow takes the stress away from me and I don't have to make an effort to find anything. I could also help myself in a certain way. That's also important to me, that I can do as much as I can myself.

During our inception study, we became really aware again of the abundance of visual information and similar packaging that consumers are exposed to," says Margarita Esau-Held, project manager at H-BRS. "Therefore, the key challenge is to translate information such as notices, explanations, product features, colours, materials, which are all visually available at the same time, auditorily. And in doing so, to design a temporal sequence of information that meets people's diverse information needs."

wiwi institut fuer verbraucherinformatik ivi sta 20220523 foto eric lichtenscheidt 04 fs22.jpg
The Institute for Digital Consumption combines consumer sciences and business informatics. It deals with the digitalisation of consumer fields and consumer behaviour. Photo: Eric Lichtenscheidt

In developing the digital shopping assistant, H-BRS and its partners in the project are relying on the increasing popularity of voice assistants in everyday life. However, the new type of shopping assistant will be specially tailored to the needs and skills of non-sighted people and thus go far beyond known voice assistants.

H-BRS is working on the ARGUS project with the start-ups SonicView and VAGO solutions and Petanux. The project has a duration of three years and is funded by the Federal Ministry of Education and Research. Within the H-BRS, the Institute for Digital Consumption (IVI) is responsible for the project. Based at the Sankt Augustin Campus, the IVI has extensive research experience in the fields of digitalisation of consumer fields, human-computer interaction and artificial intelligence. It also has extensive expertise in data management along the food supply chain.

The findings from the discussions and observations will be incorporated into the smart assistant, which will take shape more and more in the coming months. The exchange with those affected has already produced one important insight: the one solution that works equally well for all blind and visually impaired people will probably not exist - the needs are too different for that. On the other hand, the new shopping companion has the potential to be adapted for user groups other than people with visual impairments and to make shopping an informative and interactive experience.

Institute for Digital Consumption (IVI) at H-BRS

Kontakt

fb01_margarita_esau-held

Margarita Esau-Held

Research associate

Location

Sankt Augustin

Room

G135

Address

Grantham-Allee 20

53757, Sankt Augustin

Telephone

+49 2241 865
portraet_martin_schulz_kum_20210609_cut_web.jpg (DE)

Martin Schulz

Science editor

Location

Sankt Augustin

Room

E 240

Address

Grantham-Allee 20

53757, Sankt Augustin

Telephone

+49 2241 865 9560