Storytelling tool Journey

A trip down memory lane

A trip down memory lane

Individualized storytelling in reminiscence therapy
Thanks to its eye tracking technology, our experimental tool Journey allows for guided nonlinear storytelling in reminiscence therapy. Journey is geared towards caregivers for people with depression or dementia and aims to revive positive memories through stimulating material: Audio-visual stories can be personalized to meet the interests of the patients even if they are unable to communicate them because of their disease.


Positive, autobiographical memories are especially important for people with dementia or depression. In order to revive such memories, reminiscence therapy makes use of visual and acoustic stimulants. The better the stimulating material matches the patients’ interests and experiences, the more efficient the therapy is. However, some of the patients are no longer able to clearly communicate their interests. This is why, up to now, memory maintenance and biographical work have been rather passive approaches with little interaction between the patient and the caregivers or therapists, who do not receive any feedback on whether the stimulants used are interesting for the patients or on what visual elements are of particular interest to them. With Journey, therapists and caregivers now have an inexpensive, easy-to-use tool at hand to improve this process.

Eye tracking technology allows the software to identify picture elements that the patients pay particular attention to. On a second screen, e. g. an iPad, the carers see corresponding heat maps, which give them indications of the interests to focus on in the ongoing storytelling process.

The patient may, for example, look at a picture showing a farm with a tractor, a barking dog and a playing child. If the person reacts positively to the dog, the accompanying sound intensifies. In addition, more dog pictures, possibly even pictures from the patient’s own life, can be displayed. Using the second-screen app, therapists can navigate through contents related to dogs.

Easy to use and inexpensive for everyone

Journey is meant to be easily accessible for therapists and caregivers. For this reason we are using cost-efficient hardware that is readily available. The software is based on the open source framework OpenFace.

A standard webcam is used for the face tracking feature. Analyzing head and pupil movements, it identifies the picture elements that the patient pays particular attention to. The heat maps are displayed in a web-based application on a second screen in real time.

A convincing idea

Journey was created within just 48 hours during the "Hack4Aging Well" hackathon 2017 in Coimbra. The idea earned our team the first prize in the hackathon for the second time. We are currently enhancing the prototype and plan to implement further sensors in a next step to measure electrodermal activity or the pulse. This will allow us to improve the therapy by combining emotional valence and eye tracking.


Martin Ecker, Henrik Rieß, Merlin Schuster



Wilhelm-Bleyle-Straße 10
71636 Ludwigsburg