Aims: Mental illnesses such as anxiety disorder, obsessive-compulsive disorder and intrusive thoughts affect millions worldwide and have become increasingly visible in recent decades, particularly in young adults. Despite greater public awareness and discussion, significant stigma remains, undermining the self-confidence and well-being of those affected. Anti-stigmatization interventions can help reduce prejudice by promoting education and meaningful contact between people with and without mental illness. Technology-based interventions may further support these efforts by simulating such contact through approaches such as virtual perspective-taking or storytelling. In this context, social robots, as physically embodied agents with social presence and multimodal communication capabilities, may offer additional advantages for storytelling-based interventions. Especially the integration of music and sound effects may benefit the outcomes. To this end, we evaluated a robotic storyteller as an intervention method focusing on the influence of additional sound integration.

Methods: In multimodal robotic storytelling, modalities such as voice modulation and bodily expressions have been extensively studied, while non-speech sounds, namely sound effects and background music, remain largely overlooked, despite their importance in related media such as audio books and films. To address this gap, we compared a robotic storyteller narrating a story about a person experiencing a panic attack and intrusive thoughts using only voice and bodily expression with versions integrating sound effects, background music, or both. A laboratory study examined how these modalities affected prejudice, empathy, and narrative transportation, using questionnaires and behavioral observation.

Results: The comparison yielded mixed results. While the addition of diverse combinations of non-speech sounds did not affect story recipients’ prejudice differently than a robotic storytelling without additional sounds, adding both sound effects and background music led to increased transportation into the story as well as improved associative empathy. In contrast, the addition of music only decreased associative empathy. Although mediation was not indicated, they revealed transportation as a predictor for both empathy and prejudice, while being only minimally influenced by sound integration itself.

Conclusion: The effects of music and/or sound effect integration to a robotic storytelling intervention on recipients’ prejudice were mixed, recommending either the combination of both sound types or complete omission. Furthermore, transportation was indicated as an important key lever for increasing empathy and decreasing prejudice in a robotic storytelling intervention that warrants further investigation. Thus, future work is needed to gain deeper insights into robotic storytelling as an intervention tool for reducing prejudice and stigmata, including work on increasing transportation as a modifiable factor as well as the integration of pre-post-measurements.