Tactile Emotions: Multimodal Affective Captioning with Haptics Improves Narrative Engagement for d/Deaf and Hard-of-Hearing Viewers
Figure 1: Multimodal afective captions, combining visual cues and vibrations felt via a wrist-worn device, enrich the viewing experience for d/Deaf or Hard-of-Hearing individuals by portraying speaker emotions, improving engagement.
Authors
- Roshan Lalintha Peiris (ORCID: https://orcid.org/0000-0002-4191-3565)
- Matt Huenerfauth (ORCID: https://orcid.org/0000-0001-6290-2681)
- Caluã de Lacerda Pataca (ORCID: https://orcid.org/0000-0001-5046-9884)
- Michelle L. Olson (ORCID: https://orcid.org/0009-0009-8297-1743)
- Lloyd May (ORCID: https://orcid.org/0000-0003-4692-8261)
- Saad Hassan (ORCID: https://orcid.org/0000-0003-2792-2964)
- Toni D'aurio
Institutions
- Tulane University (US)
- Rochester Institute of Technology (US)
- Stanford University (US)
Publication Details
- Published
- 2025-04-25
- DOI
- https://doi.org/10.1145/3706598.3713304
- Citations
- 4
- Primary Topic
- Subtitles and Audiovisual Media
- Type
- article
- Field-Weighted Citation Impact
- 41.69
Funders
- National Science Foundation
- U.S. Department of Health and Human Services
- US-UK Fulbright Commission