SoundModVR: Sound Modifications in Virtual Reality to Support People who are Deaf and Hard of Hearing

OVERVIEW

Interactive VR sound tools for DHH users to enhance custom audio accessibility

• Allows DHH users to modify sound, not just substitute audio with visuals/haptics
• Immersive gaming, VR training, virtual education, accessible entertainment environments

BACKGROUND

Virtual reality applications have rapidly grown, offering immersive environments for entertainment, education, and other fields. However, these experiences often exclude deaf and hard of hearing (DHH) users due to reliance on audio cues. Traditionally, accessibility efforts have focused on substituting these cues with visual or haptic feedback, helping those with profound hearing loss but neglecting the broad spectrum of hearing abilities within the DHH community. Many individuals retain some hearing capability and could benefit from increased control over VR audio, but existing solutions provide little customizability, leading to frustration and reduced engagement. This gap highlights the need for more nuanced approaches that enable users to modify rather than just replace audio, making VR environments more flexible and personalized for people with diverse hearing experiences.

INNOVATION

This work introduces a set of 18 sound modification tools for VR, designed explicitly for DHH users' varied needs. The tools span categories including sound prioritization, real-time parameter adjustments, spatial audio assistance, and the introduction of helpful additional sounds. Through user studies with DHH participants across diverse VR scenarios, the toolkit demonstrated significant potential to enhance engagement and customizability while reducing exclusion. Furthermore, a Unity-based toolkit was provided to VR developers, who found it intuitive and easy to debug, though further modularization and documentation could improve adoption. Potential real-world applications include more inclusive VR games and simulations, accessible educational modules, and adaptable virtual social or workspaces, all supporting richer and more personalized audio experiences for DHH users.

ADDITIONAL INFORMATION

PROJECT LINKS:

DEPARTMENT/LAB:

• Dhruv Jain, Computer Science and Engineering

LICENSE: