Leveraging Surface Acoustic Waves for Robust Hand-to-Surface Gestures

OVERVIEW

MEMS microphone to identify hand-to-surface touch interaction events

• Differentiates between hover and touch events; unobtrusive and robust 
  
• Applies to augmented reality, smart surfaces, human-machine interaction, touch-sensitive furniture 
  

BACKGROUND

Current methods for identifying hand-to-surface interactions generally fall into two categories: direct user instrumentation and vision-based detection. Direct user instrumentation, such as wearable sensors, is burdensome and cumbersome. Vision-based approaches, while contactless, often struggle to differentiate between hovering and actual touch events. Historically, surface interaction detection has been a challenge in noisy environments, further limiting the accuracy and applicability of these technologies. The necessity for a technology that can accurately interpret hand-to-surface interactions is particularly critical as augmented reality (AR) and other user interface technologies become increasingly prevalent. The shortcomings of inaccuracies and bulkiness in existing methods underscore the need for an innovative and unobtrusive solution that can seamlessly integrate with everyday objects.

INNOVATION

Researchers have developed a new approach that leverages a single MEMS contact microphone to identify unique Surface Acoustic Waves (SAWs) generated by finger touches on various materials such as wood, glass, metal, and particleboard. These waves, captured even over long distances and in noisy environments, enable the system to recognize six distinct gestures—including taps, swipes, and flings—with an impressive 98.6% accuracy. This innovative method significantly improves user interaction in AR applications compared to traditional computer vision alone. The technology's small footprint, cost-effectiveness, and robust performance make it highly adaptable for integration into any surface, laying the groundwork for a broad range of applications in smart surfaces, human-machine interactions, and touch-sensitive furniture.