Perceptual integration of multisensory haptic, visual, and auditory feedback for roughness discrimination in augmented reality

Abstract

Understanding how our different senses interact to shape our perception is essential to design realistic and immersive virtual and augmented reality (VR/AR) experiences. The present study investigated how roughness perception can be modulated through haptic, visual, and auditory cues in AR using a vibrotactile wristband. Participants compared virtual textures varying in vibration frequency/amplitude, visual grain size, and friction sound. Results revealed strong linear relationships between stimulus parameters and perceived roughness, with haptic frequency and visual cues driving the highest discrimination performance. Adding non-informative sensory feedback reduced perceptual sensitivity, acting as noise. Individual differences emerged: participants who rated haptic as the easiest modality showed greater sensitivity to haptic variations, while visual-reliant participants performed better with visual cues. We conclude that roughness in AR can be systematically manipulated, but is vulnerable to perceptual interference from irrelevant inputs, where our work provides actionable insights for implementing optimized and adaptive AR/VR interfaces.