Exploration Strategies Influence​ Visual-Auditory Sensory Substitution in a Passive Guidance Setup

Overview & Synopsis

Sensory substitution devices (SSDs) assist or replace sensory functions by engaging other modalities such as audition or touch (Auvray, 2006). They allow users, especially those with visual impairments, to perceive their environment differently (Bach-y-Rita et al., 1969). These devices also provide valuable insights into perception, highlighting the crucial role of action within the perceptual process. However, the precise role of action in perception remains debated (Froese & Ortiz-Garin, 2020) and the importance of exploration strategies has been emphasized (Bermejo et al., 2015). We conducted two studies on the role of action and exploration strategies in 2D shape perception using an SSD coupled with a pantograph, a mechanical system that transfers arm movements from one participant to another in real time. In the first study, active participants voluntarily controlled the sensor, while passive participants were guided by their active partners through the pantograph. Results showed that both active and passive participants improved their shape perception (p < .001), with no significant difference between the groups across three sessions of 10 trials (20 per condition). This suggests that self-initiated action is not essential for shape perception. Additionally, no differences were observed in a fourth session, where all participants acted independently. However, a significant positive correlation between the performances of active and passive participants (p < .05) suggests that the exploration strategies used by active participants in sessions 1-3 influenced the performance of passive participants once they became active in session 4 (Akar et al., in revision). In a subsequent study, we examined the effect of exploration strategies on perceptual performance in passive conditions. The experimenter assumed the active role, using four distinct guiding strategies, with passive participants grouped by strategy (n = 12 per group). Results showed that participants guided with a micro-scanning strategy (oscillatory movements along the shape's contour) achieved significantly higher perceptual accuracy and confidence (p < .05). Our findings confirm the feasibility of passive guidance and highlight the critical role of exploration strategies in sensory substitution. These insights could inform the design of assistive technologies and rehabilitation programs.