Sound-driven object recognition fostering semantic and spatial navigation for visually impaired individuals

Abstract

In an increasingly visually dominated world, the significance of alternative sensory modalities in shaping our experiences cannot be left behind. The use of senses beyond sight, such as hearing, can enrich and advance the existing frameworks in audio-based navigation for individuals with visual impairment. Navigating the world around us is crucial for individuals with visual impairment and also for individuals with other cognitive and physical disabilities, becoming one of their biggest challenges In this work, we overview how the sounds from our everyday lives can impact the way we see things, create mental pictures and how they shape our actions. In this scenario, human-computer interfaces to the natural world are inherently multisensory. Currently, many studies in this field encounter problems regarding using sounds to convey spatial information, in other words; sonification. However, despite its recent interest and fast development, the full potential of sonification and the multidimensionality of sound is only starting to be explored, particularly in contexts with multiple dimensions and live variables. In this work, we will go through the design of an online sound perceptual test to assess people's perception of different sounds and how they intuitively associate them with various objects in our daily surroundings. This test will be the starting point for the design of a sonification technique, which will be tested with a group of 10 participants in a simulated environment. They will need to fulfil different tasks related to autonomous navigation and object recognition based on the sound cues and the sonification technique.