Spatial Audio in 360° Videos: Does it influence Visual Attention?

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

8 Citations (Scopus)

Abstract

Immersive technologies are rapidly gaining traction across a variety of application domains. 360° video is one such technology, which can be captured with an omnidirectional multi-camera arrangement. With a Virtual Reality (VR) Head Mounted Display (HMD), users have the freedom to look in any direction they wish within the scene. While there is a plethora of work focused on modeling visual attention (VA) in VR, little research has considered the influence of the audio modality on VA in VR. It is well known that audio has an important role in VR experiences. Listeners can experience sound in all directions with high quality spatial audio. One such technique, Ambisonics or 3D audio, provides a full 360° sound soundscape. In this paper, the results of an empirical study that examined how (if at all) spatial audio influences visual attention in 360° videos are presented. The same videos were accompanied with either non-spatial (stereo) or spatial (third order Ambisonics) sound. Pose and gaze fixations, pupil diameter, audio energy maps (AEMs) and associated analysis for 20 users watching ten 360° videos across various categories (in Indoor and Outdoor conditions) are presented. The findings reveal that users have different viewing patterns and physiological responses for the different sound conditions. With the increasing use of spatial audio in VR and 360° videos, this knowledge can help develop effective techniques for optimizations in terms of processing, encoding, distributing, and rendering content.

Original languageEnglish
Title of host publicationMMSys 2022 - Proceedings of the 13th ACM Multimedia Systems Conference
PublisherAssociation for Computing Machinery, Inc
Pages39-51
Number of pages13
ISBN (Electronic)9781450392839
ISBN (Print)9781450392839, 9781450392839
DOIs
Publication statusPublished - 14 Jun 2022
Event13th ACM Multimedia Systems Conference, MMSys 2022 - Athlone, Ireland
Duration: 14 Jun 202217 Jun 2022

Publication series

NameMMSys 2022 - Proceedings of the 13th ACM Multimedia Systems Conference

Conference

Conference13th ACM Multimedia Systems Conference, MMSys 2022
Country/TerritoryIreland
CityAthlone
Period14/06/2217/06/22

Keywords

  • 360° Video
  • Ambisonics
  • Audio-Visual Attention
  • Fixations
  • Saliency Maps
  • Spatial Audio
  • Virtual Reality

Fingerprint

Dive into the research topics of 'Spatial Audio in 360° Videos: Does it influence Visual Attention?'. Together they form a unique fingerprint.

Cite this