TY - GEN
T1 - Spatial Audio in 360° Videos
T2 - 13th ACM Multimedia Systems Conference, MMSys 2022
AU - Hirway, Amit
AU - Qiao, Yuansong
AU - Murray, Niall
N1 - Publisher Copyright:
© 2022 ACM.
PY - 2022/6/14
Y1 - 2022/6/14
N2 - Immersive technologies are rapidly gaining traction across a variety of application domains. 360° video is one such technology, which can be captured with an omnidirectional multi-camera arrangement. With a Virtual Reality (VR) Head Mounted Display (HMD), users have the freedom to look in any direction they wish within the scene. While there is a plethora of work focused on modeling visual attention (VA) in VR, little research has considered the influence of the audio modality on VA in VR. It is well known that audio has an important role in VR experiences. Listeners can experience sound in all directions with high quality spatial audio. One such technique, Ambisonics or 3D audio, provides a full 360° sound soundscape. In this paper, the results of an empirical study that examined how (if at all) spatial audio influences visual attention in 360° videos are presented. The same videos were accompanied with either non-spatial (stereo) or spatial (third order Ambisonics) sound. Pose and gaze fixations, pupil diameter, audio energy maps (AEMs) and associated analysis for 20 users watching ten 360° videos across various categories (in Indoor and Outdoor conditions) are presented. The findings reveal that users have different viewing patterns and physiological responses for the different sound conditions. With the increasing use of spatial audio in VR and 360° videos, this knowledge can help develop effective techniques for optimizations in terms of processing, encoding, distributing, and rendering content.
AB - Immersive technologies are rapidly gaining traction across a variety of application domains. 360° video is one such technology, which can be captured with an omnidirectional multi-camera arrangement. With a Virtual Reality (VR) Head Mounted Display (HMD), users have the freedom to look in any direction they wish within the scene. While there is a plethora of work focused on modeling visual attention (VA) in VR, little research has considered the influence of the audio modality on VA in VR. It is well known that audio has an important role in VR experiences. Listeners can experience sound in all directions with high quality spatial audio. One such technique, Ambisonics or 3D audio, provides a full 360° sound soundscape. In this paper, the results of an empirical study that examined how (if at all) spatial audio influences visual attention in 360° videos are presented. The same videos were accompanied with either non-spatial (stereo) or spatial (third order Ambisonics) sound. Pose and gaze fixations, pupil diameter, audio energy maps (AEMs) and associated analysis for 20 users watching ten 360° videos across various categories (in Indoor and Outdoor conditions) are presented. The findings reveal that users have different viewing patterns and physiological responses for the different sound conditions. With the increasing use of spatial audio in VR and 360° videos, this knowledge can help develop effective techniques for optimizations in terms of processing, encoding, distributing, and rendering content.
KW - 360° Video
KW - Ambisonics
KW - Audio-Visual Attention
KW - Fixations
KW - Saliency Maps
KW - Spatial Audio
KW - Virtual Reality
UR - http://www.scopus.com/inward/record.url?scp=85137146455&partnerID=8YFLogxK
U2 - 10.1145/3524273.3528179
DO - 10.1145/3524273.3528179
M3 - Conference contribution
AN - SCOPUS:85137146455
SN - 9781450392839
SN - 9781450392839
T3 - MMSys 2022 - Proceedings of the 13th ACM Multimedia Systems Conference
SP - 39
EP - 51
BT - MMSys 2022 - Proceedings of the 13th ACM Multimedia Systems Conference
PB - Association for Computing Machinery, Inc
Y2 - 14 June 2022 through 17 June 2022
ER -