TY - JOUR
T1 - Perceptually congruent sonification of auditory line charts
AU - Fitzpatrick, Joe
AU - Neff, Flaithri
N1 - Publisher Copyright:
© 2023, The Author(s), under exclusive licence to Springer Nature Switzerland AG.
PY - 2023/12
Y1 - 2023/12
N2 - In recent years, there has been increasingly more discussion about sonification design, evaluation strategies, and the future of sonification in general. Concerns relate to the efficacy and usability of sonification methodologies and the lack of evidence supporting design choices. In sonification publications to date, there is little discussion on how the principles of psychoacoustics or auditory scene analysis (ASA) might be used to improve design methodologies. Proposed in this article is the perceptually congruent sonification (PerCS) model that highlights the links between sonification mappings and key perceptual factors that are most likely to affect the interpretability of sonified trends. The model explores the perceptual constraints associated with design choices such as presentation rate, tonality, space, and frequency range. The model highlights the dependencies these mappings have on our perception of frequency, timbre, loudness, and space: the core determinants of stream segregation (a key phenomenon of ASA). This model-centred approach is used to identify perceptually congruent design choices for auditory line charts, a popular and accessible form of parameter-mapping sonification. This article focuses predominately on identifying an efficient and perceptually congruent frequency range for auditory line charts, as there are no contemporary studies on this design choice, with auditory graphing tools leaving little guidance on what might be appropriate. Several psychoacoustically justified ranges are proposed by the PerCS research and are contrasted in a user study to determine the performance of narrow vs wide frequency ranges. Results show that a shorter range between 400 and 1400 Hz performs just as well as a wider range between 65 Hz and 1480 Hz.
AB - In recent years, there has been increasingly more discussion about sonification design, evaluation strategies, and the future of sonification in general. Concerns relate to the efficacy and usability of sonification methodologies and the lack of evidence supporting design choices. In sonification publications to date, there is little discussion on how the principles of psychoacoustics or auditory scene analysis (ASA) might be used to improve design methodologies. Proposed in this article is the perceptually congruent sonification (PerCS) model that highlights the links between sonification mappings and key perceptual factors that are most likely to affect the interpretability of sonified trends. The model explores the perceptual constraints associated with design choices such as presentation rate, tonality, space, and frequency range. The model highlights the dependencies these mappings have on our perception of frequency, timbre, loudness, and space: the core determinants of stream segregation (a key phenomenon of ASA). This model-centred approach is used to identify perceptually congruent design choices for auditory line charts, a popular and accessible form of parameter-mapping sonification. This article focuses predominately on identifying an efficient and perceptually congruent frequency range for auditory line charts, as there are no contemporary studies on this design choice, with auditory graphing tools leaving little guidance on what might be appropriate. Several psychoacoustically justified ranges are proposed by the PerCS research and are contrasted in a user study to determine the performance of narrow vs wide frequency ranges. Results show that a shorter range between 400 and 1400 Hz performs just as well as a wider range between 65 Hz and 1480 Hz.
KW - Mappings
KW - Perception
KW - Psychoacoustics
KW - Sonification
UR - http://www.scopus.com/inward/record.url?scp=85169160278&partnerID=8YFLogxK
U2 - 10.1007/s12193-023-00413-w
DO - 10.1007/s12193-023-00413-w
M3 - Article
AN - SCOPUS:85169160278
SN - 1783-7677
VL - 17
SP - 285
EP - 300
JO - Journal on Multimodal User Interfaces
JF - Journal on Multimodal User Interfaces
IS - 4
ER -