Abstract In this paper several meaningful audio icons of classic arcade games such as Pong, Donkey Kong, Mario World and Pac-Man are analyzed, using the PRAAT software for speech analysis and musical theory. The analysis results are used to describe how these examples of best practice sound design obtain their meaning in the player's perception. Some aspects can be related to the use of tonal hierarchy (e.g. Donkey Kong and Mario World) which is a western culture related aspect of musical meaning. Other aspects are related to universal expressions of meaning such as the theory of misattribution, prosody, vocalization and cross-modal perceptions such as brightness and the uncanny valley hypothesis. Recent studies in the field of cognitive neuroscience support the universal and meaningful potential of all these aspects. The relationship between language related prosody, vocalization and phonology, and music seems to be an especially successful design principle for universally meaningful music icons in game sound design.
DOCUMENT
The realization of one’s musical ideas at the keyboard is dependent on the ability to transform sound into movement, a process called audiomotor transformation. Using fMRI, we investigated cerebral activations while classically-trained improvising and non-improvising musicians imagined playing along with recordings of familiar and unfamiliar music excerpts. We hypothesized that audiomotor transformation would be associated with the recruitment of dedicated cerebral networks, facilitating aurally-cued performance. Results indicate that while all classically-trained musicians engage a left-hemisphere network involved in motor skill and action recognition, only improvising musicians additionally recruit a right dorsal frontoparietal network dedicated to spatially-driven motor control. Mobilization of this network, which plays a crucial role in the real-time transformation of imagined or perceived music into goal-directed action, may be held responsible not only for the stronger activation of auditory cortex we observed in improvising musicians in response to the aural perception of music, but also for the superior ability to play ‘by ear’ which they demonstrated in a follow-up study. The results of this study suggest that the practice of improvisation promotes the implicit acquisition of hierarchical music syntax which is then recruited in top-down manner via the dorsal stream during music performance. In a study of audiomotor transformation in Parkinson patients, we demonstrated a dissociation between dysprosody in speech and music. While patients’ speech could reliably be distinguished from that of healthy individuals, purely on the basis of aural perception, no difference was observed between patients and healthy controls in their ability to sing improvised melodies.
LINK
In this chapter we discuss the implications of our research in the wider context of current models of brain function, endeavoring to understand the consequences of score-dependence and improvisation in terms of the ‘predicting brain’, the dual-stream model of perception and action, the procedural-declarative model of learning and memory, ideomotor learning and sensorimotor mapping, and the implicit acquisition of hierarchical music syntax.
DOCUMENT
