Sound and music provide powerful ways for impacting the human experience involved in the engagement with games and media. In this course, you will learn how to apply and develop computational methods to extract, process and utilize music information from digital sound and music in the context of newly emerging research areas within games and media. You will learn how sound and music information is crucial for the human experience, and how the computational modelling of sound and music contributes to the enrichment of this experience in games and media. This encompasses that you will get to know both basic concepts on how human listeners extract, make sense of and give meaning to information from sound and music, and how these basic concepts are used, researched and applied through computational technology.
The course is structured around three main modules:
A: Sound and music for games
B: Analysis, classification, and retrieval of sound and music for media
C: Generation and manipulation of sound and music for games and media
The course will cover key topics for sound and music technology in the context of games and media, such as interactivity and immersion in games through sound and music (A), classification and retrieval of similar musical objects in multimedia (B), and the utilization of the emotional and affective qualities of music in games and media (A, C). You will learn what specific technologies are developed and required within these key topics, such as automatic pattern discovery, sound separation, voice separation, automatic segmentation, and feature extraction and manipulation (B). For studying, discussing and employing these technologies you will get to know different representation forms of music information in audio and symbolic data (A), different musical dimensions such as melody, rhythm, harmony, timbre and loudness (A, B), and how they are modelled through computational features (A, B, C). Moreover, you will learn about different general strategies for developing computational models for sound and music processing, such as model-based versus data-driven approaches, and about the challenges of evaluating these models.