Multimedia retrieval (MR) is about the search for and delivery of multimedia documents, such as text, images, video, audio, and 2D/3D shapes.
This course teaches MR from a bottom-up perspective. After introducing what MR is by means of examples and use-cases, the MR pipeline is presented.
Next, each of the building blocks of this pipeline is discussed in detail, starting with the most basic one (data representation), going through the modeling of human perception of media, feature extraction, matching, evaluation, scalability, and presentation issue.
At the end of the course, students should understand the theory, techniques, and tools that are involved in designing, building, and evaluating every block in the MR pipeline.
The overall aim is thus for students to be able to design, build, and evaluate end-to-end MR systems for different types of multimedia data.
The course covers multimedia retrieval from a multidisciplinary perspective. Aspects taken into account: MR data representation; data (signal, image, shape) processing; understanding and working with high-dimensional data; connections between MR, machine learning, and data visualization; computational scalability and complexity aspects of working with big data collections; and human factors in interactive systems design.
The course takes a predominantly practical stance: after the theoretical principles of MR are introduced, we focus on how MR is to be practically implemented to be successful.
Various design and implementation decisions for the MR pipeline building-blocks are discussed, focusing not only on their theoretical merits, but also ease of implementation/parameterization, robustness, and speed.
Trade-offs between alternative solutions to a given problem are discussed.
Lectures, self-study, presentations, and a project.
The course has no compulsory textbook, as a significant amount of information is presented in detail in slides, papers, notes, and demos.
However, the following books are strongly recommended as optional reading material, as they give additional details on the material discussed in the course:
- H. Eidenberger, "Handbook of Multimedia Information Retrieval", 2012, Atpress, ISBN 9783848222834.
- L. Da Fontoura Costa, R. Marcondes Cesar Jr, "Shape Analysis and Classification: Theory and Practice", CRC Press
- A.C. Telea, "Data Visualization - Principles and Practice", 2nd edition, 2014, CRC Press, ISBN 9781466585263
Visit the course page to find out which chapters from the above books cover which topics of the course.