Multimedia signal processing

Course objectives

Multimedia technologies and systems; architecture and applications. Signal representation on a computer. Spectrum. Linear systems. Convolution. Multimedia signal sources. Fundamentals of compression and coding. Speech signal, modeling and analysis. Parametric models; speech coding standards. Speech synthesis and recognition. Audio signal. Psychoacoustic model, audio coding principles and standards. Human visual perception model. Image formats, coding, and standards. Video signal and its properties. Spatial, time and subjective redundancy. Video compression standards. Storage, processing and transmission of multimedia content.

Enrolment requirements and/or entry competences required for the course

-

Learning outcomes at the level of the programme to which the course contributes

  • Apply theoretical knowledge of the fundamentals of the six core disciplines and their relationship within cognitive science.
  • Apply specific knowledge and skills from selected disciplines constituting cognitive science.
  • Critically evaluate cognitive science findings and synthesize information to be employed in a collaborative professional environment.
  • Participate in data-driven innovation projects and apply appropriate data science tools.

Course content (syllabus)

  1. Introduction. Information and media. Multimedia signals. Text, sound, speech, music, volume, video. Multimedia systems. Information acquisition, processing, analysis, transfer and storage.
  2. Mathematical prerequisites. Representation of signals on a computer. Signal types, elementary signals. Modelling of signals on a computer.
  3. System representation. Modelling of systems on a computer. Linear systems. Impulse response and convolution.
  4. Spectrum. Spectral domain signal processing. Transfer function. Real spectra.
  5. Replacing missing information. Interpolation of signals and images.
  6. Audio signal in time and frequency domain. Probability density function; Frequency spectra; Rate of change.
  7. Audio signal sampling theorem, Sampling and quantization errors
  8. Redundancy and irrelevancy of audio signals, Acoustical characteristics of voice signal, Voice generation
  9. Psychoacoustic models, Formats of compressed signals, Linear prediction of coefficients (LPC)
  10. Quantization effects, General concepts of bit-rate reduction, signal redundancy and entropy, Discrete cosine transform
  11. Human visual perception model. Image compression basics.
  12. Video signal and its properties. Sampling rates for video and analog-to-digital conversion. Sampling structure. Chroma subsampling.
  13. Discrete cosine transform based coder and decoder (quantization process, zigzag scanning, RLC and VLC).
  14. Video compression and standards. Interframe prediction. Motion compensation. Motion vectors.
  15. Storage, processing and transmission of multimedia content.

Student responsibilities

Class attendance, midterm exam, final exam, laboratory exercises

Required literature

  • Yun Q. Shi, Huifang Sun (2017.), Image and Video Compression for Multimedia Engineering, CRC Press
  • Ze-Nian Li, Mark S. Drew, Jiangchuan Liu (2021.), Fundamentals of Multimedia, Springer Nature

Optional literature

  • -