Module Description

Module: Information Theory and Coding


Information Theory and CodingLecture3Summer Semester
Information Theory and CodingRecitation Section (large)1Summer Semester

Module Responsibility:

Prof. Gerhard Bauch

Admission Requirements:


Recommended Previous Knowledge:

  • Mathematics 1-3
  • Probability theory and random processes
  • Basic knowledge of communications engineering (e.g. from lecture "Fundamentals of Communications and Random Processes")

Educational Objectives:

Professional Competence

Theoretical Knowledge

The students know the basic definitions for quantification of information in the sense of information theory. They know Shannon's source coding theorem and channel coding theorem and are able to determine theoretical limits of data compression and error-free data transmission over noisy channels. They understand the principles of source coding as well as error-detecting and error-correcting channel coding. They are familiar with the principles of decoding, in particular with modern methods of iterative decoding. They know fundamental coding schemes, their properties and decoding algorithms.


The students are able to determine the limits of data compression as well as of data transmission through noisy channels and based on those limits to design basic parameters of a transmission scheme. They can estimate the parameters of an error-detecting or error-correcting channel coding scheme for achieving certain performance targets. They are able to compare the properties of basic channel coding and decoding schemes regarding error correction capabilities, decoding delay, decoding complexity and to decide for a suitable method. They are capable of implementing basic coding and decoding schemes in software.

Personal Competence

Social Competence

The students can jointly solve specific problems.


The students are able to acquire relevant information from appropriate literature sources. They can control their level of knowledge during the lecture period by solving tutorial problems, software tools, clicker system.

ECTS-Credit Points Module:



Written exam

Workload in Hours:

Independent Study Time: 124, Study Time in Lecture: 56

Course: Information Theory and Coding


Gerhard Bauch


German & English


Summer Semester


  • Fundamentals of information theory

    • Self information, entropy, mutual information

    • Source coding theorem, channel coding theorem

    • Channel capacity of various channels

  • Fundamental source coding algorithms:

    • Huffman Code, Lempel Ziv Algorithm

  • Fundamentals of channel coding

    • Basic parameters of channel coding and respective bounds

    • Decoding principles: Maximum-A-Posteriori Decoding, Maximum-Likelihood Decoding, Hard-Decision-Decoding and Soft-Decision-Decoding

    • Error probability

  • Block codes

  • Low Density Parity Check (LDPC) Codes and iterative Ddecoding

  • Convolutional codes and Viterbi-Decoding

  • Turbo Codes and iterative decoding

  • Coded Modulation


Bossert, M.: Kanalcodierung. Oldenbourg.

Friedrichs, B.: Kanalcodierung. Springer.

Lin, S., Costello, D.: Error Control Coding. Prentice Hall.

Roth, R.: Introduction to Coding Theory.

Johnson, S.: Iterative Error Correction. Cambridge.

Richardson, T., Urbanke, R.: Modern Coding Theory. Cambridge University Press.

Gallager, R. G.: Information theory and reliable communication. Whiley-VCH

Cover, T., Thomas, J.: Elements of information theory. Wiley.