Title of the course:
Introduction to information theory
Number of contact hours per week:
Department of Probability Theory and Statistics
oral or written examination
Probability theory and Statistics
A short description of the course:
Source coding via variable length codes and block codes. Entropy and its formal properties.
Information divergence and its properties. Types and typical sequences. Concept of noisy
channel, channel coding theorems. Channel capacity and its computation. Source and channel
coding via linear codes. Multi-user communication systems: separate coding of correlated
sources, multiple access channels.
Csiszár – Körner: Information Theory: Coding Theorems for Discrete Memoryless Systems.
Akadémiai Kiadó, 1981.
Cover – Thomas: Elements of Information Theory. Wiley, 1991.