Lecturer: Carlo Bertoluzza  

Course name: Teoria dell'informazione
Course code: 064102
Degree course: Ingegneria Elettronica
Disciplinary field of science: ING-INF/05
L'insegnamento è caratterizzante per: Ingegneria Elettronica e delle Telecomunicazioni
University credits: CFU 5
Course website: n.d.

Specific course objectives

Aim of the course: expose the mai results of communication through channels with and without noise. In particular Shannon theory and Error correcting codes theory.

Course programme

In 1948 C.E.Shannon showed that it is possible to transmit data through a noisy channel at finite rate near to the channel capacity an with an error probability near to zero. This result is exposed in the first part of the course. The second part analyzes the error correcting codes. In this theory we suppose that if a bounded number of errors occurs in the transmission, then the redundacy introduced by a suitable coding procedure is able to find and correct the errors.

Source coding

• 1. Source codes and decipherability problem: the Kraft theorem,
• 2. Shannon entropy and their properties.
• 3. Alphabeth probability, Markovian and memoryless sources.
• 4. Lenght of a code and optimality.
• - Shannon inequalities.
• - Shannon and Huffmann coding algorithms.
• 4. Source entropy and redundancy.

Shannon theory

• 1. Channel codes and general decoding scheme.
• 2. Channel capacity and its form for binary memoryless simmetric channels.
• 3. Error probability and Fano's inequality.
• 4. Lower bound of probability error: the inverse theorem.
• 6. Mutual information and decoding rule.
• 7. Shannon fundamental theorem.

Error correcting codes

• 1. Ideal observer and minimum distance decoder.
• 2. Linear codes, subgroupa and vectorial subspaces.
• 3. Hamming and Bose-Cchauduri-Hogquengem (BCH) codes.
• 4. Non linear Hadamard code.
• 5. Cyclic codes and polinomial generator.
• 6. Non-binary Reed-Solomon codes.
• 7. Reed-Muller bases and Reed-Muller codes.
• 8. Variable lenght convolutional codes.
• 9. Low Density Parity Check (LDPC) codes.

Course entry requirements

Elementary probability theory in finite spaces (Bayes theorem, finite random variables, law of the large numbers. Boole algebra and binary string's finite fields.

Course structure and teaching

Lectures (hours/year in lecture theatre): 40
Practical class (hours/year in lecture theatre): 0
Practicals / Workshops (hours/year in lecture theatre): 0
Project work (hours/year in lecture theatre): 0

Suggested reading materials

R.J. McEliece. Information and Coding. Addison Wesley, 1977.

J.I. Hall. Notes on coding theory. Michigan State University, 2001.

J.Gill. Information course. Stanford University, 2002.

Testing and exams