ENEE 627: Information Theory
Spring 2015

Instructor : Alexander Barg, Professor
Department of Electrical and Computer Engineering/Institute for Systems Research
Office: 2361 A.V.Williams Building. E-mail abarg at  umd  dot edu

TA: Cihad Gulcu,  gulcu at umd dot edu.
TA office hours Monday, Wednesday 11:30 am - 12:30 pm; AVW 1301

Class times: Tuesday, Thursday 3:30-4:45pm CHE2118
Instructor availability outside class hours: after class (preferred), TBD
Homepage: http://www.ece.umd.edu/~abarg/627

Done for the semester, have a good summer

Midterm exam 4/16, in class. Coverage of the midterm -- Ch.2; 3; 5; 7, 11.1, 11.2. You are allowed 2 pages of notes (write on 2 sides of one sheet or on one side of 2 sheets).
Exams from a previous year: 1 2

Home assignments:
hw1 due in class on Thursday 2/19  Solutions
hw2 due on Tuesday 3/3  Solutions
hw3 due on Tuesday 3/31  Solutions
hw4 due on Tuesday 4/7  Solutions
Solutions of midterm problems
hw5 due on Thursday 5/7   Solutions
Final exam with solutions

Main topics:

Entropy, relative entropy, mutual information Ch. 2
Typical sets, AEP property, Ch.3, Ch. 11 (Sect.11.1,11.2)
Data compression, source codes, Ch. 5
Capacity of discrete memoryless channels, direct and converse coding theorems, Ch. 7 (notes)
   Effective version of Shannon's theorem: polar coding (notes)
Differential entropy (Ch. 8). Gaussian channel and its capacity (Ch. 9)
Information theory and statistics (Ch. 11)

Course calendar (online)

Grading: several home assignments (20%), midterm (40%), final (40%).
Textbook: T. M. Cover and J. A. Thomas "Elements of Information Theory" 2nd Ed, Wiley Interscience, 2006
Other books (recommended, will not be used in an essential way):
        I. Csiszr and J. Krner, Information Theory, Cambridge University Press 2011
        R. Gallager, Information Theory and Reliable Communication, Wiley 1969
Prerequisites: Probability and random processes.