Department of Electrical and Computer Engineering/Institute for Systems Research
Office: 2361 A.V.Williams Building. E-mail abarg at umd dot edu
Cihad Gulcu, gulcu at umd dot edu.
TA office hours Monday, Wednesday 11:30 am - 12:30 pm; AVW 1301
Class times: Tuesday, Thursday 3:30-4:45pm CHE2118
Instructor availability outside class hours: after class (preferred), TBD
Done for the semester, have a good summer
Midterm exam 4/16, in class. Coverage of the midterm -- Ch.2; 3; 5; 7, 11.1, 11.2. You are allowed 2 pages of notes (write on 2 sides of one sheet or on one side of 2 sheets).
Exams from a previous year: 1 2
• Entropy, relative
entropy, mutual information Ch. 2
• Typical sets, AEP property, Ch.3, Ch. 11 (Sect.11.1,11.2)
• Data compression, source codes, Ch. 5
• Capacity of discrete memoryless channels, direct and converse coding theorems, Ch. 7 (notes)
Effective version of Shannon's theorem: polar coding (notes)
• Differential entropy (Ch. 8). Gaussian channel and its capacity (Ch. 9)
• Information theory and statistics (Ch. 11)
Course calendar (online)
Grading: several home assignments (20%), midterm (40%), final (40%).
Textbook: T. M. Cover and J. A. Thomas "Elements of Information Theory" 2nd Ed, Wiley Interscience, 2006
Other books (recommended, will not be used in an essential way):
I. Csiszár and J. Körner, Information Theory, Cambridge University Press 2011
R. Gallager, Information Theory and Reliable Communication, Wiley 1969
Prerequisites: Probability and random processes.