ENEE 627 HW 3. Date due Tue.3/31 in class
1. Textbook problems 5.4, 5.5, 5.25, 5.28, 13.9
2. We consider binary block codes for a DMS $X.$ Let $f:{\mathcal X}^k\to \{0,1\}^n$ be an encoding map. Prove that a decoder $\phi:\{0,1\}^n\to{\mathcal X}^k$ minimizes the error probability $e(f,\phi)$ if and only if for all $y^n,$ the sequence $\phi(y^n)$ has the maximum probability among all $x^k$ such that $f(x^k)=y^n.$
3. Let $F\subset {\mathcal X}^k,$ where ${\mathcal X}$ is a finite set. Let $P_F\stackrel{\triangle}=\frac 1{|F|}\sum_{{\mathbf x}\in F}P_{\mathbf x},$ where  $P_{\mathbf x}$ is the type of ${\mathbf x}\in {\mathcal X}^k.$ Prove that $|F|\le 2^{k H(P_F)}.$