ENEE 621: Estimation and Detection Theory

Course Goals:

The objective of this course is to introduce the student to the fundamental concepts of estimation and detection theory. Topics in estimation theory will include Bayesian parameter estimation (minimum mean-squared error and maximum a posteriori estimators) and non-Bayesian parameter estimation (maximum likelihood estimators). The fundamental Cramer-Rao Lower bounds on estimator error covariance will be addressed. Basic properties of estimators such as efficiency and consistency will also be studied. In detection theory, basic concepts will include simple and composite hypothesis testing and sequential detection. The problem of detecting a discrete-time or continuous-time signal in white or colored noise will be considered in depth.

Course Prerequisite:

ENEE 620 or equivalent.

References:

  1. Poor, An Introduction to Signal Detection and Estimation, Springer Verlag (1988).
  2. Anderson and Moore, Optimal Filtering, Prentice-Hall, (1979).
  3. Davis, Linear Estimation and Stochastic Control, Chapman and Hall, London (1977).
  4. Ferguson, Mathematical Statistics: A Decision-Theoretic Approach, Academic Press, New York, NY (1967).
  5. Helstrom, Statistical Theory of Signal Detection, Pergamon Press, Oxford (1968).
  6. Srinath and Rajasekharan, An Introduction to Statistical Signal Processing with Applications, Wiley (1979).
  7. Van Trees, Detection, Estimation, and Modulation Theory, Parts I and II, Wiley (1968).

Core Topics:

I. Estimation Theory

  • Bayesian Parameter Estimation: mean-squared error and maximum a posteriori probability criteria.
  • Non-Bayesian Parameter Estimation: maximum likelihood estimation.
  • Properties of Estimators: sufficient statistics, bias, consistency, efficiency; Cramer-Rao bounds, asymptotic efficiency and normality, minimum variance unbiased estimators.
  • Linear Least-squares Estimation: projection theorem, properties of linear estimators.

II. Detection Theory

  1. Hypothesis testing: likelihood ratio, Bayes' criterion, minimax criterion, Neyman-Pearson criterion, sufficient statistics, performance evaluation -- receiver operating characteristics.
  2. Multiple hypothesis testing.
  3. Composite hypothesis testing: generalized likelihood ratio, uniformly most powerful test.
  4. Sequential detection: Wald's test.
  5. Detection of signals in noise: discrete time, continuous time.
  6. Detection of known signals in white noise: discrete-time approximations: Brownian motion approach, bandlimited noise approach; correlation receiver, matched filter receiver.
  7. Detection of known signals in colored noise: Karhnen-Loeve expansion, whitening filter approach, singular detection.
  8. Detection of known signals in noise: signal-to-noise ratio criterion.
  9. Detection of signals with unknown parameters: deterministic and random parameters.

Optional Topics:

Estimation Theory

  • Minimum Description Length (MDL) principle.
  • Numerical techniques for estimation: the E-M algorithm.

Detection Theory

  • Asymptotic optimality in simple and composite hypothesis testing for i.i.d. observations: Stein's lemma. Robust and nonparametric detection; locally optimal detection.