ENEE 621: Estimation and
The objective of this course is to introduce the
student to the fundamental concepts of estimation and detection theory.
Topics in estimation theory will include Bayesian parameter estimation
(minimum mean-squared error and maximum a posteriori estimators) and
non-Bayesian parameter estimation (maximum likelihood estimators). The
fundamental Cramer-Rao Lower bounds on estimator error covariance will
be addressed. Basic properties of estimators such as efficiency and
consistency will also be studied. In detection theory, basic concepts
will include simple and composite hypothesis testing and sequential
detection. The problem of detecting a discrete-time or continuous-time
signal in white or colored noise will be considered in depth.
ENEE 620 or equivalent.
- Poor, An Introduction to Signal Detection and Estimation,
Springer Verlag (1988).
- Anderson and Moore, Optimal Filtering, Prentice-Hall, (1979).
- Davis, Linear Estimation and Stochastic Control,
Chapman and Hall, London (1977).
- Ferguson, Mathematical Statistics: A Decision-Theoretic
Approach, Academic Press, New York, NY (1967).
- Helstrom, Statistical Theory of Signal Detection,
Pergamon Press, Oxford (1968).
- Srinath and Rajasekharan, An Introduction to Statistical Signal
Processing with Applications, Wiley (1979).
- Van Trees, Detection, Estimation, and Modulation Theory,
Parts I and II, Wiley (1968).
I. Estimation Theory
- Bayesian Parameter Estimation: mean-squared error and
maximum a posteriori probability criteria.
- Non-Bayesian Parameter Estimation: maximum likelihood
- Properties of Estimators: sufficient statistics, bias, consistency,
efficiency; Cramer-Rao bounds, asymptotic efficiency and normality,
minimum variance unbiased estimators.
- Linear Least-squares Estimation: projection theorem,
properties of linear estimators.
II. Detection Theory
- Hypothesis testing: likelihood ratio, Bayes' criterion,
minimax criterion, Neyman-Pearson criterion, sufficient statistics,
performance evaluation -- receiver operating characteristics.
- Multiple hypothesis testing.
- Composite hypothesis testing: generalized likelihood ratio,
uniformly most powerful test.
- Sequential detection: Wald's test.
- Detection of signals in noise: discrete time, continuous time.
- Detection of known signals in white noise: discrete-time
approximations: Brownian motion approach, bandlimited noise approach;
correlation receiver, matched filter receiver.
- Detection of known signals in colored noise: Karhnen-Loeve
expansion, whitening filter approach, singular detection.
- Detection of known signals in noise: signal-to-noise
- Detection of signals with unknown parameters: deterministic
and random parameters.
- Minimum Description Length (MDL) principle.
- Numerical techniques for estimation: the E-M algorithm.
- Asymptotic optimality in simple and composite hypothesis testing
for i.i.d. observations: Stein's lemma. Robust and nonparametric
detection; locally optimal detection.