Course Information

Estimation Theory - Fall 2016

 

Instructor: Prof. Songhwai Oh (오성회)
Email: songhwai (at) snu.ac.kr
Office Hours: Friday 2:00-4:00PM
Office: Building 133 Room 405
Course Number: 430.714
Time: MW 3:30-4:45 PM
Location: Building 301 Room 106
TA: Yoonseon Oh (오윤선)
Email: yoonseon.oh (at) cpslab.snu.ac.kr
Office: Building 133 Room 610
 

Course Description

This course introduces classical and modern topics in estimation theory to graduate level students. Topics include minimum variance unbiased estimators, the Cramer-Rao bound, linear models, sufficient statistics, best linear unbiased estimators, maximum likelihood estimators, least squares, exponential family, multivariate Gaussian distribution, Bayes risk, minimum mean square error (MMSE), maximum a posteriori (MAP), linear MMSE, sequential linear MMSE, Bayesian filtering, Kalman filters, extended Kalman filter, unscented Kalman filter, particle filter, data association, multi-target tracking, and Gaussian process regression. Lectures will be in English.

Announcements

  • [12/05] The final exam will be held in in Room 508, Bldg. 203 on 12/14 (Wed) from 3:00-5:30PM. The exam is closed-book but you can bring one sheet (A4) of hand written notes on both sides. You have to turn in this cheat sheet with your exam.
  • [10/10] The midterm will be held in class (Bldg. 302, Room 508) on 10/24 (Mon). The exam is closed-book but you can bring one sheet (A4) of hand written notes on a single side (the other side must be blank). You have to turn in this cheat sheet with your exam.
  • [09/07] The class room is changed to Room 106.
  • [08/29] Please read Ethics of Learning.

Schedule

Week Reading Date Lecture Date Lecture
1 Kay Ch. 1, 2
Simon Ch. 1, 2
9/5
  • Introduction
  • Review on linear system theory
9/7
  • Review on probability
  • Minimum variance unbiased estimators
2
Kay Ch. 3.1-3.9
9/12
  • Cramer-Rao lower bound (CRLB)
9/14
  • Holiday
3 Kay Ch. 4, Ch. 5

9/19
  • Linear models
9/21
  • Sufficient statistics
4 Kay Ch. 6
  Ch. 7.1-7.6
  Ch. 8
9/26
  • More on Sufficient statistics
  • Best linear unbiased estimators
9/28
  • Maximum likelihood estimation
  • Least squares
5 Kay Ch. 10 10/3
  • Holiday
10/5
  • Exponential family
  • Bayesian approach
6 Kay Ch. 11 10/10
  • Multivariate Gaussian
  • Bayes risk, MMSE, MAP
10/12
  • No class
7 Kay Ch. 12 10/17
  • Linear MMSE
10/19
  • Sequential linear MMSE
8 Simon Ch. 5 10/24
  • Midterm
    • Bldg. 302, Room 508
10/26
  • Bayesian filtering
9 Simon Ch. 6
  Ch. 7
10/31
  • Kalman filter
11/2
  • Alternate Kalman filter formulations
10 Simon Ch. 9 11/7
  • Kalman filter generalizations
11/9
  • Optimal smoothing
11 Simon Ch. 13,
    Ch. 14
11/14
  • Optimal smoothing
11/16
  • Nonlinear Kalman filtering
12 Simon Ch. 15 11/21
  • Unscented Kalman filter
11/23
  • Particle filtering
13   11/28
  • Bayesian filtering, data association,
    and multi‐target tracking
11/30
  • Gaussian process regression
14   12/5   12/7  
15   12/12   12/14
  • Final exam
    • Bldg. 302, Room 508
    • 3:00 - 5:30 PM

Textbooks

  • [Recommended] Steven M. Kay, "Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory", Prentice Hall, 1993.
  • [Recommended] Dan Simon, "Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches", Wiley-Interscience, 2006.

Prerequisites

Students must have a solid background in linear algebra, linear system theory, and probability.

Topics

  • Introduction and review of probability and linear system theory
  • Minimum variance unbiased estimators
  • Cramer-Rao lower bound
  • Linear models and sufficient statistics
  • Best linear unbiased estimators and maximum likelihood estimators
  • Least squares, exponential family, and Bayesian approaches
  • Multivariate Gaussian distribution
  • Bayes risk, minimum mean square error (MMSE), and maximum a posteriori (MAP)
  • Linear MMSE and sequential linear MMSE
  • Bayesian filtering
  • Kalman filtering
  • Advanced topics in Kalman filtering
  • Extended Kalman filter, unscented Kalman filter, and particle filter
  • *Data association and multi-target tracking
  • *Gaussian process regression (*if time permits)