SYST/STAT 664 Course Syllabus

SYST/STAT 664 Bayesian Inference
and Decision Theory
Spring Semester, 2009
The objective of this course is to introduce students to the theory of Bayesian inference and decision making and to introduce applications in information technology and engineering. Students will learn the fundamentals of the Bayesian theory of inference, including probability as a representation for degrees of belief, the likelihood principle, the use of Bayes Rule to revise beliefs based on evidence, conjugate prior distributions for common statistical models, methods for approximating the posterior distribution. Graphical models are introduced for representing complex probability and decision models by specifying modular components. A brief overview is given of basic concepts in decision analysis, including influence diagrams, decision trees, and utility theory.

Texts and software

I teach from course notes (to be posted on this web site prior to this lecture) and not from a text. Nevertheless, I believe it is important for students to have a textbook because lecture notes by their nature lack the detail provided in a text. There is no perfect text for this material.

Required text: The required text is the most comprehensive text and reference book on Bayesian methods I have found. The hyperlink below contains reviews, exercises, data sets and software.

Gelman, A., Carlin, J., Stern, H. and Rubin, D., Bayesian Data Analysis (2nd edition), Chapman & Hall, 2004.

Alternate text: Although it is far less comprehensive, and not as useful as a reference, many students find the text by Peter Lee more accessible. Again, the hyperlink contains additional information, including exercises, solutions, errata and software.
Lee, Peter, Bayesian Statistics: An Introduction (3rd edition), Arnold, 1997.
Some students may wish to purchase both books, although this can be quite costly.

Software:

All students are expected to have access to a software package that can perform elementary data analysis functions such as scatterplots, histograms, hypothesis tests, confidence intervals, linear regression, etc. Many of the homework and exam exercises can be managed with a full-featured spreadsheet package such as Microsoft Excel.
We will make use of R, a powerful statistical graphics and computing language, and MCMCpack, an R package for Bayesian computation.

Requirements

Grades will be based on the following:

Homework assignments 20%

Midterm exam 30%

Final exam 30%

Project 20%

Homework problems will be due one week from the day they are assigned. Eight to ten assignments will be given through the semester. Students are encouraged to work together on homework exercises, but solutions must be written up individually. Exams will be take-home and will be similar to the homework problems. Students are expected to work by themselves on the exams.

Schedule

The midterm exam will be posted on February 28 and will be due on March 19 at the start of class. It will cover Units 1-3 and part of 4 (depending on what we have covered before the exam is distributed). The final exam will be posted on April 23 and will be due on Thursday, May 7 at 10:00PM. The final exam will be cumulative. All students are required to do a data analysis project. The project is due on Monday, May 11 at 11:59 PM (revised due date).

Communication

The course web site is http://ite.gmu.edu/~klaskey/SYST664/SYST664.html

All students are required to have electronic mail accounts. Announcements and information about homework assignments are distributed between class meetings via electronic mail. All registered George Mason University students have email accounts. Information is available from the Computing and Information Systems office about how to activate your account electronically. There is also a Blackboard site for this course.

A listserv has been set up for this class. The listserv permits any student or teacher to send messages to the entire class.

Course Schedule

The topics for each unit are listed below, along with readings from the text.

Unit 1 Bayesian Inference and Decision Theory: An Overview Week 1-2 Gelman, et al., Chapter 1
Lee: Chapter 1

Unit 2 Random variables, Parametric models and Inference from Observation
Weeks 2-3 Gelman, et al., Chapters 1, 2
Lee: Chapter 2

Unit 3 Statistical Models with a Single Parameter Weeks 4-6 Gelman, et al., Chapter 2
Lee: Chapters 2 and 3

Unit 4 Hypothesis Tests and Parameter Estimation Weeks 6-7 Lee, Chapter 4, 7.7

Unit 5 Statistical Models with Multiple Parameters Week 8-9 Gelman, et al., Chapter 3

Unit 6 Graphical Models and Hierarchical Inference
Weeks 9-11 Gelman, et al., Chapter 5
Lee: Chapter 8

Unit 7 Bayesian Computation Weeks 11-12 Gelman, et al., Chapters 10-13
Lee: Chapter 9

Unit 8 Bayesian Regression
Weeks 12-13 Gelman, et al., Chapter 14; Lee: Chapter 6

Unit 9 Other Topics
Week 14 To be determined

Unit 1	Bayesian Inference and Decision Theory: An Overview	Week 1-2	Gelman, et al., Chapter 1 Lee: Chapter 1
Unit 2	Random variables, Parametric models and Inference from Observation	Weeks 2-3	Gelman, et al., Chapters 1, 2 Lee: Chapter 2
Unit 3	Statistical Models with a Single Parameter	Weeks 4-6	Gelman, et al., Chapter 2 Lee: Chapters 2 and 3
Unit 4	Hypothesis Tests and Parameter Estimation	Weeks 6-7	Lee, Chapter 4, 7.7
Unit 5	Statistical Models with Multiple Parameters	Week 8-9	Gelman, et al., Chapter 3
Unit 6	Graphical Models and Hierarchical Inference	Weeks 9-11	Gelman, et al., Chapter 5 Lee: Chapter 8
Unit 7	Bayesian Computation	Weeks 11-12	Gelman, et al., Chapters 10-13 Lee: Chapter 9
Unit 8	Bayesian Regression	Weeks 12-13	Gelman, et al., Chapter 14; Lee: Chapter 6
Unit 9	Other Topics	Week 14	To be determined