SYST/STAT 664

Bayesian Inference and Decision Theory

Kathryn Blackmond Laskey
Department of Systems Engineering
George Mason University
Course Description

Spring, 2009
ST2 Room 15


Many students find statistics to be a confusing collection of rules and procedures, the applicability of which seems to have little rhyme or reason.  The Bayesian approach provides a unified and intuitively appealing approach to the problem of drawing inferences from observations.   Bayesians view statistical inference as a problem in belief dynamics, or use of evidence about a phenomenon to revise and update our knowledge about it. Bayesian statistics is a scientifically justifiable way to integrate informed expert judgment with empirical data.  For a Bayesian, statistical inference cannot be treated entirely independently of the context of the decisions that will be made on the basis of the inferences.  This course introduces students to the Bayesian theory and modern computational methods for Bayesian inference.  Students will learn the commonalities and differences between the Bayesian and frequentist approaches to statistical inference, how to approach a statistics problem from the Bayesian perspective, and how to combine data with informed expert judgment in a sound way to derive useful and policy relevant conclusions.  Students will learn the necessary theory to develop a firm understanding of when and how to apply Bayesian and frequentist methods, and will also learn practical procedures for inference, hypothesis testing, and developing statistical models for phenomena.  Specifically, students will learn the fundamentals of the Bayesian theory of inference, including probability as a representation for degrees of belief, the likelihood principle, the use of Bayes Rule to revise beliefs based on evidence, conjugate prior distributions for common statistical models, methods for approximating the posterior distribution.  Bayesian networks, influence diagrams, and general graphical models are introduced for representing complex probability and decision models by specifying modular components.  Assignments make use of modern computational techniques and focus on appling the methods to practical problems.

Course Syllabus

Distance Learning

This course is being offered as both a regular lecture course and a distance education course.  Students may attend lectures in person or may participate from home over the Internet. Students logged in electronically can hear lectures, see vugraphs, see what the instructor writes on the electronic whiteboard, and ask questions. All lectures will be recorded and can be played back at students' convenience.  Instructions for using the student client software will be provided prior to the first class.

Class Email List

Everyone in the class will be subscribed to the class email list. The email list will be used to communicate with students outside of class. If you wish to subscribe using an email other than your GMU email, you may do so by visiting the GMU listserv web site.  All email correspondence with the instructor should say STAT664 or SYST664 in the subject line.

Lecture Notes

Lecture notes for each chapter will be made available before the first class covering that chapter. You will need to download Adobe Acrobat Reader to read these lecture notes.  Links to the previous year's notes are provided for those who like to read ahead.  Be advised -- I make changes every year.

Homework Assignments

Homework is due at class time on the assigned due date. If it is submitted before 23:59 the day after the due date, you will receive 75% credit.  If it is submitted up to 1 week late, you will receive 50% credit.  If you have extenuating circumstances, please contact me in advance, and I will consider giving you additional time to complete the assignment for partial credit. There is a Blackboard dropbox for electronic assignments. Assignments will be posted here and on Blackboard.

Exams

Study Aids

Sites of Possible Interest

A recent web search on "Bayesian" yielded over 5 million hits.  When you have some time, it would be a good idea to throw in a modifier or two to reflect your individual interests and browse.  There is lots of very interesting stuff out there.  I've culled a few tidbits to get you started.  Many of these sites also contain useful links to other Bayesian sites.
Professional Societies:
International Society for Bayesian Analysis
American Statistical Association Section on Bayesian Statistical Science
A Non-Comprehensive Sampling of Research Groups and Stat Departments with Bayesian Orientation:
Carnegie Mellon University Department of Statistics
Duke University Department of Statistical Science
University of Washington Statistics Department
Microsoft Machine Learning and Applied Statistics Group
Free Bayesian Statistical Software:
MCMCpack (an R package for Bayesian analysis)
Bayesian Analysis Using Gibbs Sampling (BUGS)
FirstBayes
Companion software to Peter Lee text in R and C++
Kevin Murphy's Bayesian Network Toolbox for MATLAB
Miscellaneous
The Bayesian Songbook (includes Frequentist Frenzy by world renowned songwriter Kathryn Blackmond Laskey)
Bayesians Worldwide