Structural Uncertainty

Model can be decomposed as M=(S,q)
- S - Structural assumptions (conditional independence, normality, connections in neural network, etc.)
- q is a structure-specific parametere (local probability distributions, mean and covariance of normal distribution, weights in neural network, etc.)
Traditional approach to statistical inference:
- Pick “best” S
- Estimate q assuming S is the correct structure
Problems with traditional approach
- Overfitting
- Poor performance off training set
- Underestimation of variance