Issues

Identifiability
- Several concepts may be “observationally equivalent” (not distinguishable even with infinite data)
- We usually care about good performance off training set and not about the “correct” concept
The “correct” model may not be “good for purpose”
- Computational complexity
- Storage requirements
- Unintelligibility
Many machine learning algorithms include “bias”
- Bias pushes system toward “good” models
- Priors in a Bayesian analysis act as bias
- Priors should be about belief not utility!