How to Achieve Robustness

Invest effort in analyzing the behavior of your model under plausible alternate assumptions
Examine what your model says about features you didn’t design it to fit
Examine your model’s behavior on improbable cases

Robustness means a high a priori probability that P(y|x,M) is “close enough” to P(y|x)