Models can be wrong. There are other models predicting exactly the opposite (clean Obama win) which have ALSO "correctly predicted" the results of the last several elections. (I can dig one up if anyone cares, but you've probably seen them in news reports.) Models always make simplifying assumptions, and frequently they are "trained" formally or informally using past results--and testing models against their own training data proves nothing. It's also provable from computer science that purely empirical testing of models will fail: the No Free Lunch theorem establishes that there
is no optimal empirical method for generalizing statistically from past results to future results. You HAVE to understand the "why", or your model is just black magic and may break at any time.
Ultimately the test of a good model/hypothesis/theory is twofold: 1.) Does it accurately predict outcomes OTHER than the ones used in its own creation, e.g. future outcomes or historical outcomes? 2.) Does it yield useful insights into the underlying causal mechanisms?
Remember this next time someone talks about their (political, economic, climate, etc.) models. Ask, "What data did you use to make this model, and what data did you use to test it? Are the predictions the model makes about human behavior or climate plausible given what else we know about human psychology, meteorology, or physics?" Until you've asked these questions and listened to the answers, their model isn't really telling you anything. Until then, it's just noise.
Ultimately the test of a good model/hypothesis/theory is twofold: 1.) Does it accurately predict outcomes OTHER than the ones used in its own creation, e.g. future outcomes or historical outcomes? 2.) Does it yield useful insights into the underlying causal mechanisms?
Remember this next time someone talks about their (political, economic, climate, etc.) models. Ask, "What data did you use to make this model, and what data did you use to test it? Are the predictions the model makes about human behavior or climate plausible given what else we know about human psychology, meteorology, or physics?" Until you've asked these questions and listened to the answers, their model isn't really telling you anything. Until then, it's just noise.
Hahahahaaaa!!! That is ME laughing at YOU, cruel world.
-Jordan Rixon
I could not love thee, dear, so much,
Loved I not Honour more.
No comments:
Post a Comment