skyking162

I just started reading The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century. It came highly recommended as an entertaining look at the evolution of the power of statistics.

The preface immediately set off my “no duh, that’s brillaint!” detector and reflects the roadblocks encountered by the sabermetric revolution:

Science entered the nineteenth century with a firm philosophical vision that has been called the clockwork universe. It was believed that there were a small number of mathematical formulas (like Newton’s laws of motion and Boyle’s laws for gases) that could be used to describe reality and to predict future events. … It took over forty years for popular culture to catch up with this scientific vision.

Eventually, as more precise measurements resulted in the need for large fudge factors in the equations, the paradigm of determinism was shoved aside.

Gradually, science began to work with a new paradigm, the statistical model of reality. By the end of the twentieth century, almost all of science had shifted to using statistical models.

Popular culture has failed to keep up with this scientific revolution. Some vague ideas and expressions (like “correlation”, “odds”, and “risk”) have drifted into the popular vocabulary, …but few non-scientists have any understanding of the profound shift in philosophical view that has occurred.

So it appears mainstream society lags behind science both in its understanding of the universe and in its approach towards increasing that understanding. Science prefers statistical modeling, while society prefers intangibles. Science seeks out change, while society curses change. Is it any wonder the use of statistical analysis receives a hostile welcome by most baseball fans, even though “stats” are a beloved part of the game?

To me, the most frustrating roadblock isn’t that most people don’t have training in academic statistics. The more appalling shortcoming is a lack of an intuitively statistical approach to life. Our society can’t accurately estimate probabilities. We see patterns where they don’t exist and are blind when they do. We can’t sniff out biases if they’re shoved up our noses. We can’t deal with the unknown, therefore allowing our decisions to be controlled by fear. I’m reminded of the old high school adage that if nothing else, understanding statistics is important for avoiding the manipulations of advertising. It sounds rather cliched, but it is representative of the overall problem.

Why is society so slow to embrace the changing philosophies of science? I think one reason is that science is viewed by many as cold and impersonal. Because shifts in scientific paradigms don’t often affect us on an emotional level, we resist them as impersonal and therefore not advantageous to our day-to-day lives.

Will a statistical approach help me with my relationship? Should I realize there’s only a 20% chance that Betty (beauty factor of 10) will date me and instead pursue Cindy (beauty factor of 7) since my expected happiness with her is higher because I know she’ll date me? Where’s the human intuition in that? Where’s the romanticism? How could two people possibly love each other if they acted rationally!?

It’s these arguments of statistics stripping the humanity away from baseball that are most often put forth in defense of traditionalism. And yet I would bet the proponents of statistical analysis are also some of the most obsessed baseball fans out there. (You’d need a statistical study to prove that, however.) We don’t only care about the statistical side of things, but since people tend to notice differences and not similarities, that leaves the statistical approach as the naked kid in the classroom. It’s only via forced, extended exposure that society becomes accustomed to the shock and accepts new scientific paradigms as the norm. Statistical modeling will never be sexy, but eventually people will catch on to its effectiveness and realize it doesn’t strip away our humanity. Sooner is better than later.

* * * * *

In addition to wondering how we can bring a statistical philosophy to society, I’m left with a tangential question — what’s the next scientific revolution? After determinism came statistical modeling. What’s next? Has it already arrived? And how much fun will it be for the next generation of baseball fans to try to convince us stats geeks to throw our statistical views of baseball aside and try something new? Will we?

Popularity: 2% [?]

This entry was posted on Thursday, May 18th, 2006 at 10:05 am and is filed under Baseball. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.