Fortunately, I was also exploring Understanding Uncertainty today as well, and came across this wonderful interactive illustration of exactly these laws. Great for seeing mortality rates in action (not often one says that…).
This ties in with my earlier post on how graphics, especially interactive ones, will help make statistical and probablistic analysis more mainstream and prevalent as more people are exposed to it. However, it does still require a sceptical and trained mind to understand the implications of sampling procedures, tests against the null etc. I do wonder if more easily available statistical methods (Google Analytics, SurveyMonkey’s graphing tools) will create a low hurdle for those with no statistical training to present “analysis” as if they do.
Historically, the mere fact you were able to put together a chart would identify you as a nerd. Now that statistics is the next sexy profession, everyone will try to jump on it. But just because you can make a chart doesn’t mean your analysis is meaningful or correct. I’ve seen more and more presentations by individuals who had zero training in statistics, but still presented “statistical evidence” by bar charts or pie charts (the hallmark of the statistically naive). My favorite was a presentation where a comparison of responses between two groups was done on count data – the fact that 200 (out of 2,000) respondents in Group A responded “yes”, while 40 (out of 200) in Group B responded “yes” was taken as evidence that Group A were more amenable to the change. Yikes.
Varian is correct in that:
The ability to take data—to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it—that’s going to be a hugely important skill in the next decades… Because now we really do have essentially free and ubiquitous data. So the complimentary scarce factor is the ability to understand that data and extract value from it.
I think statisticians are part of it, but it’s just a part. You also want to be able to visualize the data, communicate the data, and utilize it effectively. But I do think those skills—of being able to access, understand, and communicate the insights you get from data analysis—are going to be extremely important. Managers need to be able to access and understand the data themselves.
But don’t underestimate the importance of that last sentence. In fact, that might be the most important sentence. The ability of a decision-maker to discern good, trustworthy statistical analysis from amateur hour will both increase the quality of decisions, and dissuade amateurs from attempting to use simplistic analysis to make their point. If the weakest link runs the show, the fact you’ve used propensity scores doesn’t matter.