It's Significant but Does It Matter?

May 17th, 2016

Tags: big data, dig data analytics, analytics, statistical significance, data analysis, p value

In the world of Big Data Analytics we enjoy having lots and lots of data. In the world of statistics the old adage was more data is always better than less. It used to be that getting your hands on the data was costly and cumbersome, but now the age of Big Data. Wow, we have lots and lots of data probably more than we really need to make sound business decisions.

The p Value is the statistic that provides us with the confidence to declare that a difference exists, or it doesn't. It is like guilty or innocent. If the p Value is less than 0.05 we declare that a difference exists beyond a shadow of doubt. If the p Value is greater than 0.05 then that culprit, that variable, that thing we are studying is declared to be innocent of creating any change. It is innocent, but did we flex it enough or have enough data? Well, as in a trial we have to go along with the jury decision.

The case with Big Data Analytics is data sets are so large that our ability to detect a change or a difference is dramatically enhanced. We see p Values that are 0.000 and then some number. That is way wicked small, surely much much less than 0.05, which we have always used as the line of demarcation to declare guilt or innocence. The next question we must answer is, "Does it matter that we have detected this difference?" If that change or difference we have detected is way wicked small, say much less than a fraction of a percent, who cares? Does it really matter? To matter the difference must truly make a difference. It has to move that needle to a new level.

Big Data Analytics is powerful stuff, but just because some variable in the analysis is statistically significant we still have to ask, "Does it matter?"

Our courses are designed to help you make sound business decisions based upon statistical analysis.

Leave a Comment