I have observed in the general population a tendency to use statistics and claim them as factual when trying to make a point. This regularly occurs when one wishes to compares things. I find this particular use of statistics to be a sadly narrowed view of anything in life. One of the more common usage of this behavior can be found in sports - particularly sports like the NBA and baseball. My friends who know I have an absolute passion for Kobe Bryant will regularly use LeBron James and Michael Jordan's 'stats' as a way to compare and judge. They seem to always converge on the same conclusion, but does this make their judgement right? In everyday life, people love using percentages, ratios, and all kinds of numbers to make their point appear supported. Educators also use averages and numerical marks to rank students. But does this mean the numbers tell the whole story?
In science, particularly a science like psychology, statistical analysis is one of the most important tools of the trade. Like any tool, it can be used correctly to produce the desired result or misused (through either lack of understanding how it should be used or intentional manipulation) to fabricate a desired result.
The complexity of statistics and its varied analytic methodologies give the tool the power needed to produce an outcome. You would probably want to assume that this is an advantage of such a tool. But as Simmons, Nelson, and Simonsohn (2011) argue in their paper, a construct they term the "researcher degrees of freedom" (which I like to interpret as a personal choice for use of certain complex statistics) can lead to false-positive conclusions to be reported. It is the premise of research in psychology to find "statistically significant" results so they can reject the null hypothesis (this is like the nagging voice in your head that tells you not to do something because it can't be done). A false-positive is the incorrect rejection (or muting the volume) of this nag.
They also go on to state "it is common (and accepted practice) for researchers to explore various analytic alternatives, to search for a combination that yields statistical significance, and to then report only what worked." Put simply, this statement can explain what most of us tend to do - we explore different percentages, ratios, and stats, search for the combination that best fits our argument and then we report them to our friends. While many methods in research have been developed to rid us of this simple but flawed occurrence, the researchers of this paper nonetheless report it to transpire regularly - even among what is supposed to be objective and highly intellectual individuals. They then go on to explain this as being the result of "ambiguity in how best to make (these) decisions" and "the researcher's desire to find a statistically significant result." The former can be interpreted simply as a layman's lack of absolute understanding for whatever it is they are arguing and the latter as a self-serving bias because humans tend to want to be right.
While this paper goes on to produce its own experiment and simulations to make their case, I will not be continuing to analyse the rest of the paper for this post. I am personally not reviewing this research paper, but I do have something to add. While most scientists aspire to find the truth about human nature, I do not believe they would willingly endanger our species with fabricated results. Much of statistics is built upon being objective, but the initial input and final interpretation of this data is subject to human intervention. As humans we can never be perfect (ironically it was psychologists who showed us this) and thus we must admit that mistakes are possible and will happen (luckily, even when psychology studies are wrong, they don't necessarily cause harmful physiological damage). To be wrong is a necessity to eventually finding the truth if we are willing to explore further. Any attempt to find the truth the first few times something is investigated is a difficult ask. As social beings, collaboration should be encouraged to undertake further research and that way, the truth cannot be subjected to just one human mistake (or in some cases, human manipulation).
Exploration is important for humans. When people make their case by promptly bringing up some supporting statistic, they are hindering the opportunity to explore. If we as a species are handcuffed by this with a narrowing of our views based on statistics, the possibilities of learning are now redundant. There are many human factors that affect an outcome in our human world. To simply boil these down to a number would be like viewing our world in the machinery of robots and computers.
For anyone that is rather bored and wishes to explore numbers visually, here is a link to Gap Minder (yet another brilliant Google collaboration).
And for anyone who has 6 minutes to spare, this is a really funny use of statistics for TED Talks.
Reference:
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22, 1359-1366.
No comments:
Post a Comment