Bad Science...

Incompetence or Malice?

I have a lot of disgust with how the scientific process is often distorted to sway public opinion.  All too often, data are presented in a way that is downright misleading, and most of today's general public don't have the scientific training (or even exposure) to recognize that they are being bamboozled. 

For example...  These graphs were used to support someone's pet theory relating the study parameter to biological age as determined by what stage of puberty a child falls under  (in this case, making the case that middle and high school students need a later start to the school day)  This presentation highlights a commonly-used tactic to make one's data look more compelling than it really is).  As we will discuss, these data are totally meaningless...

In this case, the ranges of values for each category far exceed the trend that was concluded from the data. Also, the text of the report states that the error ranges shown are only +/- one half standard deviation (showing only ~34% of the overall range of the variation). If a range chosen showed 90% of the variation (as would be required to really demonstrate any correlation, the slope of the line would be completely unnoticeable (and even more meaningless).

Furthermore, the "puberty" categories used are completely subjective -- and were not selected a priori.  In other words, the category that each subject was placed in was not determined before evaluating the students, but after -- allowing the researcher to assign the category based on the behavior being studied studied.  

We can't evaluate the effect of the poor practice in assigning the categories, but CAN examine the data with a more realistic presentation.  If I were to graph the left chart properly (full scales on the axes and showing the full range of +/- 1 standard deviation), it would look like this:

Given the range of variability in the data for each category, I would have a very tough time claiming any validity for a meaningful trend in this data.  Yet, these are the kind of data tricks/deceits that are used to persuade voters (and the political machines in DC) to take all manner of extreme actions to "improve" our lives.

Author Robert Heinlein once observed, "Never attribute to malice that which can be adequately explained by stupidity, but don't rule out malice."

Sometimes it's tough to know which motivation to blame.