Misguided study affected by confirmation bias

Abstract

Some studies attempt to “prove” that helmets reduce injuries rather than genuinely attempt to understand their actual protective effects. Bold claims are made despite not being supported by the data. While such studies are well meaning, their misleading claims can push people towards counterproductive solutions. 

 

One example is this study, that states it is designed to

demonstrate the safety benefits of helmet use“.

It seems that the researchers assume that helmets are effective and attempt to prove it. This lacks scientific discipline. It seems like an attempt to prove a preconception. With such an approach, there is a strong tendency for confirmation bias: data that does not lead to the preconception tends to get ignored.

The bias is revealed in this newspaper article:

“But the percentage of injured cyclists who needed treatment for serious head injuries dropped from 10.3 per cent in 2005 to 2.5 per cent last year.”That could be put that down to helmet use,”  “

What is odd about this conclusion is that helmet usage in the area was lower in 2009 than in 2005. Lower helmet usage was correlated with lower head injuries. How can lower head injuries can be credited to helmets when fewer people were wearing helmets?

The study report indicates unhelmeted cyclists had an Injury Severity Score (ISS) of 7, much higher than helmeted cyclists with an ISS of 4. ISS reflect all injuries, not just head injuries (25% of injuries). A higher ISS indicates more severe accidents. Differences in head injuries cannot be fully attributed to helmets.  This is a common error is statistics, not checking for confounding variables. This renders the study conclusion invalid:

“The study confirmed the utility of helmet use in preventing serious head injury after cycling accidents.  This was the only factor in this study to influence the severity of injury.”

How can the researchers have missed accident severity as a relevant factor?

Despite the lack of scientific discipline, the bias of the researchers, and the flaws in the study, the government claims that this “confirms” that helmets are effective.

This is typical of studies attempting to “prove” the effectiveness of helmets. While such studies are well meaning, they can mislead people towards false “solutions” to cycling safety, while more effective measures are neglected.

You can skip to the end and leave a response. Pinging is currently not allowed.

3 Responses to “Misguided study affected by confirmation bias”

  1. کود says:

    Nice replies in return of this matter with real arguments and explaining the whole thing.

  2. Luke says:

    I have a copy of this article in my Evernote database. My snapshot shows a Doctor’s name. May I ask why that Doctor’s name has been removed from your commentary on his study?

    • admin says:

      The point of this commentary is not to pick on a particular individual, but to illustrate a common pattern. Many studies about bicycle helmets are done by people who may have good intentions, but do not have sufficient scientific discipline and emotional detachment from this topic to be able to make an objective scientific study. The result is a study that makes bold claims that are not supported by the underlying data. This tends to mislead people by exaggerating the protection provided by helmets.

      This can lead to more effective safety measures being neglected, resulting in cycling environment not as safe as what could be achieved by assessing each safety measure objectively.

      Unfortunately, good intentions do not always result in a positive outcome.

Leave a Reply

Current day month ye@r *

Powered by WordPress | Designed by: Free Web Hosting | Thanks to CD Rates, Las Vegas Condos and Comer Para Perder