Misguided study affected by confirmation bias


Some studies attempt to “prove” that helmets reduce injuries rather than genuinely attempt to understand their actual protective effects. Bold claims are made. Yet the data does not support them. While such studies are well meaning, their misleading claims can push people towards counterproductive solutions. 


One example is this study, that states it is designed to

demonstrate the safety benefits of helmet use“.

It seems that the researchers assume that helmets are effective and attempt to prove it. This lacks scientific discipline. It seems like an attempt to prove a preconception. With such an approach, there is a strong tendency for confirmation bias. People seek to confirm their existing beliefs.

The bias is revealed in this newspaper article:

“But the percentage of injured cyclists who needed treatment for serious head injuries dropped from 10.3 per cent in 2005 to 2.5 per cent last year.”That could be put that down to helmet use,”  “

What is odd about this conclusion is that helmet usage in the area was lower in 2009 than in 2005. Lower helmet usage was correlated with lower head injuries. How can lower head injuries can be credited to helmets when fewer people were wearing helmets?

The study report indicates unhelmeted cyclists had an Injury Severity Score (ISS) of 7, much higher than helmeted cyclists with an ISS of 4. ISS reflect all injuries, not just head injuries (25% of injuries). A higher ISS indicates more severe accidents. Differences in head injuries cannot be fully attributed to helmets. Accident severity affected the outcome. This is a common error is statistics, not checking for confounding variables. This renders the study conclusion invalid:

“The study confirmed the utility of helmet use in preventing serious head injury after cycling accidents.  This was the only factor in this study to influence the severity of injury.”

How can the researchers have missed accident severity as a relevant factor?

This is confirmation bias. Researchers look for data that matches their existing beliefs. They tend to miss data that does not confirm their belief.

Despite the lack of scientific discipline, the bias of the researchers, and the flaws in the study, the government claims that this “confirms” that helmets are effective.

This is typical of studies attempting to “prove” the effectiveness of helmets. While such studies are well meaning, they can mislead people towards false “solutions” to cycling safety, while more effective measures are neglected.

3 thoughts on “Misguided study affected by confirmation bias”

  1. I have a copy of this article in my Evernote database. My snapshot shows a Doctor’s name. May I ask why that Doctor’s name has been removed from your commentary on his study?

    1. The point of this commentary is not to pick on a particular individual, but to illustrate a common pattern. Many studies about bicycle helmets are done by people who may have good intentions, but do not have sufficient scientific discipline and emotional detachment from this topic to be able to make an objective scientific study. The result is a study that makes bold claims that are not supported by the underlying data. This tends to mislead people by exaggerating the protection provided by helmets.

      This can lead to more effective safety measures being neglected, resulting in cycling environment not as safe as what could be achieved by assessing each safety measure objectively.

      Unfortunately, good intentions do not always result in a positive outcome.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>