In 2000, the Australia Safety Transport Bureau (ATSB), a government agency, released a “meta-analysis”, that claims to provide overwhelming evidence that bicycle helmets reduce the risk of brain injury. This claim was rebutted in 2003, highlighting a lack of understanding of scientific mechanism of brain injury, failing to take into account rotational acceleration:
“This examination concentrates on injury to the brain and shows that the meta-analysis and its included studies take no account of scientific knowledge of its mechanisms. Consequently, the choice of studies for the meta-analysis and the collection, treatment and interpretation of their data lack the guidance needed to distinguish injuries caused through fracture of the skull and by angular acceleration. It is shown that the design of helmets reflects a discredited theory of brain injury. The conclusions are that the meta-analysis does not provide scientific evidence that such helmets reduce serious injury to the brain, and the Australian policy of compulsory wearing lacks a basis of verified efficacy against brain injury. “
The ATSB did not reply to the rebuttal, thus giving up on its claim. Despite being discredited, this “analysis” is still used by the RTA (a government agency) to defend the helmet law.
In 2011, a new meta-analysis re-assessed the meta-analysis done by the ATSB in 2000. It reports:
“This paper shows that the meta-analysis of bicycle helmet efficacy reported by Attewell, Glase, and McFadden (Accident Analysis and Prevention 2001, 345–352) was influenced by publication bias and time-trend bias that was not controlled for. As a result, the analysis reported inflated estimates of the effects of bicycle helmets“.
Publication bias is the tendency to cherry-picked results that support a pre-determined conclusion.
Time-trend bias is the tendency to pick a specific time period that produces the desired result.