Irreproducibility is holding back successful translational research, new survey finds

Published: 15-Dec-2014

Sigma-Aldrich conducts survey with the American Association for the Advancement of Science (AAAS) and the Academic Drug Discovery Consortium (ADDC)


The lack of experimental reproducibility is a 'major barrier to successful translational research', according to a new survey from Sigma-Aldrich.

The US company's second annual State of Translational Research Survey Report, based on a survey conducted with the AAAS and the ADDC, sheds light on practices that compromise the reproducibility of published research, underlines the translational research community's desire for experimental standards, and suggests multi-faceted strategies for improving the reproducibility of translational science.

More than 60% of respondents focused on the issue of irreproducibility, with only 22% stating that they were able to reproduce published work from other labs. In contract, 77% reported being able to reproduce their own lab's published work.

'The issue of irreproducibility of published studies is now widely acknowledged by the research community as a challenge to effective translational research as it could delay the creation of new treatments for patients and compromise the public's trust and confidence in science,' said Amanda Halford, Vice President, Academic Research at Sigma-Aldrich. 'Sigma-Aldrich is committed to helping the research community overcome this complex issue.'

Researchers are working to improve the rigour of recent experiments and in the last year, 98% of respondents reported performing more rigorous quality controls, increasing sample sizes, and ensuring thorough documentation. Fewer than one in six respondents enlisted the help of another lab to reproduce their findings or had non-authors either validate primary data or review all replicates before manuscript submission.

The causes of irreproducibility are complex and currently incompletely understood

Respondents suggested that the main causes of experimental irreproducibility were poor controls, the rush to publish, and insufficient samples sizes. Half of respondents also blamed the reproducing lab's failure to understand or follow experimental protocols.

The most frequently cited product groups that cause issues with reproducibility were animal models, antibodies and cell lines.

Scientific journals were expected by the majority (74%) to implement additional measures to ensure the reproducibility of published studies.

'The causes of irreproducibility are complex and currently incompletely understood. Addressing this issue will require multiple stakeholders to work together to identify and implement best practices for improving reproducibility, particularly in the preclinical research space. Potential strategies include validation and standardisation of research reagents and replication of key experimental results that will form the basis of drug development programmes,' said Elizabeth Iorns, Founder and CEO of Science Exchange and Co-Director of the Reproducibility Initiative.

Survey results indicated that several simple quality control practices could be performed more consistently. These practices include testing for mycoplasma, validating reagents for purity and identity, and screening for misidentified cell lines.

We need to have more of those difficult, but important, conversations about restructuring the research enterprise at all levels

'We need to have more of those difficult, but important, conversations about restructuring the research enterprise at all levels, from the research conduct of an individual scientist to the edicts, funding, and grant requirements from the NIH and other organisations worldwide. Then we must actually put in place the necessary resources to accommodate reform,' said Barbara Slusher, Co-founder and President of ADDC.

But solving the problem of irreproducibility will not happen overnight and will require a concerted effort from researchers, universities, funding agencies, publishers and industry, said Sean Muthian, Director of Strategic Marketing and Collaborations at Sigma-Aldrich.

The 2014 State of Translational Research Survey Report chronicles additional viewpoints and practices within the academic translational research community, such as scientific journals' role in ensuring reproducibility, data or protocol documentation and study retractions.

You may also like