Pharma R&D Today
Ideas and Insight supporting all stages of Drug Discovery & Development
Quality, Reproducibility, and Integrity in Early Research
Posted on November 30th, 2016 by Dr Andrew A. Parsons in Pharma R&D
It is sad to hear about more paper retractions regarding early stage research with the R&D industry (1). Perhaps not surprisingly, the number of scientific publications being retracted for fraud and error has increased in recent years, at a faster rate than the rate of increased publications (2). This does not show the industry or academia in a good light, and undermines the confidence that people place in the industry and science generally.
The good news is that journals and their editors appear more vigilant and able to retract articles appropriately. However, the issues regarding fraud or misconduct seem only a minor contributing factor to the low reproducibility of preclinical studies. This lack of reproducibility in early research is a problem for not only the funders of research but also those innovators aiming to translate the application of science into new healthcare products. A recent review estimates irreproducibility rates between 51% and 89% of all studies (3). This means that approximately $28B/year is spent on preclinical studies that are not reproducible in the U.S. alone (3). Stated in this manner, this is clearly an issue that needs resolution.
There are a number of strategies that can be developed to manage these issues from the perspective of a single company and the research system as a whole. As a research leader, our team spent a considerable amount of time “validating or re-testing” potential license opportunities to ensure reproducibility of studies. At predetermined development milestones, one can also introduce criteria to ensure reproducibility. For example, external or other independent groups conduct validation experiments. As suggested by Freedman and colleagues (3), we can also improve training in terms of study design and standardization of biological reagents and materials. It is interesting that as soon as an agent is selected for development, there comes into play a range of standards involving good laboratory and manufacturing practices that bring rigor into all the preclinical activities. These are missing in early research, and as an early research leader, I often saw these standards as added bureaucracy which would slow down the intensive iterative process of finding the “development candidate”. There are different mindsets involved in identifying the best (at a specific time point) candidate for development (finding one in many) to how to ensure safety and efficacy of the particular agent in development. This is not an over-the-wall approach to gatekeeper checkpoints, as often key attrition assays (e.g. for toxicity) are brought early into the critical path to de-select inappropriate candidates. It is perhaps more a reflection of the speed at which progress is tracked.
However, with the dramatic increase and de-centralization of early research activities, perhaps it is time to re-address the balance. Clearly, better training and study design will make an impact, as well as having standardized biological materials and reference materials. Developing and applying industry standards to ensure the heritage and identity of materials, such as cell lines and types, will be quick wins. However, will this really introduce a change in culture?
Culture emerges around the processes we put in place organizationally. Perhaps one approach that could be developed is from the quality systems with pharmacovigilance and product manufacturing. This is the role of the Quality Person. The Quality Person is a named role in European regulation, where the individual is accountable for ensuring that companies meet legal obligations for the safety reporting and surety of the drug product. Perhaps it is time to be really specific on the early research standards we wish across industry and academia and develop the role of an early research Quality Person?
Traditionally, this role of ensuring the quality of the research conducted has been part of the role of Principal Investigator in studies or perhaps the Research Leader/Manager. The role of the Quality Person brings an independence and the chance to see things with new eyes that can really focus quality attributes in the system. The role in early research will almost certainly be different to that of vigilance or product manufacture. Appropriate standards and roles will need development. However, saving at least part of the costs of supporting non-reproducible studies would seem a good potential payback.
- Steen, Casadevall, & Fang (2013). PLoS ONE 8(7):e68397.doi:10.1371/journal.pone.0068397
- Freedman, Cockburn, Simcoe (2015). PLoS Biol. 13(6):e1002165.doi.101371/journal.pbio.1002165
See more posts on this topic:
All opinions shared in this post are the author’s own.
R&D Solutions for Pharma & Life SciencesWe're happy to discuss your needs and show you how Elsevier's Solution can help.
Dr Andrew A. Parsons
Director of Reciprocal Minds Limited & Chairman of Pharmasum Therapeutics AS
- Webinar: Using machine learning to identify adverse events from scientific literature
- Managing Data and the Digital Revolution with Emerging Technologies
- Artificial Intelligence Comes to the Life Sciences
- Overcoming Commercial, Technical & Regulatory Challenges in Drug Discovery
- Seeing Research Through a Scientist’s Eyes