Let’s Talk About Validation!

In drug discovery and chemical safety research, validation lies at the heart of every assay, reagent and experiment. Validation is fundamental for life sciences research effectiveness and, according to Scannell and Bosley, through the concept of “predictive validity”, one of the most powerful drivers of R&D productivity.

There are two aspects to validation: technical and biological. Technical validation includes reagent and equipment characterization, as well as statistical considerations such as running multiple experiments with sufficient replicates to test the reproducibility and assess the variability of a particular result. Although technical validation gets the most attention (e.g. the “reproducibility crisis”, see here, here and here), biological validation, arguably, has the bigger impact on predictive validity.

Validation at the biological level controls the degree to which we can interpret results in the context of other information and knowledge. A high level of biological validation enables more connections to be made between an experimental result and other studies and findings, expanding our understanding of the biological system as a whole.  Sufficient characterization of assays increases our confidence in these connections (strengthening our priors, to use decision theory parlance) and their usefulness. There are initiatives to support reagent and cell validation (such as Force11’s Resource Identification Initiative, the Reproducibility2020 challenge from GBSI, and others), as well as efforts to improve and share information on the selectivity and utility of chemical probes (e.g. the SGC’s Chemical Probes Portal). Now it’s time to extend the concept of validation standards, particularly biological validation, to characterization of the assays themselves.

So why is now the time? Cell-based in vitro assays, particularly target agonistic phenotypic assays, have been gaining in importance for life sciences research and the subject of increasing interest for use in regulation. Phenotypic assays, including 2D, co-cultures, 3D, organs-on-a-chip and microphysiological systems, are being developed for a diverse range of applications including pharmaceutical drug discovery, alternative methods for chemical toxicity testing, and consumer products research.

For phenotypic drug discovery, in-depth biological characterization of assays has been shown to simplify the deconvolution of compound mechanisms and facilitate early triage of undesirable chemical matter. For examples of how this can work, see here for a study using DiscoverX’s BioMAP platform, and another here using 3D models from Leo Price. The process of characterizing phenotypic assays for pathway responses further contributes to our understanding of disease and tissue biology, impacting not only specific programs, but also the safety and efficacy of drug candidates in other programs.

Standardizing the biological characterization of phenotypic assays can be accomplished by adopting common validation criteria. We are proposing that assay characterization be done using a small set (10-20) of chemical probes selected to assess responses to a core set of key pathways. We have initiated a community effort to collate a set of highly curated probes (the Phenotypic Assay Validation Set) at the Chemical Probes Portal for exactly this purpose. Assays can be more readily compared, and results connected to each other, if they are characterized for their responses using the same common set of probes.

So if you are interested in contributing your feedback to this effort, check it out here!  or join the conversation at SLAS‘s Phenotypic Drug Discovery Group on LinkedIn.