Assay Validation – Are We There Yet?

Where are we with validation of assays to replace animals for product testing? The short answer: not very far.

While many innovative platforms with potential as animal alternatives have been developed, adoption into drug discovery and consumer product development programs has been slow.

A recent paper, “An evaluation framework for new approach methodologies (NAMs) for human health safety assessment”, by Stanley Parish from HESI and collaborators from industry, academia and the government, provides some insight.  Unfortunately, it appears we are still fighting the battle to shift the paradigm in product testing from “an empirical approach based on apical effects in animal studies” to an approach based on “human biology and mechanistic relevance”.

We have long argued for the need to establish mechanistic relevance of assays (see previous post here). We’ve even made specific recommendations for characterization of phenotypic assays to provide a basic understanding of what pathway mechanisms can be detected (see also here). Through this effort, we had hoped to inspire and encourage the development of some of the criteria that Parish, et al., define in their article.

But as the paper describes, we’re not quite there yet. The authors suggest that evaluation of new approach methodologies (NAMs) should be based on how they will be used or their “context-of-use”. This is certainly a practical suggestion since expectations will differ depending on whether a method is to be used in discovery or for regulatory purposes, or for application to prioritization of chemicals, hazard identification or risk assessment. The article then lays out “core principals” for NAMs. These include accuracy (based on defined standards), transparency of methods (biological and technical basis of the NAM), limitations of the NAM and domain of applicability (e.g., chemical or biological).

Following the core principles, various criteria are given for assessing the fit-for-purpose of a NAM and these criteria are then prioritized for the different contexts-of-use. For chemical prioritization, criteria around chemical applicability domain, the source and species for cells or tissues, and the assay description are given highest priority. For hazard screening, additional prioritized criteria include the quality of verification data sets, status of metabolic competence, acceptance criteria for testing, independent peer review and the endpoint or pathway for prediction. Risk assessment then adds the criteria of explanation of mechanistic basis, assay robustness, data accessibility, biological comparison with in vivo data, statistical evaluation and level of uncertainty and biological variability.

It was disappointing that the criteria of “explanation of mechanistic basis” was considered important only for the most restricted context-of-use, risk assessment. Even the “quality of verification data sets” was not given importance for all contexts of use.

Why do we think this is a problem?

With so many possible NAMs, we need proof that they are useful. Verification data sets are often used to support the use of NAMs and indeed there are efforts to develop reference chemical lists for this purpose. But often these data sets were developed using animals and fail to capture human-specific mechanisms. Coupled with the problem that human verification data are extremely limited, our confidence in NAMs based solely on verification data sets alone will be weak.

We suggest that the criteria of “explanation of mechanistic basis” be given a higher priority for all contexts of use.  Only by using mechanistic criteria can we get to where we need to go: towards the development of expert systems that combine assays and models to predict human health outcomes.

Photo by Simon Berger on Unsplash.