Fact or opinion? Ensuring the validity of biomarker measurements

Liam Heaney is a Lecturer in Exercise Metabolomics within the School of Sport, Exercise and Health Sciences at Loughborough University (UK). His research is focused on the development and application of gas and liquid chromatography-mass spectrometry assays for the measurement of biofluid biomarkers in both healthy and diseased populations. Previously, Liam graduated with a BSc (Sports & Exercise Science), MSc (Exercise Physiology) and PhD (Analytical Chemistry/Exercise Metabolomics) from Loughborough University. Following this, Liam worked as postdoctoral researcher within the Cardiovascular Translational Research Group at the University of Leicester (UK) where his research focussed on the measurement of circulating small molecule metabolite biomarkers in acute cardiovascular disease (heart failure and myocardial infarction). As a recent edition to the academic team at Loughborough, Liam’s research continues to assess clinical biomarkers across multiple disease states, as well as focusing on metabolomic analyses for sports performance, and the application of mass spectrometry in the development of novel assays for sports doping.

@HeaneyLM

Biological markers, more commonly termed as biomarkers, serve to provide data regarding biological processes with the aim to inform and direct subsequent courses of action. This is perhaps most prominent in the clinical setting where biomarkers provide direct or indirect information on the diagnosis, severity, prognosis and therapeutic monitoring of a wide range of health and disease conditions [1,2]. Although biomarkers provide the clinician with vital information that can guide treatment strategies, it is essential that both the measurement assays employed and the biological role of the markers themselves are validated as robust and reproducible to ensure the most accurate details are established. This is of particular importance in the growing era of personalized and stratified medicine, with a growing viewpoint that panels of biomarkers provide improved data for precise clinical decision making [3,4].

Clinical biomarker assays have traditionally focused on the use of immunoassay protocols for the measurement of disease-specific proteins, for example the measurement of B-type natriuretic peptides in cardiovascular disease [5]. More recently, research into the application of mass spectrometry (MS)-based biomarker measurements has increased in popularity, with a 13-fold increase in annual research outputs in this topic over the previous 15 years ([6]). The increased development of MS-based biomarker assays has allowed a wider range of molecules to be studied and has shown potential benefit for future clinical analyses. A high-profile example of this is the discovery of the microbiome-mediated metabolite trimethylamine N-oxide (TMAO) and its strong associations with cardiovascular disease [7]. This breakthrough was made by applying a non-targeted metabolomics strategy using liquid chromatography (LC)–MS, with TMAO having since been characterized as a direct aggressor of cardiovascular disease in animal studies [8,9] as well as continually demonstrating to show strong associations with poor outcome in hospitalized heart disease patients [10,11]. Discoveries such as these have prompted researchers to develop methods that focus on the analyte of interest [12,13], but also to expand into multiplex assays to simultaneously measure multiple analytes involved in related metabolic pathways, e.g. trimethylamine, choline and betaine for the instance of TMAO [14].

The discovery and characterization of novel biomarkers, such as TMAO, requires careful assay development and validation for potential implementation into routine clinical MS assays. This must ensure that the methods have the required sensitivity and specificity, are reproducible when applied by multiple users across differing instrument setups, and ultimately reflect a true value of the biomarker as present during sample collection. It is important that these processes are validated through the correct developmental testing of sample collection, storage and preparation; precise instrument setup, pre-analysis testing and regular in-analysis assessment of known quality control samples; and extensive validation experiments to determine the accuracy, precision, recovery and reproducibility of the assay. For this, the concept of designing a novel biomarker assay requires an invested amount of time, effort and intellect to ensure the best possible outcome.

In the effort to clarify and standardize the validation of biomarker assays, the United States Food and Drug Administration (FDA) and European Medicines Agency (EMA) have produced in-depth recommendation reports to guide bioanalytical scientists to develop protocols that are suitable for inclusion into routine clinical analyses [15,16]. These documents provide an array of recommendations, of which I will touch upon some of those most apparent for MS-based procedures. To begin, perhaps the biggest elephant in the room is the ability to demonstrate that the assay is suitable in a matrix-specific manner. Whilst it may be straight forward to spike analytes into a healthy matrix (e.g. plasma) that are only present if a clinical issue is present (as is the case for B-type natriuretic peptides), the characterization of small molecule metabolites that will often be present in basal levels of healthy populations is more challenging. For this reason, many researchers have attempted to create substitute matrices that include gel-filtration of plasma [13], and the use of high-salt (phosphate-buffered saline) and protein (bovine serum albumin) solutions designed to mimic plasma content [17]. However, despite having more comparable properties to the real matrix than solvent-based standards, these matrices still do not fully reflect the environment of the complete matrix and therefore it is important to carefully consider the most suitable scenario on a biomarker-by-biomarker basis and consult previously published literature for guidance [18].

Another important aspect of assay suitability is its capability to provide the same result regardless of where or when a specific sample is analyzed. The success of a reproducible assay can be improved by the application of both biochemical- and user-based strategies. For the latter, features could range from selection of the sample medium (e.g. plasma or serum) and its collection (e.g. anticoagulant used in blood tubes), to specifics in the timing of a sample (e.g. fasted/unfasted, morning/afternoon). Therefore, it is important that a clear understanding, coupled with extensive testing, of the effects of sample collection, processing/treatment and storage must be sought, with any implications of altering protocols reported and avoided where required. From a biochemical perspective, extended testing of the assay across a wide dynamic range of sample concentrations is necessary, with the general recommendation for analyses of low (close to limit of quantitation), medium (a value within the ‘normal’ analytical range) and high (an extreme level) sample concentrations to be repeated across a prolonged period of time [15,16,17]. This, along with repeated testing of extraction and analysis of individual samples, provides strong evidence for or against the technical reproducibility of the assay. Importantly for MS-based analyses, the inclusion of a stable isotopically-labelled standard of the analyte(s) of interest will improve the technical reproducibility of the assay, notably when comparing across different instrument setups (e.g. location, manufacturer and model). This affords the high specificity associated with MS when coupled with chromatography, but also provides a platform to adjust analyzed data with run-specific information about instrument sensitivity. This is particularly applicable when using highly competitive atmospheric ionization techniques (e.g. electrospray and atmospheric pressure chemical ionization) commonly applied in LC–MS, but equally as an additive in complete ionization techniques, such as electron ionization, by allowing for adjustments to be made for MS instrument response.

Once the hard work in the laboratory has been completed to develop and validate a biomarker assay, it is important to remember that the attention to detail does not stop there. Although developing a robust and reproducible assay provides confidence for accurate measurements to be obtained, if the monitoring and maintenance of the assay are not followed correctly then problems will quickly present themselves. This starts at the beginning of every day or batch where the operator must ensure that the instrument is adequately maintained and that suitable pre-analytical checks are performed. These will include assessing the condition of the analytical column, the sensitivity and specificity of the MS instrument, and the results obtained on standard quality control samples. Once all of these steps are performed and data are considered to fall within a tight pre-specified range, only then should sample analyses be initiated. Another facet to consider is the regularity in which analyte calibration experiments should be performed, and whether the inclusion of a ‘within-sample calibration’ would be of benefit. Many laboratories will feel that they have a good grasp on the time it takes for their instrument response to drift out of range of a previous calibration test, but this does not help the matter for implementing a single analytical assay across multiple instruments and laboratories that may be distributed around the world. Therefore, prior to full dissemination of methods for routine clinical testing, assessment of instrument drift should be implemented and understood across multiple sites and a rigid timescale of calibration repeats applied as standard.

To conclude, the full extent to which the importance of careful consideration of experiments performed during assay development and validation far extends beyond this brief discussion. However, a number of the most common, but frequently overlooked, protocols for ensuring biomarker assay reproducibility and validity have been noted and I recommend readers consult the previously mentioned guidelines published by the FDA and EMA [15,16]. In order to most effectively implement these assays into routine clinical use, the development of standardized MS-based assay kits would be beneficial, and strict criteria must be set to ensure a true reading is obtained to positively inform for successful translation into personalized and stratified medicine.