Comparing biological markers of Alzheimer's disease across blood fraction and platforms: Comparing apples to oranges

1. Introduction

Despite tremendous scientific advancements, there remains a significant concern regarding the lack of reproducibility of research findings [1], [2], [3], [4] with most believing that “at least 50%” of academic findings will not be replicable within industry laboratories [4]. In fact, the National Institutes of Health recently highlighted this problem and outlined a plan to address the issue [2]. In recent years, there has been an explosion in the search for blood-based biomarkers related to Alzheimer's disease (AD) for a variety of functions, such as detection, diagnosis, risk estimation, as well as clinical trial enrichment, stratification, and treatment response. However, this work has not been immune to the problem of replicability as conflicting findings are commonplace in the field. In an effort to generate consistent methods and protocols to increase replicability and move the field of blood-based biomarkers for AD forward, the international collaboration of the blood-based biomarker professional interest area (BBB-PIA) of the Alzheimer's Association's International Society to Advance Alzheimer's Research and Treatment was formed, which has published consensus statements regarding the current state of the field along with most of the immediate research needs [5], [6]. More recently, the BBB-PIA published the first ever consensus-based guidelines for preanalytic processing for blood-based AD biomarker research [7]. The purpose of the present study was to examine two potential sources contributing to failures to replicate in the blood-based biomarker field of AD, (1) blood fraction (i.e., serum vs. plasma) and (2) analytic platform. These initiatives have been of paramount importance and additional topics require careful consideration.

A major concern for blood-based AD biomarker studies is the selection of the most suitable blood fraction. The type of blood fraction is important not only for the abundance of specific analytes but also for the role of additives such as heparin, citrate, or ethylenediaminetetraacetic acid (EDTA), which can significantly impact both stability and detectability of biomarkers [8], [9]. However, to date, there remains little consistency in the type of blood fraction assayed across studies. One of the most extensively studied plasma-based biomarkers is amyloid β (Aβ), which is one of the hallmarks of AD pathology investigated at autopsy and is a well-validated marker of AD in cerebrospinal fluid samples. Work by Watt et al. [10], however, highlights many of the issues regarding plasma Aβ studies. Although some markers appear to be robust in both serum and plasma (e.g., C-reactive protein), other markers appear to be more robust in one fraction over the other. For example, EDTA inhibits many proteases, which may preserve many proteins better than serum; however, EDTA can interfere with some mass spectrometry assays. Recent reviews on the topic highlight the variability in blood-fraction selection as a major contributor to inconsistent findings in blood-based biomarker studies [11], [12]. On the one hand, several markers have been found to be significant across multiple studies and cohorts, despite different blood fractions used (e.g., pancreatic polypeptide [PPY] and C-reactive protein [CRP]) [13], [14], [15], [16]. Few studies, however, have directly compared plasma to serum-based findings in AD. When examining the association between serum- and plasma-based proteomics in the Texas Alzheimer's Research & Care Consortium (TARCC; available at http://www.txalzresearch.org/), a total of 40 proteins (from >100 candidate proteins) were highly correlated across blood fractions (R² ≥0.75; ≥56% shared variance of proteins) [17]. In another study using the TARCC and Alzheimer's Disease Neuroimaging Initiative (ADNI) data, only 11 proteins (from >100) were highly correlated across serum and plasma (R² ≥0.75) and significantly associated (P < .05) with AD status (CRP, adiponectin, PPY, fatty acid-binding protein [FABP], interleukin 18 [IL-18], beta 2 microglobulin [β2M], tenascin C [TNC], I309, factor VII [FVII], soluble vascular cell adhesion molecule-1 [sVCAM-1], and monocyte chemoattractant protein-1). The serum-plasma biomarker algorithm yielded an area under the curve (AUC) = 0.88 across cohorts [18]. These data suggest that some markers are consistent across blood fraction and may be useful for diagnostic purposes; however, others are likely less comparable despite statistically significant correlations.

Another key issue for blood-based AD biomarker studies is the selection of the most appropriate assay platform. Many cohorts have used the Myriad Rules Based Medicine (Myriad RBM) platform (e.g., ADNI, TARCC, and the Australian Imaging, Biomarker & Lifestyle Flagship Study of Aging) [13], [14], [16], [18]; however, many other approaches have been used, including the Meso Scale Discovery (MSD; available at http://www.mesoscale.com) [19] and SOMAscan [20] multiplexed protein technologies. Recently, several investigations have focused on identifying and validating biomarkers or biomarker algorithms across platforms [14], [19], [20], [21]; however, most studies have not attempted cross-platform validation and others have failed to cross-validate across platforms [22]. The use of different assay methodologies likely has substantially contributed to the inconsistencies within the blood-based AD biomarker field.

The present study was undertaken to directly compare serum- and plasma-based protein concentrations for putative AD biomarkers as well as data obtained from the same participants at the same blood draw using Myriad RBM versus MSD.

2. Methods

3. Results

Compared with normal controls (NC), the AD group was significantly older (P < .001), had fewer years of formal education (P < .001), and scored lower on the MMSE (P < .001) and higher on the CDR-SB (P < .001). There were no significant differences between groups with regard to sex or presence of dyslipidemia, diabetes, or hypertension. The AD group included significantly more APOE ε4 carriers (Table 1). Table 2 lists means and standard deviations of protein levels across blood fraction and assay platforms (RBM plasma data for NCs were not available).

As listed in Table 3, nearly all the markers were statistically significantly correlated across blood fraction, only sTNFR1, FABP, I309, IL-18, IL-10, IL-6, IL-5, PPY, eotaxin3, and SAA were correlated substantially high to share at least 50% of the shared variance. However, although the correlations were statistically significant for others, the amount of variance shared was less than 50% for thrombopietin (THPO), IL-7, TARC, TNF-α, alpha-2-macroglobulin, β2M, FVII, CRP, TNC, soluble intercellular adhesion molecule 1 (sICAM-1), and sVCAM-1. As an example, this implies that approximately 44% of what was measured as CRP in serum was similarly measured in plasma, whereas 66% of the measurement was error or something else.

Next, the variable importance plots from our previously generated RF analyses [19] were examined (Table 4). We previously demonstrated that the overall accuracy of the algorithm using our specific profile was superior when using serum (AUC = 0.96) versus plasma (AUC = 0.76) [19]. When examining the protein importance plots across serum versus plasma, there was minimal overlap across blood fractions in ranking among the top 10 biomarkers (of our 21-protein profile). In fact, only IL-5, IL-6, and IL-7 were consistently ranked among the top 10 biomarkers across serum and plasma.

Next, data from 17 common markers assayed using the MSD and RBM platforms were compared. As listed in Table 5, 14 of the 17 correlation coefficients are statistically significant (P < .05); however, the amount of shared variance in protein concentrations was <50% for 12 of the 17 markers and >50% only for FABP, CRP, FVII, IL-18, and β2M. Additionally, as listed in Table 4, only two of the top 10 markers (IL7 and TNF-α) were common among the top 10 biomarkers across the MSD and RBM platforms.

4. Discussion

The current findings clearly illustrate the importance of blood fraction and assay platform on obtained results. In fact, our findings highlight that a blood-based algorithm that is highly accurate in detecting AD could (and likely would) be very different if it was conducted in serum versus plasma or on an ECL versus a Luminex-based platform. Therefore, as the science currently stands, accurate blood-based algorithms for detecting AD likely have internal consistency only when performed on a specific blood fraction and by a specific laboratory. Therefore, if transition to clinical practice was the goal, the laboratory developed test (LDT) would be the only viable option. The international working group recently published guidelines for processing of blood samples when conducting work in the area of AD biomarkers [7]. The present study builds on this prior work and points to the urgent need for greater standardization if a blood-based biomarker test is to be reliable and clinically applicable for the detection of AD.

First, the selection of blood fraction is a nontrivial choice. Although there have been many blood-based biomarkers of AD identified, studies have frequently used different blood fractions. A blood-based algorithm for detecting AD in serum will likely not be the same as one in plasma. In fact, only a single study to date has published a proteomic profile that was accurate in detecting AD in both serum and plasma [18]. Importantly, blood fraction must be taken into consideration in studies examining or reviewing the state of the science. A review (or meta-analysis) on specific biomarkers that does not consider blood fraction will likely be highly uninterpretable. It is likely that an approach that takes into account both serum and plasma markers will be the most robust and reliable and should be investigated further.

When looking at platforms, the current results demonstrate that protein concentrations are not consistently comparable across platforms. This variability emphasizes the need to cross validate biomarker profiles across platforms in cross-sectional and longitudinal specimens, particularly those identified on large-scale discovery platforms. A seminal article in this field by Ray et al. [26] identified a proteomic signature that was highly accurate in detecting and predicting AD; however, the findings did not cross validate across platforms [22]. It is unlikely that a discovery-based platform will demonstrate the properties, precision, replicability, and accuracy necessary to become a LDT and, therefore, cross validation on platforms with greater precision is of paramount importance. One example of a putative biomarker that has been consistently measured across blood fractions and platforms is that of clusterin (ApoJ). Lovestone and colleagues have identified an association of clusterin with AD in genetic studies [27], using proteomics across multiple platforms [20], [21], and within primary neurons [28]. These and other evolving validation studies can offer novel insights into the pathobiology of AD and new therapeutic options. Using a serum-based profile approach, O'Bryant et al. [14], [29] identified an algorithm that was highly accurate in detecting AD on the Myriad RBM discovery platform. The algorithm was then cross validated to the MSD platform (also in serum), and across species (humans and mouse model) and tissues (serum and brain microvessels) [19]. Such steps are ultimately necessary to ensure the confidence in the biomarkers or biomarker profiles themselves.

There are limitations to the present study. First, the analyses are cross sectional in nature and, therefore, any links between blood biomarkers and disease incidence or progression cannot be assessed. Although the current sample reflects a sizable collection of serum- and plasma-based data from the same individuals at the same blood draw, larger samples are needed to validate these findings as well as examine additional markers and sources of variability. A study simultaneously examining multiple markers across multiple assay platforms would be of tremendous value to the field (across multiple neurodegenerative diseases). Such a study would allow for the validation of approaches and markers when used in combination, allow researchers to optimize specific markers for fit-for-use purposes, as well as offer a unique opportunity to take a systems biology approach to understanding neurodegenerative disease-specific versus overlapping pathologies. Additionally, our recent work shows that the link between blood-based biomarkers and disease status (AD vs. controls) and disease outcomes (i.e. cognition) varies by ethnicity [15], [30]. However, the current findings are from primarily non-Hispanic whites and may not generalize to other ethnic or racial groups. Despite these limitations, our findings strongly emphasize the need to consider blood fraction and assay platform when interpreting or comparing findings across studies to increase replicability of findings across laboratories and methodologies. Additional work is needed to directly compare biomarkers across cohorts, blood fractions, assay platforms, and stages of neurodegenerative disease to push this work closer to clinical utility.

5. Conclusion

The current findings not only point toward a significant potential source of variability across studies but they also provide further demonstration of measurement consistency in select putative AD biomarkers. CRP and PPY have been consistently touted as key biomarkers for multiple cohorts [13], [14]. It is also important to note that these more robust markers could, in fact, be contributing to the statistical significance many of the significant algorithms generated to date. If the more robust markers can be identified and validated across blood fractions and assay platforms, these efforts will most certainly move the field forward.