Validation of a serum screen for Alzheimer’s disease across assay platforms, species and tissues

Introduction

Alzheimer’s disease (AD) is the most common neurodegenerative dementia with over 5 million Americans affected by the disease. Every 68 seconds an American develops AD, which is the fifth leading cause of death for those over the age of 65 [1]. AD has an annual health care cost similar to that of cardiovascular disease (CVD) and higher than cancer [2]. While death rates from CVD and cancer have declined in recent decades, death rates have steadily increased for AD [3]. However, there are currently no rapid, cost-effective means for detecting the disease, which is needed to move clinical and research efforts forward. Current state-of-the-art diagnosis of AD is obtained through specialty clinic settings incorporating medical examinations, neuropsychological evaluations, clinical blood work, and neuroimaging [4]; this process has been validated against autopsy findings [5]. On the other hand, AD is common but under diagnosed in primary care settings [5–7].

While the search for blood-based biomarkers of AD was largely unsuccessful for decades, recent work shows great promise. In a seminal study, Ray and colleagues [8] analyzed 120 plasma- proteins and developed an algorithm that accurately distinguished AD patients from healthy controls (89% accuracy) as well as accurately identified patients with mild cognitive impairment (MCI) patients who progressed to AD (81% accuracy) [8]. This study represented the first major support for the notion that an AD biomarker profile can yield excellent accuracy. However, enthusiasm waned when the findings did not cross-validate on an independent assay platform [9]. Despite this initial setback, others have continued to identify promising signals in peripheral blood suggesting that a blood-based AD screen is on the horizon [10–21].

We have analyzed serum proteins from AD and control subjects in the Texas Alzheimer’s Research & Care Consortium (TARCC) to generate [15] and refine [13] a biomarker algorithm that yielded excellent accuracy for AD identification [13,15]. Additionally, our group was the first to cross-validate a blood-based approach across independent cohorts [16], which has now been accomplished by the Australian Imaging, Biomarkers and Lifestyle (AIBL) group [20]. However, a large portion of the recent literature including those from the TARCC [15], the Alzheimer’s Disease Neuroimaging Initiative (ADNI)[16, 22], AIBL [20], Washington University St. Louis [22], and University of Pennsylvania [22] have utilized the same luminex-based multi-plex platform rather than seek cross-validation in independent assay platforms, which is the point where the Ray et al procedure failed to validate [9]. Kiddle et al [23] recently demonstrated that select blood biomarkers of AD can be cross-validated across an independent platform. The identification of a blood-based AD screening tool that can be implemented by any clinical or research laboratory across the globe would be an important advancement and is preferable to a model requiring a single isolated reference laboratory. The primary purpose of the current study was to cross-validate our prior blood-based algorithm on an independent assay platform that can be easily implemented across multiple laboratory settings. The current study was also undertaken to directly compare serum vs. plasma based biomarker algorithms. We also sought to validate the approach across tissues (serum and brain microvessels) and species (humans and AD mouse models) given the recent suggestion that biomarkers identified in human studies should be translated back to animal models for further validation [24]. Lastly, we sought to conduct preliminary analyses on the ability of the algorithm to distinguish AD from a non-AD neurological disease (Parkinson’s disease).

Methods

Results

The demographic characteristics of the AD and normal control samples are presented in Table 1. There were 150 AD subjects and 150 normal controls used for serum analysis examining levels of 21 proteins. Using random forest (RF) analysis of the serum protein levels, we found a sensitivity (SN) of 0.90, specificity (SP) of 0.90 and area under the ROC curve (AUC) of 0.96 (See Figures 1–3 and Table 2). Examination of plasma samples from the same subjects resulted in an algorithm with a much lower accuracy estimate of SN, SP, and AUC of 0.65, 0.79, and 0.76, respectively. Therefore, the remaining analyses focused solely on serum. Inclusion of age, gender, education, and APOE4 genotype into the algorithm with the RF biomarker profile increased SN, SP, and AUC to 0.95, 0.90, and 0.98, respectively (See Table 2).

The RF analysis was re-run to determine the optimal algorithm for predicting AD using the smallest number of serum biomarkers. Using only the top 8 markers from the biomarker profile (Figure 4), it yielded a SN, SP, and AUC of 0.88, 0.92 and 0.95, respectively (see Table 2). The addition of age, gender, education and APOE4 genotype increased SN, SP, and AUC to 0.92, 0.94, and 0.98, respectively.

Expression of IL-6 and TNFα was significantly higher (p<0.001) in the cerebromicrovasculature of AD patients (n=9) compared to vessels from control tissue sections (n=9; Figure 5). Similarly, a comparison between control and AD transgenic mice showed that there was a significant increase (p<0.001) in cerebrovascular expression in AD mice of IL-6 and TNFα (n=6/group; Figure 6). In the logistic regression model, IL-6 and TNFα successfully classified 100% of the human and mouse model microvessels demonstrating that these biomarkers can be combined to classify brain samples from diseased versus non-diseased humans and mice. In addition, we analyzed peripheral serum from 3XTg AD mice and control mice (n=9/group) using the Meso Scales Discovery (MSD) panel, and analyzed 4 of the top 8 markers (IL7, IL5, IL6, TNFα). Using logistic regression, 100% of the mice were correctly classified using all 4 markers. For comparison purposes, using only these 4 of the top 8 markers, the overall accuracy in our human samples was 86% in logistic regression analyses (SN = 0.85, SP = 0.87) (AD n=150, NC n=150). The overall accuracy of the SVM was 100% in discriminating AD vs. PD vs. NC with all of the individuals being correctly classified within their respective categories. Interestingly, the inflammatory and vascular markers were most significantly different between the AD and PD groups (i.e. IL7, TNFα, thrombopoietin, VCAM1) were elevated in the AD as compared to PD cases (p<0.0001 for each).

Discussion

The cumbersome nature of the current AD diagnostic process is a hindrance to both clinical care and therapeutic trials. Biomarkers of disease presence would greatly facilitate access to care as well as streamline therapeutic trial subject identification. In fact, several ongoing studies utilize neuroimaging biomarkers (i.e. amyloid-β scans) as patient selection criteria. While tremendous strides have been accomplished in the areas of neuroimaging and CSF biomarkers of AD, their limited accessibility and cost make them unsuitable for first-line screening [38]. Blood-based biomarkers are ideally suited as the first-step in a multi-stage diagnostic process, and they would greatly enhance the utility of neuroimaging and CSF biomarkers. In fact, it has been proposed that blood-based biomarkers “will most likely be the prerequisite to future sensitive screening of large populations at risk of AD and the baseline in a diagnostic flow approach to AD [38].” From a clinical standpoint, while fewer than half of the physicians interviewed in a recent study felt screenings for AD were important, the vast majority of the general public and caregivers felt such screenings are very important [39]. Additionally, the average physician visit duration in an ambulatory setting for those age 65 and above is approximately 18 minutes [40], making limited time available for even brief cognitive assessments. A blood-based screening tool would fit within the existing medical model following the state-of-the-art practices currently in wide-spread use in the areas of cardiology, oncology, infectious disease and others. Specifically, a positive blood test finding results in follow-up neuroimaging, CSF or specialty clinic examination for confirmatory diagnosis.

Here we cross-validate a blood-based biomarker profile with an independent assay platform. In the current study, a set of serum-based biomarker proteins from our prior work (using a Luminex-based platform) was assayed on an assay platform using ECL. The resulting algorithm with 21 serum-proteins alone yielded an AUC=0.96, which is substantially better than most prior findings. As has been the case in our prior work, the inclusion of demographic factors (age, gender, education, and APOE4 status) yielded an even better AUC (0.98). Using the most informative 8 protein markers, the algorithm yielded an AUC of 0.95 alone and 0.98 when combined with demographic factors. The 8 proteins in this algorithm are IL7, TNFα, IL5, IL6, CRP, IL10, tenascin C, and ICAM1. While there has been a tremendous surge in efforts aimed at identifying blood-based tools for detecting AD, many of these approaches are very laborious [19] or require specialty reference laboratories [15]. The current findings demonstrate that an 8–21 protein algorithm can yield excellent accuracy on an ECL platform that is currently in place in both academic and industry settings globally.

A noteworthy finding is the clear superiority of serum to plasma-based biomarker profiles in this study. The accuracy of the plasma-based markers alone (AUC = 0.76) is similar to our prior work using 11-proteins that spanned both serum and plasma from TARCC and ADNI [16]. There is a tremendous inconsistency in the literature as to which blood fraction should be utilized in the search for blood-biomarkers of AD; however, our work has previously shown that the particular blood fraction is an important consideration [41]. In the current study, using the same 21 proteins from the same research participants at the same blood draw, the serum-based algorithm was clearly superior to that obtained from plasma.

The pathobiologic relevance of proteins in the 8-biomarker algorithm, such as IL-6, TNFα and tenascin C, is supported by data in the current study as well as published results which show their up-regulation in the AD brain. The immunofluorescent data presented herein that demonstrate an increase in IL-6 and TNFα levels in the cerebrovasculature of AD patients, relative to controls, are in agreement with our previously published results that document elevated levels of several inflammatory proteins, including IL-6 and TNFα, in isolated microvessels from human AD brains compared to controls [42, 43]. We have previously documented in the LaFerla triple transgenic AD mouse an increase in inflammatory proteins, including IL-6 [31], and here further highlight the importance of this protein in the Tg2576 mouse model of AD. The demonstration that tenascin C, a protein associated with matrix metalloproteinase (MMP) metabolism [44], is elevated in serum from AD patients agrees with data showing an increase in MMP-2 and MMP-9 in the cerebrovasculature of AD patients [43]. Taken together, the current study provides an important link between inflammatory proteins in serum and inflammatory changes in the AD brain, in both human AD and animal models. These results suggest that the 8-protein algorithm, validated clinically, also closely reflects pathologic processes ongoing in the AD brain. Combining these individual markers into a multi-marker profile, as is done with our human-serum based approach, classified 100% of human and mouse model brain tissue as well as mouse model AD versus control using peripheral serum. Therefore, we have now validated our algorithmic approach across species (mouse and human) and tissue (serum and brain) demonstrating the robustness of the methodology.

A key need for the AD field is the identification of patients who do not have established AD (i.e. not using case-control design) as well as the identification of pre-AD. Therefore, the next step for this line of work is to examine the performance of the serum-based tool in a non-specialty clinic population, where base rates of disease presence are substantially lower. All of the major findings on blood-based biomarkers of AD have been conducted on known-case designs with expert diagnoses. If a blood-based tool is to be utilized as the first step in a multi-stage diagnostic process, it is important that the PPV and NPV be calculated within a community-based or non-specialty-based ambulatory clinic given the substantial differences in base rates of disease presence [3]. For example, if one utilizes the SN and SP of 0.94 and 0.92, respectively to calculate PPV and NPV in a dementia clinic with a base rate of AD of 50%, the resulting PPV and NPV would be 0.92 and 0.94 respectively. However, with an estimated 11% base rate of AD among those ages 65 and above [3], the same SN and SP would yield a PPV and NPV of 0.59 and 0.99, respectively. That is, a clinician would be correct nearly 100% of the time if s/he simply said none of his/her patients age 65 and above suffered from AD; however, a 41% false negative rate would be obtained. Here we held SP at 0.98 allowing a resulting decrease in SN to 0.86. With an 11% base rate, PPV and NPV were 0.84 and 0.98, respectively using the current 8-protein screener, which is excellent for a front-line screening tool. For comparison purposes, the 11% base rate was applied to several of the recently published blood-based tools, including those from the current group. As can be seen from Table 3, no prior work yields acceptable predictive power within a community-based setting. Such a clinical trial on the effectiveness of this 8-protein tool is warranted. Additional work is needed to “lock down” the pre-analytic protocols and analytic protocols to set the stage for independent verification across cohorts and laboratories.

Preliminary results suggest that the algorithmic approach can be useful in discriminating AD from non-AD neurodegenerative disease (PD). These results must be cross-validated in a larger sample preferably with multiple non-AD disease states in order for analyses to examine (1) AD-specific protein signatures, (2) non-AD protein signatures, as well as (3) general neurodegenerative disease protein signatures. Such work could have significant implications for the utility of the blood-based screening approach within non-specialty clinics as well as the search for novel therapeutic interventions for AD and non-AD neurodegenerative diseases.