Prediction of cognition in Parkinson’s disease with a clinical-genetic score: a longitudinal analysis of nine cohorts

Study Design and Participants

Figure 1 shows a summary of our workflow. Table 1 describes the cohorts. Details are available in the appendix. 3,200 patients with PD were longitudinally assessed with 27,022 study visits in nine cohorts^2,7–13 from North America and Europe between 1986 and 2016 (table 1 and appendix figure 1). 96.3% of study visits occurred within 12 years of follow-up with a median follow-up time of 6.4 years from disease onset (inter-quartile range, 4.6 years) (appendix figure 2). We thus focused our primary analysis on the 12-year time frame from disease onset. Serial Mini Mental State Exam (MMSE) scores¹⁴ were longitudinally collected in seven cohorts. Montreal Cognitive Assessment (MoCA)¹⁵ scores were collected in two cohorts and converted to MMSE scores according to a published formula¹⁶. To ensure consistency across studies, an MMSE score with the cutoff of ≤ 25 was defined as an indicator of significant global cognitive impairment as recommended by the International Parkinson and Movement Disorders Society (MDS) Task Force¹⁷. Cohort-specific definitions of Parkinson’s disease dementia were used (appendix table 1). The MDS-Unified PD Rating Scale (UPDRS) part II and III scores were obtained from four cohorts, and estimated for the five remaining cohorts using the UPDRS or SPES/SCOPA-motor scales¹⁸ based on published conversion formulas^19,20 (table 1). Hoehn and Yahr scales (HY)¹⁴ were longitudinally collected in all cohorts. Ethnicity was self-reported. GBA mutations were retrospectively genotyped and defined as in Ref.⁴. For several cohorts this analysis evaluated previously collected longitudinal phenotypic data; for the active HBS, PPMI, PDBP, DIGPD cohorts both retro- and prospectively collected longitudinal data elements were included. Because stratification of patients in clinical trials is a potential application of the prognosis score, we a priori assigned the two trial cohorts, DATATOP and PreCEPT, together with PPMI to the validation population (figure 1). Written informed consent for each cohort was obtained from the participants under the supervision of each local ethics committee. The Institutional Review Board of Partners HealthCare approved the current analyses.

Building the Cognitive Risk Score

The discovery population was used to evaluate candidate clinical and GBA risk factors and to build the predictive score using a statistic similar to the Framingham Cardiovascular Risk score²¹. A step by step calculation of the score is included in the appendix table 4, figure 6. Nine clinical and genetic risk factors previously associated with dementia in PD were considered for evaluation based on prior evidence^2,3 and applicability to our datasets: GBA mutation status, age at onset of PD, gender, years of education at baseline, baseline MMSE, MDS-UPDRS II, MDS-UPDRS III scores, Hoehn & Yahr stage, and baseline depression status. GBA was included because of unequivocal evidence from multiple, large longitudinal studies (e.g.^4,5). Other candidate progression loci are controversial and need further longitudinal replication⁶. The Cox proportional hazards statistic was used to estimate the influence of these risk factors on time (years from PD onset) to reaching the endpoint of global cognitive impairment (i.e., duration of diagnosed PD illness at point of cognitive impairment) during longitudinal follow-up in the discovery population. The Cox regression coefficients, which were incorporated into the cognitive predictive score, each index the hazard rate throughout the time period analyzed, which is assumed to be constant throughout that period. This “proportional hazard assumption” was valid for all predictors except the MDS-UPDRS II, which was discarded for this reason. The eight remaining risk factors were entered into a multivariable Cox model. Backward elimination was performed to remove uninformative variables from the model based on the lowest Akaike’s information criterion (AIC). Hoehn & Yahr stage was eliminated from the model during this process (AIC of 2,088.2 without HY vs. 2,089.5 with HY). The final multivariable Cox regression model then included the remaining seven risk factors (figure 1). To adjust for differences among the seven cohorts, a “cohort” term was included as a random effect (using a “frailty” Cox model). The score was then validated in 1,132 patients with 19,127 follow-up visits over 8.6 (median, 6.5; IQR 3.1) years. Cochran’s Q-test and the I² index were used to test for heterogeneity across studies (appendix table 2). In all analyses, p values less than or equal to 0.05 were considered statistically significant.

Application to Dementia

Dementia is characterized as a loss of cognitive ability severe enough to interfere with normal activities of daily living¹⁷. We conservatively tested dementia prediction in individuals with PD (without cognitive impairment at enrollment) in the validation population that was not used to build the predictive score. 1,132 patients with 19,127 longitudinal study visits were available for this analysis.

Stability

To test the stability of the predictive score we rebuilt and retested the score in 10,000 training and test sets randomly generated from the entire study population.

Generalized Mixed Random and Fixed Effects Longitudinal Meta-analysis (LMM)

To evaluate longitudinal trajectories of serial MMSE and MoCA scores in patients with high (> 0.196) vs. low cognitive predictive scores (≤ 0.196) at enrollment, we performed LMM adjusting for disease duration at enrollment using appropriate cohorts from the validation population.

Detailed methods, statistical analyses, supplemental results, and illustrative case studies are shown in the appendix.

The Online Calculator can be found at http://scherzerlaboratory.org/tools/login.php

Clinical-genetic score and prediction of cognitive decline in the discovery population

3,200 patients with PD from nine cohorts were screened for eligibility. 235 patients were excluded because of a MMSE ≤ 25 at baseline; 135 were excluded because their first study visit occurred more than 12 years from disease onset. Six cohorts representing 1,350 patients (after exclusion of an additional 334 patients with missing covariates) with 5,165 longitudinal visits over 12.8 (median, 2.8; interquartile range (IQR) 3.1) years) were assigned to the discovery population (figure 1): HBS, CamPaIGN, PICNICS, PROPARK, DIGPD and PDBP. The proportional variance in the cognitive predictive scores accounted for by the model as a whole was 97.4%. Each of the seven predictors included into the model significantly contributed to the information content of the score. Age at PD onset was responsible for 56.5% of the variance, followed by MMSE at enrollment (7.7%), years of education (5.4%), MDS-UPDRS part III score at enrollment (4.7%), gender (2.6%), depression at enrollment (1.9%), and GBA carrier status (1.5%).

Our cognitive risk score showed high accuracy (quantified by AUC estimates) for predicting, whether a patient will develop global cognitive impairment within ten years from disease onset. In the discovery population, the AUC was 0.863 (95% CI 0.821–0.901; figure 2A); with a specificity of 0.717 (95%CI 0.652–0.775) and a sensitivity of 0.865 (95%CI 0.802–0.918) at the optimal cutoff (0.196). Patients with cognitive predictive scores in the highest (fourth) quartile had a dramatically increased hazard ratio (HR) for global cognitive impairment of 21.6 (95% CI, 10.9 – 42.9) compared to those in the lowest risk quartile (the reference quartile; p < 0.0001, table 2). Kaplan-Meier survival curves of subjects in the highest and lowest risk quartiles, respectively (figure 2C), revealed that 95.8% (95% CI 92.7%–99.1%) of patients in the lowest quartile of predictive scores survived for ten years without global cognitive impairment in contrast to only 34.9% (95% CI 26.5%–46.2%) in the highest quartile (a 60.9% difference; p < 0.0001 log-rank test; figure 2C).

Prediction of cognitive decline in the validation population

The score built in the discovery study (consisting of the seven predictors and β coefficients from the discovery study) was locked-in and applied “as is” to the new patients. Similarly, the quartile ranges and the optimal cutoff identified in the discovery study were rigorously applied to the validation population. The predictive score was accurate in the validation population with an AUC of 0.854 (95% CI 0.779–0.913) (figure 2B). Specificity was 0.744 (95% CI 0.604–0.868) and sensitivity was 0.733 (95% CI 0.617–0.831) at the pre-set cutoff of a score of 0.196. Adjusting for a 26.7% prevalence of Mild Cognitive Impairment in non-demented patients with PD patients²² yielded a negative predictive value of 0.884 (95% CI 0.818–0.923) with a positive predictive value of 0.510 (95% CI 0.400–0.639). Patients with predictive scores in the highest (fourth) quartile had a substantially increased HR for global cognitive impairment of 18.4 (95% CI, 9.4 – 36.1) compared to those in the lowest risk quartile (table 2). Kaplan-Meier survival curves of subjects in the highest and lowest quartile of predictive scores, respectively, in the validation population are shown in figure 2D. 96.3% (95% CI 94.1% – 98.6%) of patients in the lowest quartile of predictive scores survived for ten years without global cognitive impairment while only 27.4% (95% CI 12.6%–59.8%) of patients in the highest quartile of predictive scores survived for ten years without global cognitive impairment in the validation population (a 68.9% difference; p < 0.0001 log-rank test; figure 2D).

Prediction of dementia

Cohort-specific definitions of a clinical diagnosis of PD dementia (PDD) were employed as described in the Methods. The AUC for predicting dementia within ten years from onset was 0.877 (95% CI 0.788–0.943) — even higher than for global cognitive decline (figure 3). At the cutoff of 0.196, dementia was predicted with a sensitivity of 0.861 (95% CI 0.716–0.944) and specificity of 0.721 (95% CI 0.594–0.841). The negative predictive values for dementia ranged from 0.920 (95% 0.877–0.954) to 0.941 (95% CI 0.919 – 0.961) based on systematic estimates of prevalence of dementia amongst patients with PD ranging from 31.1% (high quality studies)²³ to 24.5% (all studies)²³. Patients with predictive scores in the highest (fourth) quartile had a substantially increased HR for dementia of 21.9 (95% CI, 6.5 – 73.1) compared to those scoring in the lowest quartile (table 3). 98.9 (95% CI 97.6% – 99.9%) of patients in the lowest quartile of predictive scores survived for ten years without dementia while only 48.3% (95% CI 21.3%–62.8%) of patients in the highest quartile of predictive scores survived for ten years without dementia (a 50.6% difference; p < 0.0001 log-rank test).

Stable predictive accuracy in 10,000 re-sampled training and test sets

In 10,000 iterations, age at onset, enrollment MMSE score, and years of education remained in the model after stepwise pruning in 100% of iterations, enrollment MDS-UPDRS III in 98.30%, GBA carrier status in 91.79%, depression in 90.61%, and gender in 78.52% (figure 4A). HY stage (which did not make it into our clinical-genetic score) was added to the model in 34.86% of iterations. Across 10,000 re-sampled test sets, the mean AUC was 0.833 (95% CI, 0.785–0.876) consistent with stable performance. Moreover, the score stably and accurately predicted dementia across the 10,000 re-sampled training and test sets with an average training AUC of 0.879 (95% CI 0.837–0.916) and an average testing AUC of 0.872 (95% CI 0.817–0.920) (figure 4B).

Prediction of longitudinal trajectories in MoCA scores

PD patients with high enrollment predictive scores had a significantly more rapid longitudinal decline in MMSE scores over time with p < 0.0001 compared to the patients with low enrollment predictive scores (figure 5A). The MoCA test may be more sensitive than the MMSE for detecting cognitive change in patients with PD¹⁵. We analyzed the longitudinal trajectories of MoCA scores in PPMI and PreCEPT, two of the validation cohorts, which have included this scale into their assessment battery. Patients with high predictive scores at enrollment had a much steeper decline in MoCA scores over time compared to patients with low predictive scores with p < 0.0001 (figure 5B).

Improving power in a hypothetical clinical trial

We performed a power analysis to estimate sample size requirements for a three-year clinical trial of a hypothetical drug designed to halt cognitive decline (as measured by serial MoCA or MMSE, respectively) in a trial of patients predicted to be at high risk of cognitive decline based on an enrollment cognitive risk scores at or above the cutoff of 0.196. Sample sizes of 137 (152) per placebo and 137 (152) per drug group were sufficient in order to yield 80% power for serial MoCA (or MMSE assessments). By contrast, if instead unselected patients with PD were enrolled to test the same experimental drug (over the same time period, assuming the same α, same standard deviation, and same test-retest correlations), 801 (802) patients would need be enrolled for each of the active treatment and placebo arms to achieve 80% power for serial MoCA (or MMSE assessments) (figure 5C). Detailed methods and two illustrative case studies are shown in the appendix. Medications inhibiting acetylcholine esterase activity (AChE) can be used to enhance cognitive function in patients with PD and could conceivably inflate MMSE scores. However, we found no statistically significant influence of AChE inhibitors (rivastigmine, donepezil and galantamine) on MMSE scores in the three cohorts with pertinent medication information (PPMI, PDBP, HBS), neither at enrollment nor during longitudinal follow-up.

For longitudinal analyses, examining the performance and stability of the prediction accuracy for various time frames (not just the ten-year time frame) using time-dependent ROC curves is of interest. For our score, the time-dependent prediction accuracy (as measured by the incident/dynamic average iAUC²⁴) was stable at various time points (from 1 to 11 years from disease onset; appendix figure 4). It somewhat degraded during year 12, likely due to the relatively low number of patients who completed 12 years of follow-up.

Collectively, the clinical-genetic cognitive predictive score was robustly associated with both binary and continuous longitudinal cognitive outcomes, including global cognitive impairment, level 1 dementia, and decline in MMSE and MoCA scores over time.