Defining optimal cutoff scores for cognitive impairment using MDS Task Force PD-MCI criteria

Introduction

Mild cognitive impairment in Parkinson’s disease (PD-MCI) has been increasingly recognized as a state of cognitive decline that is beyond that expected with normal aging, but does not meet dementia criteria ^{1, 2}. Although historically described with different criteria, PD-MCI is common, occurring in 20-50% of patients ^{2, 3} and may reflect a transitional state that precedes dementia in PD ^4-6. Recent efforts have focused on developing specific and standardized diagnostic criteria for PD-MCI since prior studies varied greatly in definitions used ^{2, 3, 7, 8}. As such, a Movement Disorder Society (MDS) Task Force proposed diagnostic criteria for PD-MCI in order to enhance clinical and research efforts on identifying clinical characterizations of PD-MCI, predictors of conversion to dementia, and patients who might benefit from early intervention studies, not only in single-site PD cohorts but across multiple clinical and research sites ⁹. The MDS Task Force criteria delineate inclusionary and exclusionary features for PD-MCI and provide the clinician or researcher with two diagnostic methods: an abbreviated assessment (Level I) or comprehensive assessment (Level II), which can also classify PD-MCI subtypes as single or multiple domains impaired. For Level II criteria, the MDS Task Force recommends formal neuropsychological testing that includes at least two tests for each of the five cognitive domains (i.e., attention/working memory, executive function, language, memory, visuospatial function). Impairment should be present on at least two neuropsychological tests, represented by either two impaired tests in a single cognitive domain or one impaired test in two different cognitive domains and may be demonstrated in one of three ways: performance approximately 1-2 standard deviations (SD) below appropriate norms, significant decline demonstrated on serial cognitive testing, or significant decline from estimated pre-morbid levels.

The proposed MDS PD-MCI diagnostic criteria represent an essential first step towards a uniform definition of PD-MCI across multiple clinical and research settings but await field study of applicability and validation. Several questions for Level II criteria regarding definitions of impairment on neuropsychological tests remain unanswered, including the optimal SD cutoff score below appropriate norms to use. These issues are important to investigate in the validation of the MDS PD-MCI criteria as they can affect the sensitivity of detecting PD-MCI and ultimately, the clinical care and counseling of such patients and their potential eligibility in research studies ^10-12. Studies examining the MDS PD-MCI Task Force criteria are emerging but at present, have utilized a 1.5 SD cutoff below appropriate norms in their application of Level II criteria ^{5, 13, 14}. Other PD studies highlight how different cutoff scores, number of tests, and cognitive domain representation can influence the characterization of PD-MCI ^{11, 12}. These studies, however, were published prior to the MDS PD-MCI Task Force criteria and differ in the cognitive domains and tests studied. Accordingly, the aim of our study was to examine one of the unresolved issues raised during the development of the MDS Task Force criteria, namely optimal SD cutoff scores for determining impairment and its effect on PD-MCI characterization and subtype.

Subjects and Methods

Results

Discussion

Our study demonstrates several findings. First, the MDS PD-MCI Task Force Level II diagnostic criteria can be applied in clinical research settings and provide a reliable diagnosis of PD-MCI, with good concordance with PD-MCI classification by consensus diagnosis criteria. Second, PD-MCI impairment was best defined using a cutoff of 2 SD below norms in our cohort. Third, multiple domain impairment was more common than single domain PD-MCI, using all SD cutoff scores, with predominantly executive dysfunction.

This study represents a first step in the application and validation of the MDS PD-MCI Level II criteria and in addressing one of the criteria’s unresolved issues, specifically SD cutoff scores used to define neuropsychological test impairment. Compared to other SD cutoff scores, the 2 SD cutoff score below norms provided the best discriminative properties for classifying subjects as PD-MCI with good concordance with our consensus PD-MCI classification. While most studies of MCI, either in PD or non-PD populations ^{10, 39-41}, have utilized 1 SD, 1.5 SD, 1.96 SD, or 2 SD cutoff scores, we extended our score analyses to a lower limit of 2.5 SD below norms, and thus, on the other extreme end, confirm that the 2 SD cutoff score below norms provides the best PD-MCI diagnostic classification. Moreover, a normative cutoff score of 1 SD may identify 16% of healthy individuals as impaired, assuming that measures are normally distributed in healthy populations, and thus, has been considered too liberal to permit meaningful specificity in the assessment of cognitive decline ¹⁰. While the MDS PD-MCI Task Force Level II diagnostic criteria recommend a broad definition of impairment ranging between 1-2 SD below norms, to avoid missing the diagnosis of PD-MCI in high-functioning people, our study suggests that the 1 or 1.5 SD cutoff scores are too inclusive and sacrifice specificity. Therefore, at present, we would recommend a more conservative 2 SD cutoff below norms for defining PD-MCI by MDS PD-MCI Level II criteria. As our PD cohort had a mean age of 72.7 (6.0) years and mean education of 15.3 (3.2) years, examination of this cutoff in PD-MCI cohorts with different levels of education and diverse backgrounds may provide additional insights and further refinement of the criteria.

To the best of our knowledge, our study is the first to examine this issue with the MDS PD-MCI Level II criteria. Few studies have examined the recently published MDS PD-MCI criteria, but those in the literature thus far have applied a 1.5 SD cutoff score below norms. In addition, these studies explored different issues from our study, namely the demonstration of a decline from pre-morbid levels, requirement of cognitive complaints, inclusion of 2 tests abnormal per domain ¹⁴ or the discriminatory power of individual neuropsychological tests ¹³. Other recent studies, prior to the MDS PD-MCI criteria, affirm that the issue of defining impairment cutoff scores is an important and influential one regarding the frequency and characterization of PD-MCI, with PD-MCI estimates varying from approximately 10-90% using cutoffs of 1 SD, 1.5 SD, or 2 SD below norms ^{2, 3, 11, 12, 42-44}. These studies, however, also varied in other features including number of cognitive domains assessed (range 3-5), number of tests per domain (range 2-7), number of abnormal tests per domain required (i.e., 1 test abnormal per 1 or 2 cognitive domains or 2 tests abnormal per 1 or 2 cognitive domains), educational level of participants (mean 11-16 years), and definition of PD dementia ^{1, 45}. Moreover, application of more liberal or conservative cutoff scores may have additional clinical and research significance, beyond cross-sectional frequency estimates. In studies of non-PD populations, MCI diagnostic approaches influenced estimates of MCI stability or progression to dementia, in general and by subtype categorization ^{39, 40}. This may have particular relevance in advancing our understanding of PD-MCI subtype differences and selecting PD-MCI patients for trials of therapies to halt or slow conversion to PD dementia.

Classification of PD-MCI into subtypes is important for exploring whether impairments in different cognitive domains have distinct underlying neurobiological substrates and clinical courses. In our cohort, PD-MCI multiple domain impairment was more frequent than single domain impairment, which is similar to a recent report of PD-MCI subtype distributions using the MDS PD-MCI Task Force Level II diagnostic criteria ¹⁴. This PD-MCI subtype distribution of greater multiple domain impairment persisted despite use of different SD cutoff scores in our study, though the 2 SD cutoff score classified the greatest number of subjects as having single domain impairment. These findings differ from older studies, including our prior work ³⁶, which reported greater single domain impairment, either nonamnestic ^{4, 42-44, 46, 47} or amnestic subtypes ³. Potential reasons for the greater frequency of PD-MCI overall and in multiple domain impairment include differences in PD-MCI diagnostic criteria, number of tests, number and type of cognitive domains, PD duration (prevalent vs. incident cases), and populations (clinic-based vs. community).

Of the cognitive domains impaired in our current study, executive function was most frequently affected, followed by memory and visuospatial function, at the 2 SD cutoff score. Multiple combinations of cognitive deficits occurred in PD-MCI multiple domain subtype. Distributions and combinations of domains impaired did not appear to be driven by the number of individual cognitive tests within each domain. In most PD-MCI studies, to date, cognitive domains have been variably weighted with 2-7 tests per cognitive domain ^11-13. Excessive or imbalanced number of tests per domain may bias PD-MCI classification ⁹, and fewer than 2 tests per domain also may be inadequate for diagnosis and subtyping ¹⁰. The MDS PD-MCI criteria recommend at least 2 tests per domain with impairment demonstrated on at least 2 neuropsychological tests as either 2 impaired tests in 1 cognitive domain or 1 impaired test in 2 different cognitive domains, but the optimal number of tests per domain and number of tests impaired per domain remain areas that may merit additional study. The optimal grouping of tests in domains also has been debated; some tests have potentially overlapping features (e.g., executive components of visuospatial tests), and others may be sensitive to deficits in more than one area (e.g., category fluency and frontal or temporal/posterior cortical dysfunction ^{6, 23}).

Compared to PD-NC subjects, our PD-MCI subjects had worse motor function but did not differ regarding age, education, PD duration, and medications used for motor and non-motor features (e.g., hallucinations, sleep, mood). Several studies demonstrate an association between older age and lower education levels and PD cognitive impairment ^{3, 6, 48}, but this association was not found in other studies ^{14, 42, 43, 47, 49}. Although our PD-MCI subjects were slightly older and had slightly fewer years of education than PD-NC subjects, this was not statistically significant and may relate to demographic patterns or sample size. The association of worse cognitive function and greater motor severity is in keeping with other studies, which also link these two features to shared neurobiological substrates, increased dementia risk, and earlier onset of dementia in PD ^{4, 6, 50-52}. Longitudinal follow-up studies of our PD cohort will permit examination of their cognitive and motor progression.

Our study has several notable strengths including our well-defined PD cohort, diagnoses by experienced specialists in movement disorders and neuropsychology, comprehensive clinical and neuropsychological assessments, as well as the application of the recently published MDS PD-MCI Task Force Level II criteria. Limitations include our university setting, sample size, and high educational levels, which may affect generalizability. While our cohort size is smaller than several large, community-based or multi-center PD-MCI cohorts ^{3, 14, 42, 44}, it is comparable to other PD-MCI studies in clinic-based populations ^{13, 43, 47, 53}. Educational levels of PD cohorts vary in the literature, depending on the country and setting, ranging from mean ~ 9-11 years ^{13, 42, 48} to ≥ 15 years ^{14, 43, 54}, the latter, comparable to our cohort. Future studies, however, with larger PD-MCI cohorts representing different educational levels, disease durations, and PD-MCI subtypes will be needed to further characterize PD-MCI. At present, there is no “gold standard” to validate the diagnosis of PD-MCI, and studies of this are in beginning phases. As such, in order to examine the issue of optimal SD cutoff scores, we compared the MDS PD-MCI Level II criteria to our consensus diagnosis, using a systematized approach similar to those applied in aging and dementia studies ^{38, 55}. Future studies with longitudinal follow up of PD-MCI subjects will permit the examination of stability or progression of PD-MCI and the use of the MDS PD-MCI criteria to predict PDD. These studies will help establish the sensitivity/specificity and optimal SD cutoff scores for the MDS PD-MCI criteria in predicting PDD.

We conclude that the MDS PD-MCI Task Force Level II criteria can be readily applied in the clinical research setting and that consideration should be given for using a 2 SD cutoff score below norms in the diagnosis of PD-MCI. The development of these criteria mark a first step in creating a uniform definition of PD-MCI that can be used in multiple centers and for interventional trials, and further validation of our observations in larger and diverse cohorts will be important steps in the testing and application of the MDS PD-MCI criteria.