Predicting motor, cognitive & functional impairment in Parkinson's

Introduction

Significant heterogeneity in Parkinson's influences clinical presentation, progression, medication response, and disease complication risk. The Oxford Discovery and Tracking Parkinson's cohorts provide around 2500 community‐ascertained patients, prospectively followed from early diagnosis, in whom these phenotypic variations can be studied.1 We used data‐driven approaches to identify fast and slow motor progressor subtypes, with differences akin to the minimally clinically important 3‐point difference on the Movement Disorders Society Unified Parkinson's Disease Rating Scale (MDS‐UPDRS) that neuroprotective treatment trials are often powered to detect.1, 2, 3 While such differences are observed in cohort studies, individualized predictions remain challenging.

Disability in Parkinson's is mainly determined by the onset of postural instability, falls, freezing of gait, and dementia.4, 5 The time to reach these disease milestones varies considerably, leading to increased outcome variation and requiring larger sample sizes to demonstrate potential treatment effects.6 A number of models have aimed to predict these clinically relevant outcomes. A 3‐step falls prediction tool by Paul et al. attached the greatest weighting to whether individuals reported falling at baseline, yet limited numbers prevented the prediction of the onset of falls in those without falls at baseline, an outcome of greater interest to the treating clinician.7, 8

Ehgoetz et al. recently reported a logistic regression model utilizing the Hospital Anxiety and Depression Scale (HADS) and the Freezing of Gait (FOG) questionnaire total score to predict the onset of future freezing, which requires external validation.5 Velseboer et al. described a similar model utilizing age, the numbers of animals named in a verbal fluency task and the UPDRS axial score to predict a composite adverse outcome but it was not possible to distinguish between death, dementia or postural instability.9 Models to date have relied on combinations of different clinical questionnaires and assessments, requiring time and skill to administer, in order to make specific predictions; to the best of our knowledge no single test has been able to predict multiple future clinical outcomes.

A multi‐device study is being planned, that uses smartphones to capture questionnaire data and to store tremor data recorded by smartwatches, alongside tablet‐based assessments, with the aim of differentiating Parkinson's from Essential Tremor.10 However so far, studies using smartphones alone have focused on equipping the clinician; in distinguishing individuals with and without Parkinson's11, 12 and working to derive smartphone scales with which to measure disease severity.13, 14 Our aim was to use smartphone data to predict outcomes of direct clinical relevance to people with Parkinson's and clinicians, alike.

Methods

The Oxford Parkinson's Disease (PD) Centre Discovery study15 is a longitudinal cohort study that recruits participants with early Parkinson's who fulfil the United Kingdom PD Brain Bank criteria for probable PD.16 Continued participation depends upon participants being ascribed a probability of Parkinson's of at least 90% by trained researchers at their latest clinic assessment.17 Antecedent approval was granted by the local research ethics committee, adhering to national legislation and the Declaration of Helsinki. Participants provide written informed consent to participate.

Since 2014, participants have been invited to perform smartphone assessments. We analysed smartphone tests performed at 18‐monthly clinic visits and up to four times a day for a maximum of seven days at home, within a 3‐month period of their clinic visit (see Fig. 1 for device details).18 Smartphone tests assess: (1) Voice (participants hold the phone to their ear, take a deep breath, and say “aaah” at a comfortable and steady tone and level, for as long as possible); (2) Balance and (3) Gait (with the phone in a trouser pocket or arm band, participants stand still and then walk a distance of 20 yards before turning and walking back); (4) Dexterity (participants tap alternately between two buttons on the screen at a comfortable rate); (5) Non‐cued reaction time (participants press on a button as it appears on the screen, keeping their finger down whilst it is there and lifting their finger off as it disappears); (6) Rest and (7) Postural tremor (participants hold the smartphone in the hand most affected by tremor if they have tremor, or their dominant hand if they do not, while their hand is at rest or held outstretched in front of them).

The seven smartphone tasks take 6–7 min in total to perform. All seven tasks constitute one smartphone recording. Incomplete recordings, where all seven tasks were not performed within a 15‐min time period, were excluded from analysis.

Clinical data collected at in‐person 18‐monthly longitudinal clinic visits were matched to smartphone recordings performed in clinic and at home within 3 months of the clinic visit; henceforth referred to as a time window. Smartphone recordings contributed at different time windows, related to different longitudinal clinic visits, were treated independently for the purpose of analysis. Smartphone recordings analyzed were collected between 8 August 2014 and 7 November 2017.

Future outcomes were defined according to the results of clinical assessments and questionnaires performed at the next 18‐month clinic visit as detailed in (Table 1) and included the new onset of (1) falls (>1 self‐reported fall in the preceding 6 months), (2) freezing (a freezing frequency of at least “about once a month” on the FOG questionnaire19), (3) Postural instability (Hoehn and Yahr20 stage ≥ 3), (4) Cognitive impairment21 (a Montreal Cognitive Assessment (MoCA)22 score < 26), (5) Difficulty doing hobbies (MDS‐UPDRS23 part II item 2.8 score ≥ 3 indicating major difficulty or an inability to do activities of enjoyment) and (6) the self‐reported need for future help at home. Individuals who had already reached a given milestone at the time of their smartphone recording were excluded from the respective analysis. Similarly, smartphone recordings missing the clinical data necessary for the assessment of the outcome definition or meeting exclusion criteria were excluded. All assessments were done on existing medication.

Altogether 998 statistical features were extracted from each smartphone recording as previously described.24 These features help characterize different motor symptoms associated with Parkinson's. Typical examples of such features include variation in speech frequency and amplitude, degree of turbulent noise in speech due to incomplete vocal fold closure, tapping speed and rhythm, mean reaction time, changes in body motion (such as detrended fluctuation analysis) and association between the tri‐axial accelerometer sensor data (such as mutual information). Statistical analyses were performed using Matlab® software (R2018a; Mathworks®, USA).

Smartphone features were used to train machine learning algorithms (random forests) to predict each binary clinical outcome of interest. Random forests is an ensemble learning technique that has commonly been used for a range of classification and regression applications.25 We chose random forests as they are relatively robust to outliers and noisy features. Moreover, in previous smartphone‐based studies, random forests have been shown to be useful in distinguishing participants with RBD and Parkinson's from controls,18 and in predicting MDS‐UPDRS part III motor examination scores.12 To try and avoid model overfitting, we used cross validation (CV) which involves splitting the dataset into non‐overlapping training and test sets. The training data was used for learning the underlying patterns of interest; the test dataset for validating the model prediction accuracy. CV helps assess the generalizability of the model to previously unseen datasets. To test the accuracy of predictions, we used two common methods of CV: (1) 10‐fold CV, and (2) leave one subject out (LOSO) CV.

Machine learning is also sensitive to the proportion of data with and without the clinical outcome of interest that is used to train the model. If there is an imbalance in outcome between groups, for example, where significantly more recordings from people without falls are used to train a falls prediction model, high prediction accuracies may simply reflect the prevalence of the outcome of interest in the dataset. Data were therefore balanced in a 1:1 ratio prior to the training of algorithms for both CV schemes by randomly under‐sampling the majority class.

For 10‐fold CV, to balance the dataset, an equal randomly selected sample of recordings from participants without the outcome of interest was merged with recordings with the outcome of interest, assuming independence between recordings and ignoring within person correlations. The recordings were randomized and partitioned into 10 separate folds. In turn, each fold was set aside to form the test set, on which the accuracy of the model, trained using the remaining mutually exclusive 9 folds, was tested. Ten repetitions of 10‐fold CV, each with randomly selected balanced recordings, were performed. Similarly, datasets were randomly balanced by outcome for LOSO CV, where only the single clinic recording performed by participants at their clinic visit was analysed; all home recordings were excluded. Home recordings were excluded from LOSO CV in order to allow for a direct comparison to be made with existing logistic regression models (detailed below); a single set of clinical data was matched to each smartphone recording performed in clinic for LOSO CV, with the clinical data also being fed into existing logistic regression models.5, 9 LOSO CV involves each recording in turn forming the test set, with the rest being used to train the model. Ten repetitions of LOSO CV, each with randomly selected balanced datasets, were performed.

Following both CV schemes (see Table S1 for description of modelling algorithm), the predicted probability of each recording belonging to either group was compared to the known future outcome. To summarize the accuracy of predictions we calculated area under the curve (AUC) values.

A level of redundancy is likely among the 998 smartphone features, with some being more important for the prediction of certain outcomes than others. Feature ranking was obtained using random forests (employing only the training data that was balanced at each LOSO CV iteration). The assigned predictor importance was averaged across all trained models for a given outcome prediction, to allow the ranking of features in order of importance. We then trained models using the top 30 smartphone features.

In order to gauge the validity of our machine learning approach and the strength of our smartphone features, we applied two existing logistic regression prediction models to our clinical data and assessed the accuracy of their predictions of (1) future freezing5 and (2) a composite future adverse outcome of dementia, postural instability, or death9 against those made using smartphone features, random forests and LOSO CV. Following personal communication with an author of the paper, direct MDS‐UPDRS values for items 3.9, 3.13, 3.10, and 3.12 were substituted for UPDRS items 27‐30 to form the axial score. Future freezing and postural instability were defined as before (Table 1). Dementia was defined according to the presence of both objective cognitive impairment (MoCA ≤ 20) as well as functional impairment (Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE)>52).21, 26

Results

A total of 1842 recordings performed by 237 participants at 268 time windows were included in the analysis of one or more outcomes (Fig. 2).

Discussion

Prediction of disease progression and their milestones is a challenge for clinicians and patients alike. We describe for the first time the innovative use of a 7‐min smartphone test in isolation to predict future clinical and functional change in early Parkinson's. Using a simple smartphone test, we can predict the onset of disease milestones including falls, freezing, postural instability, and cognitive impairment 18 months before occurrence with high levels of accuracy. Predictions are made on an individual basis using data from smartphone tests alone, without the need for clinical examination scores or patient questionnaires. A thorough evaluation of our machine learning approach is performed using two CV schemes; (1) 10‐fold CV and (2) LOSO CV as well as (3) comparison with existing logistic regression models. AUC values exceed 0.90 for 10‐fold CV using all 998 smartphone features and 0.75 with a reduction to 30 features, using either CV scheme (see Table 1). Performance is comparable with/ significantly better than that achieved with existing prediction models which use combinations of different clinical questionnaires and assessments, requiring time and skill to administer (see Table 2).5, 9, 19, 27, 28

These promising early results have the potential to provide people with Parkinson's and those partnering in their clinical care with greater information about their disease course, identifying those at risk of developing a future adverse outcome. In the age of limited clinical resources and an aging population, it is conceivable that such risk stratifying tools may assist in remote healthcare delivery, allowing people with well controlled Parkinson's to be seen in person less frequently while those flagged as being at risk of an adverse outcome could be offered closer clinical surveillance. The cost effectiveness of such an approach remains to be elucidated, but those at risk of falls or postural instability could be referred for a falls risk assessment to identify proactively falls risk factors and modify these where appropriate.29 The earlier detection of those with freezing may also allow medication adjustments to be made or cognitive training to be considered, with potential improvements in quality of life.30, 31

Author Contributions

CL, SA, FB, YBS, and MH involved in study concept and design. CL involved in drafting of the manuscript; All authors involved in data acquisition, analysis, or interpretation and critical review of the manuscript.

Conflicts of Interest

Christine Lo: Oxford Biomedical Research Centre; Siddharth Arora: University of Oxford; Fahd Baig: Oxford Discovery; Michael Lawton: Oxford Discovery; Claire El Mouden: Oxford Biomedical Research Centre; Tom Barber: Wellcome Trust Doctoral Training Fellowship; Claudio Ruffman: None; Johannes Klein: Oxford Health NIHR Biomedical Research Centre; Peter Brown: Medical Research Council (MC_UU_12024/1); Yoav Ben‐Shlomo: University of Bristol; Maarten de Vos: Oxford BRC, EPSRC, BBSRC, Roche; Michele Hu: Parkinson's UK Monument Discovery Award, Oxford BRC, University of Oxford, National Institute for Health Research, Michael J Fox Foundation, H2020 European Union, GE Healthcare, PSP Association. Consultant for Biogen and Roche Advisory Boards. Parkinson's UK: Targeting the pathological pathways to Parkinson's and Understanding the early pathological pathways in Parkinson's Disease

Supporting information

Funding Statement

This work was funded by Parkinson's UK grant .