Accuracy of Phenotyping of Autistic Children Based on Internet Implemented Parent Report

INTRODUCTION

Autism spectrum disorder (ASD) is a highly heterogeneous group of neurodevelopmental disorders encompassing Autistic Disorder, Asperger’s syndrome and pervasive developmental disorder—not otherwise specified (PDD-NOS). Children affected with ASD show abnormalities in communication and social behaviors and often manifest repetitive behaviors [APA, 2000]. Usually the symptoms can be observed by 18 months of age and the diagnosis predominantly occurs before age 4 years [Howlin and Asgharian, 1999; Mandell et al., 2005]. The prevalence of diagnosed ASD has increased significantly in the past decade for unclear reasons, and the current estimate is around 1 in 110 children [CDC, 2009]. Boys are affected much more often than girls at a ratio of approximately 4:1 [CDC, 2009].

Efforts to conduct large-scale investigations of autism have been historically constrained by the cost of conducting time-intensive developmental history and observational assessments which have become a research standard in the field of autism, nearly universally required by funding agencies and scientific journals. The recent development of rapid phenotyping methods, and advances in our understanding of the genetic and factoral structure of autism have raised serious questions about the relative value of traditional diagnostic assessments for autism, especially when weighed against their considerable costs and the scientific urgency of moving forward with larger-scale research efforts to advance understanding of the causes and possible treatments for these conditions.

Here, we present the current status of autism genetic research, and describe the critical role of a rapid phenotyping paradigm for developing a feasible and cost-effective approach to the next phase of gene discovery in autism. Over the past 20 years, the genetic basis of autism has been well established, as evidenced by higher concordance rate of ASD observed in monozygotic twins (~90%) than in dizygotic twins (~10%) as well as a high sibling recurrence risk [Bailey et al., 1995]. These observations have spurred considerable interest in defining the genetic basis of autism while some successes in associating autism with specific chromosomal anomalies have occurred, the vast majority of autism remains idiopathic, and conversely we expect that most gene variants contributing to autism risk have yet to be observed. Numerous linkage studies have implicated multiple genomic regions of interest, but the identification of specific genes that contribute to risk within these intervals remains largely elusive [Ashley-Koch et al., 1999; Barrett et al., 1999; Philippe et al., 1999; IMGSAC, 2001; Auranen et al., 2002; Yonan et al., 2003; Stone et al., 2004; Cantor et al., 2005; Szatmari et al., 2007; Weiss et al., 2009]. Recently, characterization of rare structural variation (e.g., copy number variation or CNV) has contributed to the progress towards the identification of genetic risk variants [Jacquemont et al., 2006; Kumar et al., 2008; Marshall et al., 2008; Weiss et al., 2008]. However, even these strong genetic findings demonstrate the clinical heterogeneity affiliated with specific genetic lesions [Bijlsma et al., 2009]. Genome-wide association analyses highlight variants of unknown function between CDH10 and CDH9 at 5p14.1 [Ma et al., 2009; Wang et al., 2009] with small effect size, and near SEMA5A on chromosome 5p15 with protective effect [Weiss et al., 2009]. Rare mutations or variants in A2BP1, NRXN1B, NRXN1B, NRSXN1A, NLGN3, and NLGN4 have been associated with autism [Jamain et al., 2003; Laumonnier et al., 2004; Yan et al., 2005; Feng et al., 2006; Martin et al., 2007; Yan et al., 2008]. Due to the now observed very high genetic heterogeneity of ASD, the discovery and validation sample sizes for genetic investigation remain unsatisfactory for identifying the majority of the genetic contribution to autism.

Most of the sample collection efforts for genetic studies so far have employed recruitment procedures that require substantial time investment per family and expensive evaluator time for individual phenotype assessment. This approach naturally limits the pacing of collection of biomaterials due to funding and evaluator time constraints, as well as logistical issues for the subjects. The largest single effort to date was implemented by the Autism Genetic Resource Exchange (AGRE), which lessened geographical constraints by deploying trained evaluators and blood draw specialists to homes throughout the US to recruit a large set of affected sibling pair families for genetic analysis [Geschwind et al., 2001]. A number of collaborative efforts have been formed and organized into a cohesive analytical group (i.e., Autism Genome Project Consortium). With these efforts, around 3,300 families with at least one affected child with autism are available for genetic analysis in North America and Europe as reflected in the largest genetic study published to date [Weiss et al., 2009]. These sample sets are proving to be largely inadequate to identify common variant associations. Further, since many of the genetic causes of ASD traits are proving to be, in each instance, rare variants, the field will require very large sample sizes to distinguish causative mutations/alleles from benign or non-causal rare variants present in the population.

Thus, we need to explore how to cost-effectively recruit tens of thousands of affected individuals and their families to enable the appropriate large-scale collection of affected individuals for genetic. Most genetic studies performed to date, especially those that aggregate data from multiple sites, rely on clinical evaluation, Autism Diagnostic Interview (ADI-R) or Autism Diagnostic Observation Schedule (ADOS) in an effort to standardize phenotypic criteria across various clinical sites. However, it is not at all clear that these currently employed time-consuming methods provide the most efficient approach to phenotyping for genetic studies of ASD. Although the use of these gold standard diagnostic instruments is often extended to address the qualitative and quantitative measurement of autistic symptomatology, item level reliability is challenging. The clinic-based or rater-based phenotypic approaches used to date need to be compared and contrasted to what could be achieved by allocating a similar level of resources for rapid, web-implemented, and much lower cost per sample phenotyping that permits the collection of at least an order of magnitude more samples for the same cost and time.

Here we assess the phenotyping approach implemented within the Interactive Autism Network (IAN), which is the largest online registry of families with an affected child with ASD in the US with approximately 10,000 autism families registered and 100–300 new families registering each month. High-quality and reliable web-based entry approaches have resulted in apparently high-quality phenotypic data, and broad access to the Internet makes the registry directly available to vast numbers of families. Specifically, IAN has implemented parent report of Social Communication Questionnaire (SCQ). Here, we sought to assess the overall accuracy of the parent provided autism diagnosis for children who are verbal per parent report and are between the ages of 4 and 17 by sampling from the IAN registrants for clinic-based evaluation with ADI-R, ADOS, and clinician best estimate. This is a critical step to use IAN for a large-scale recruitment of samples for genetic analysis. With this validation of the IAN-based parental reports, it is possible to integrate a US-wide system for the collection of blood or saliva for the generation of high-quality genomic DNA for genetic analysis in a feasible, scalable, and cost-effective fashion, greatly expanding the set of genomic DNAs available for genetic analysis [Rylander-Rudqvist et al., 2006].

MATERIALS AND METHODS

RESULTS

DISCUSSION

Here, we have assessed the accuracy of fully implemented web-based parent report within a disease-specific patient registry (IAN). We observe that even in the small sampling performed within this study of a large potential sample set, the rapid phenotyping paradigm has a very high concordance rate with the clinic-based assessments, which are considered the reasonable standards for affectation status for inclusion in genetic studies. The high concordance rates observed imply that parent reports, as implemented in the parent-centric IAN registry, are highly reliable and provide a 95% confidence of a 93.4% concordance rate with ADI-R or ADOS based clinician’s diagnosis in verbal children. These observations are generally supportive of other reports of the self-selected populations of families with a child affected by autism. For instance, the distribution of scores on the SRS of affected children enrolled in IAN is similar to the data provided from clinically defined populations with ASD [Constantino et al., 2007].

There were a few subjects (n = 11, 10.2%) who were negative by ADOS, but all of them were positive for ASD based on ADI-R and clinician’s BE. This is not unexpected as ADOS is a 30-min snapshot during which symptoms displayed in other settings (e.g., social settings like school where children are sometimes at their worst) may be unobserved. As seen in one of the subjects here, an additional scenario involves children who, at or before age 4, were manifesting autistic symptomatology severe enough to warrant a clinical diagnosis (by developmental history), but who may have improved over time and did not meet severity criteria by research observation at the time of this study. Conversely, the ADOS also has the risk of over-interpreting behaviors elicited in the anomalous context of a 1-on-1 assessment by an unknown examiner. Because of these shortcomings of ADOS, phenotyping for genetic studies rely primarily on ADI-R supplemented by ADOS [Sebat et al., 2007; Szatmari et al., 2007; Strom et al., 2009; Wang et al., 2009; Weiss et al., 2009]. The high concordance rate observed in the presented data is important to document, as IAN is a potentially major source of recruitment into specific autism studies.

There are two caveats to the generality of the observations made here. First, all of the probands were identified through participation in IAN with web-based registration. The simple fact of the need for web-based registration will bias the sample collected, but in many ways this is similar to the bias of individual participation in biomedical research which does not randomly sample the whole population [Rothman and Greenland, 1998]. One risk of the web-implemented approach is that there may be a potential discrete cause of ASD in a given child, and yet not be known by the parents. In this study, all of the probands in the current study had idiopathic autism, which is of course true of the vast majority of individuals diagnosed with autism currently, implying that this will be a rare issue. Diagnostic misspecification, including individuals who have an alternate diagnosis, is only modestly problematic for the identification of autism risk alleles given the already observed genetic heterogeneity of ASD. Even if 10% of the ASD cases were misdiagnosed, we would need to sample a population approximately 20% larger to compensate for the loss of genetic power [Edwards et al. 2005]. Second, we assessed individuals who were at least minimally verbal as screened through online questionnaire to the parents to analyze the more common idiopathic autism group registered in IAN and because the quantitative measures employed in our rapid phenotyping paradigm have not yet been validated for their precision when used for children with comorbid intellectual disability, a majority of whom are rendered non-verbal by virtue of this comorbidity (non-verbal children comprise a 25% minority of all ASD-affected children in the IAN register). Thus, our conclusions here may be restricted to minimally verbal affected children and the concordance rate may not be as high for harder to assess non-verbal children.

Although teacher-report SRS had very high positive predictive value, reliance of the teacher-report SRS would result in the exclusion of potentially half of the documented ASD-affected children and improvement in accuracy over parent-report alone did not reach statistical significance in this study. The major factor was a lack of response from the teacher-SRS, but there was also a decreased sensitivity relative to parent-SRS. The very high positive predictive value of teacher-report SRS and relatively low sensitivity suggests that the threshold score used for case identification may have been set too high in severity. This is currently being more closely examined in an analysis of SRS data from over 1,200 children in the IAN register (manuscript in review)—it is likely that setting a slightly lower threshold will result in optimization of sensitivity, specificity, and positive predictive value. We also observe that the collection of the teacher SRS was challenging for some families and might therefore limit the feasibility of large-scale implementation that is easily scalable.

It is important to emphasize that we view this IAN/web-based approach as complementary to and synergistic with more comprehensively phenotyped sample sets, such as those being accumulated by AGRE, the Simons Simplex Collection, and various academic collaborative research projects. While we demonstrate here that the core phenotypic assessment of the autism phenotype as reported by parents results in autism study samples generally consistent with sample sets used in published autism genetic studies that relied on trained rater assessments, there are certainly many important avenues of research that cannot be directly addressed by this sampling strategy. For instance, the analysis of complex developmental syndromes characterized by both mental retardation and autistic traits would not be feasible in this context, nor would longitudinal studies that assess developmental trajectory, changes in brain imaging over time, or novel neuropsychological endophenotypes. However, given the tremendous genetic heterogeneity of ASD and unknown environmental influences makes the need for very large sample size collection compelling. This web-based recruitment permits the characterization of individuals with specific genetic etiologies. For instance, after the individuals who have a specific genetic etiology for autism are identified, IAN has established approved means to re-contact the specific families in order to perform additional phenotyping. We view the implemented web-based processes as a powerful community tool to rapidly identify willing research participants, implement rapid and facile DNA collection strategies distributed across the country, and subsequently affiliate generated genetic data with individual IAN participants.