The Role of Replicates for Error Mitigation in Next-Generation Sequencing

Introduction

The emergence of next-generation sequencing (NGS) has revolutionized genetics and provided valuable resources for other scientific disciplines. As NGS becomes more widely accessible, its use has extended beyond basic research and into broader clinical contexts. Hence, it is increasingly more important to account for the error that arises in the sequencing process. Error can stem from the bioinformatic analysis¹, and also from experimental steps^2,3, the latter of which can often be mitigated through the use of replicate experiments.

The use of replicates permeates almost all scientific disciplines. Yet in NGS, many researchers use increased sequencing read depth and bioinformatic filters to address error in lieu of biological replication. This practice is understandable given that replicates can increase study costs substantially. However, sequencing costs have fallen dramatically⁴, and now is the time to reevaluate the value of replication in sequencing studies.

Here we discuss sources of error in sequencing and the nascent use of replication in published high-throughput sequencing efforts. In addition, we demonstrate how biological replicates can be employed to reduce sequencing error. In particular, replicates can be used to assess the specificity and sensitivity of sequence variant calling methods in a manner that is independent of the algorithms and chemistry used to call variants, thereby guiding the appropriate selection of quality score thresholds.

Experimental Error in NGS

Technological advances and the digital nature of DNA are helping to achieve highly accurate genome sequences. However, sequencing methods are imperfect. NGS applications, such as whole genome sequencing, targeted capture, RNA-Seq and ChIP-Seq, are prone to errors that result in miscalled bases, thus causing short read misalignment and mistakes in genome assembly. Reported sequencing base call accuracy claims for leading high-throughput sequencing technologies vary wildly, ranging from one error in one thousand nucleotides (99.9%)⁵ to one error in ten million nucleotides (99.9999%)⁶. Even for methods with the lowest reported rates, the absolute numbers of miscalled genomic variants remain unwieldy, with possibly thousands of false positive variants in a fully sequenced human genome. Furthermore, false positive error masquerades as rare and somatic variants, thereby obfuscating true variants of clinical interest. Known sources for experimental error can be grouped by where they occur in the sequencing workflow (Figure 1a; Box 1), i.e., during sample preparation, library preparation, or sequencing/imaging.

Replicates and Experimental Error

Many applications, such as the pursuit of rare causal variants, clinical applications, and somatic variant detection, require high fidelity in sequencing, necessitating confirmatory experiments, such as Sanger sequencing. The standard validation methods used for confirmation tend to be costly and labor-intensive, thus necessitating lower-cost alternatives. An approach that holds promise uses the tried-and-true scientific method of replication to mitigate user error, stochastic differences, and other sources of experimental error. Different types of replication are described below, including sequencing read depth, technical, biological and cross-platform replication.

Reducing error and replicates

As sequencing further permeates science and medicine, replicates will be invaluable to researchers and clinicians alike. Current efforts in sequencing error mitigation rely mainly on filtering strategies, including filtering for sequencing read depth, base call quality, short read alignment quality, variant call quality, known variants, strand bias, allelic imbalance and sequence context^{10,21,25,27,34–37}. All these post-processing techniques help reduce uncertainty in the final genotyping variant call (Figure 1b).

Bioinformatic filtering techniques can be optimized using technical, biological, and cross-platform replicates to improve specificity and sensitivity³². For example, optimal quality score thresholds for each filter may be selected using replicate genome sequences. An individual human genotype has roughly 3 million variants³⁷; however variant callers can predict >20 million variants of differing quality per genome, mainly from mismapped short reads³⁸, mosaicism, and sequencing error. Consequently, thresholds are chosen to limit the variants called in the individual’s genotype. Ideally, these thresholds are chosen with experimental confirmation³⁹, but this can be costly. We assert that replicates can abet bioinformatic filtering and reduce the number of variants requiring validation, thereby improving the quality of the sequence being mapped or assembled.

To illustrate, we use biological replicates to conduct a simple analysis for assessing the reliability of single nucleotide substitution calls (Figure 2). For genotyping, the number of replicates should be chosen to attain adequate statistical power at the loci in question. However, here we seek a set of likely false positives stemming from experimental error, which thus requires only three replicates for a voting majority. For the replicates, we obtained sequence data from three distinct tissue samples of participant PGP1 in the Personal Genome Project⁴⁰ (see Supplementary Notes).

Loci were identified in which one or more replicates contain a single nucleotide variant (SNV). In brief, SNV loci are deemed “concordant” when all replicate variant calls agree⁴¹, and SNV loci are called “discordant” when other replicates differ from a target replicate. Thus, concordant loci represent true positive variants, and discordant loci signal false positive variants. See the Supplementary Notes for precise definitions of concordance and discordance, for details on choosing a target replicate, and for implementation details.

Once discordant (false positive) and concordant (true positive) variants have been separated from each other, metrics of variant call confidence (e.g., quality scores or read depth) are used to rank-order the target variants. Using the rank-ordered sets, one can plot the accumulation rate for concordant and discordant variants with decreasing score stringency, in a representation similar to a receiver-operator characteristic (ROC) curve. Thus, variant call quality score thresholds can be chosen to maximize the proportion of all concordant variants seen at or below a particular threshold relative to the fraction of all discordant variants. This analysis (Figure 3) suggests that, while adequate read depth across the genome is essential^28,29, read depth is not the best measure of reliability of a specific variant call at a particular locus. Indeed, read depth at a particular locus is an inferior filter when compared with error-model-based quality scores. We found that this holds true for quality scores computed by software packages that process genomic³⁵ and expression^27,36 data. Even after removing regions with abnormally high read depths (enriched for misalignment errors in low-complexity sequence³⁸), quality scores considered here still outperform read depth as a filter for sequencing error.

In addition to comparing disparate error model quality scores, this approach can be used to evaluate the effect of manipulating quality score thresholds for a specific data set of interest. For example, sensitivity of a particular threshold can be evaluated by considering the false negative rate, as estimated by the number of concordant variants that are lost as a result of applying the threshold.

Post-processing Error in NGS

Even with replicates, some types of error cannot be addressed without further technological advances and improvements in bioinformatic processing. For example, insertions and deletions⁴² as well as paralogs and other repetitive sequence⁴³ often confound NGS short read alignment^44,45, resulting in mismapped reads and ultimately, variant call errors. Other sources of error can arise from limitations in software and configuration during secondary analysis, including read clipping and filtering⁴⁶, allelic bias⁴⁷, and variant call confidence models⁴⁸. These cannot be addressed with replicates alone.

Erroneous variant calls also arise from incomplete reference data. This error type arises when reads are mapped to unfinished reference genomes/transcriptomes and drafts containing misassembled regions⁴⁹. These errors will steadily decrease in frequency as reference genome assemblies and annotations such as GRChr37⁵⁰ and RefSeq⁵¹ are completed and corrected with each new build release.

Lastly, strides in haplotype phasing hold promise not only for reducing amplification errors⁶, but also for reducing the causal variation search space. For example, only through accurate haplotype phasing can we begin to discern the difference between two dysfunctional gene copies (i.e., a double mutant) and a single normal copy⁵². This difference can have important implications with regard to phenotype and clinical applications of sequencing. Unfortunately, current mainstream NGS methods do not consistently discern between these two cases. Thus, ad hoc experimental^6,53,54 and computational procedures^55,56 are required to distinguish the haplotypes of diploid cells.

Concluding Remarks

In these past decades, amazing scientific and technologic advances have provided molecular-level resolution for the inner workings of life. NGS technologies are providing insights into genetic disease associations^57–63, differences in human gut microbiota⁶⁴, amino acid essentiality in proteins⁶⁵, experimental evolution^66–68, biotherapeutic development^69–73, protein-DNA interactions⁷⁴, epigenetics⁷⁵, cancer genomics^39,76 and clinical diagnosis⁷⁷. Efforts to find biologically and clinically relevant variants are steadily improving as algorithmic advances more intelligently filter the large amounts of sequence data. For example, variants can be prioritized by considering heritability or variant association in populations^61,78, correcting for gene-specific mutation rates¹⁰, accounting for evolutionary conservation^79–81, and providing network context through systems biology approaches^82–84. Beyond strictly biological applications, sequencing is also becoming an analytical tool for more esoteric questions, such as recording fluctuations in ion concentrations⁸⁵ and even potentially detecting dark matter in astrophysics⁸⁶. All these sequencing studies, however, are limited by the accuracy of the underlying sequencing experiments.

Here we have identified sources of sequencing error and presented a method for addressing the stochastic effects. Additional approaches to address other sources of error, such as experimental bias and software limitations, are also essential. These approaches include identifying erroneous SNPs exhibiting Hardy-Weinberg disequilibrium¹¹, masking poor quality bases⁸⁷, phasing and imputing variants in difficult-to-sequence regions or uncalled regions⁵⁵ and improved methods for calling of structural variants, CNVs and indels. In conjunction with these computational approaches, the wise use of replicate genome sequencing will play an increasingly important role in reducing the noise in data processing and downstream analyses.

Supplementary Material

Biographies