Multiplex Bead Array Assays: Performance Evaluation and Comparison of Sensitivity to ELISA

Introduction

While multiplex bead array assays (MBAA) are widely ascribed to be a recent innovation, descriptions of these assays can be found in the literature as far back as 1977 (1). A wide assortment of tests have been devised for MBAA using both immunological and molecular ligands. Table 1 illustrates some of the bead array assays with potential clinical applications. The potential advantages of these assays are apparent, such as the ability to independently and quantitatively assay multiple analytes simultaneously in small volumes of material and the collection of data from numerous beads for each ligand to provide statistical rigor. The potential cost- and time-savings that could be accrued by use of MBAA in comparison to other methods provides a strong impetus for the routine use of these in both the research and clinical laboratories. As with all clinical laboratory tests, questions of reproducibility, precision, and accuracy must be addressed in order to validate these assays. This generally involves a comparison to methods considered to be the current standards.

Virtually all of the commercially available bead array kits (particularly those for cytokine detection) are supplied with reference standards and thus provide quantitative information. However, major questions remain regarding whether the quantitative data obtained by multiplex bead array assays are identical to, or at least similar to, data obtained using other methods.

Other Technologies and Assays

The assay which is most often compared to multiplex bead array assays is the enzyme-linked immunosorbent assay (ELISA). ELISAs, in general, use a similar immobilized antibody to capture a soluble ligand, with subsequent detection of the captured ligand by a second ‘reporter’ antibody. There are, however, several substantial differences between bead array assays and ELISAs. For example, MBAA uses fluorescence as a reporter system where ELISAs use enzyme amplification of a colorimetric substrate. Luminex captures ligands onto spherical beads in suspension while ELISAs generally rely upon flat surfaces in 96-well plates. Most importantly, MBAA techniques, by their very nature, are multiplexed and therefore may be subject to any perturbations that arise from analyzing multiple ligands simultaneously, such as cross-reactivities. By contrast, ELISA methodologies generally study one analyte at a time, and thus avoid any concerns arising from multiplexing.

Protein microarray kits, which use capture antibodies and reporter antibodies in a multiplex fashion similar to MBAA, should also be considered as a competing technology. However, these assay are relatively new, are not widely accepted as a ‘gold standard’ for clinical use, and also may be of limited sensitivity (2). As ELISA methods are widely used and accepted in clinical practice, this review will focus on comparison between MBAA and ELISA techniques.

Multiplex arrays – Artifacts Associated with Multi-analyte Analysis

As mentioned above, a key concern in the evaluation of multiplex bead array assays is the possibility that multiplexing, in itself, results in anomalies in the quantitation of some of the analytes (often termed ‘the matrix effect’). Interference with reliable MBAA determinations have been ascribed to anti-cytokine antibodies which may cross-react with other cytokines (3), to cross-species antibodies (3, 4), as well as to other interfering substances (5). While most of these observations have been anecdotal without complete demonstration of the mechanism(s) of interference, these reports have been sufficiently frequent to indicate the problem of the matrix effect does indeed exist. Thus it cannot be assumed that a reliable uniplex assay can automatically be incorporated into a reliable multiplex array. Rather, this transition must be validated by testing each analyte for non-reactivity against all the other antibodies to be used in the multiplex array, and by carefully assessing the diluent to be used for its ability to minimize, and not create, unwanted cross-reactions. In most instances, commercial multiplex kits have been optimized to eliminate or minimize any artifacts from multiplexing, thus there should be rigorous adherence to the manufacturer's protocols when using these kits. Any deviation from these protocols must be carefully validated to ensure the accuracy and precision of the assay.

Commercial Instruments and Kits

Bead array assays may be performed on either multi-use flow cytometers (such as the commonly available clinical cytometers from Becton Dickinson, Beckman-Coulter, Dako-Cytomation, or Partec) or on more specialized platforms such as the Luminex 100 system. While much of the early developmental work on MBAA was performed on research or multi-purpose cytometers, increasingly MBAAs are performed on dedicated instrumentation. The latter, which generally incorporate software capable of automatic gating as well as computation of absolute cytokine levels, significantly reduce the complexity in performing these assays and require less user interaction. Three of the most widely available commercial multiplex systems are from Becton Dickinson Immunocytometry Systems, Luminex Corporation, and Diasorin. Each of these will briefly be described below as examples of different approaches rather than as a comprehensive list of all manufacturers.

Cytometric Bead Assay (CBA)

The Cytometric Bead Array (CBA) system from BD Biosciences (San Jose, CA, USA) relies on different fluorescent intensities of a single fluorophore to accomplish multiplexing. As a result, the number of assays which can be performed is more limited than in array defined by two fluorochromes (6). However, one advantage of the CBA system is that it can be performed on a clinical flow cytometer already installed in the laboratory. The CBA system includes a cytometer setup kit with the requisite software, reagents and standards. The company's CBA assay kits employ their proprietary bead sets, which are internally dyed with varying intensities of a proprietary fluorophore. These sets are distinguished via one fluorescence parameter and two size discriminators. The CBA analysis software is an “add-in” for Microsoft Excel®, and is compatible with contemporary data acquisition software such as CellQuest™. The acquisition and the data collection must be performed with the acquisition software of the instrument and the list mode data files should be saved into FCS format. BD has also produced the FACSArray™ Bioanalyzer, a multifunctional system that supports both the multiplexed CBA technology and limited cellular analysis. The BD FACSArray™ instrument has two lasers (green: 532nm, red: 635nm), detects two scatter signals (forward-scatter width and side-scatter width) and four fluorescence signals (yellow, red, far-red, and near infrared). The instrument has the capacity to use 96-well microtiter plates for samples.

The company currently offers CBA kits for a wide variety of ligands, and data has been published from these kits for the quantitation of viral proteins (7) and for the multiplexed detection of cytokines from human (IL-2, IL-4, IL-5, IL-10, IFN-g, and TNF-a) (8-10).

The Luminex xMAP (Multi-Analyte Profiling) technology

The Luminex xMAP technology (formerly LabMAP, FlowMetrix) uses digital signal processing capable of classifying polystyrene beads (microspheres) dyed with distinct proportions of red and near-infrared fluorophores. These proportions define ‘spectral addresses’ for each bead population. As a result, up to one hundred different detection reactions can be carried out simultaneously on the various bead populations in very small sample volumes (4, 11).

Luminex® Corporation released the first flow cytometer designed specifically for multiplexed microbead analysis in the late 1990's. The dual-laser Luminex-100 instrument has a three-color fluorescence signal-detection system. Two colors are dedicated to microsphere classification; the third color is used for measurement of the reporter fluorescence intensity.

Luminex itself does not actually sell microsphere sets conjugated to specific capture reagents. Instead, the company offers conjugation-ready bead sets, carrying either avidin, carboxylate groups, or oligonucleotide adapters. Using these, it is possible to develop MBAA assays within any particular laboratory. Additionally, these bead sets are available to Luminex ‘partner companies”, who in turn use these to develop commercial MBAA reagent kits for sale to the end consumer. Several assay developers offer kits for the multiplex detection for human, mouse, and rat analytes, many of which are shown in Table 1. Some recent applications with Luminex-based fluorescent microspheres have included cytokine quantitation (12), hormonal analysis (13), single nucleotide polymorphism genotyping (14), growth factors (15), and characterization of the molecular interactions of nuclear receptors (16).

Coupled Particle Light Scattering (Copalis)

Copalis multiplex technology, produced by Diasorin, is unique from most other multiplex bead array approaches in that it does not use fluorescence to discriminate different bead populations, but rather differentiates monomeric latex microspheres from latex aggregates and cells on the basis of their unique light scatter properties by flow cytometry. The system can measure two types of events: polystyrene-microparticle latex coagglutination and polystyrene-gold colloid microparticle coupling. The former is useful for detecting the presence of antibodies to infectious agents or autoantigens, which are coated onto latex microparticles. By using only forward-scatter measurement, Copalis instrumentation can distinguish between different-sized particles, as well as between monomers and aggregates. The presence of specific antibodies is detected as a reduced number of non-agglutinated particles coated with the corresponding antigen. Based on this concept, a multiplex Copalis agglutination test was developed for syphilis, Epstein-Barr viral infection, SSA/SSB autoimmunity and cytomegalovirus (CMV) total antibodies. A second format of Copalis is the latex-gold sandwich assay format which has been developed for hepatitis B virus, CMV, luteinizing hormone (LH), and follicle-stimulating hormone (FSH).

Few studies reported the validation of Copalis in comparison to standard ELISAs (17-19). In these studies, which were performed for the detection of CMV, syphilis, Anti-Ro/SSA and Anti-La/SSA, the assay had a sensitivity of 86, 92.2, 88.2 and 95.2% respectively using the Copalis second format; and a specificity of 81, 97.6, 93 and 97.6%.

Despite the ingenuity of this approach, Copalis seems not to have been widely used – perhaps due to limited multiplexing capabilities and the reduced sensitivity compared with fluorescent techniques. Nevertheless, this technology illustrated the use of different sizes microspheres in other flow cytometric multiplexing assays.

Comparison of Multiplex Bead Arrays vs ELISAs

The acceptance of multiplex bead array assay for use in clinical research or in the routine clinical laboratory work is largely dependent on achieving similar results to those obtained using ELISA techniques, which are widely accepted as the current ‘gold standard’. Thus, a large number of studies published to date have compared data from multiplex bead assays to ELISA. While MBAA may be used to detect and quantitate a wide assortment of ligands, much interest has centered on the quantitation of cytokines. Therefore, for the sake of brevity, this review will focus on studies comparing MBAA to ELISA for cytokine determination.

Very few studies appeared in the peer-reviewed literature between the conceptual introduction of MBAA in the late 1970's (1) to the development of a commercial instrument in the late 1990's (20). Although commercially available kits have been extensively characterized and qualified by their respective manufacturers, there are a limited number of peer-reviewed reports of direct comparisons between multiplex bead arrays and ELISA kits, and all of these have been published within the last decade (3, 5, 21-28). It is difficult to compare the data from the different studies, as various investigators used different methods of comparison between MBAA and ELISAs, and reports varied considerably in the amount of methodological detail provided. Table 2 illustrates some studies of human cytokines, where correlations of MBAA and ELISA were expressed as r or r² values. However, even these data are difficult to compare due to methodological variations.

The majority of published studies have shown good correlations between MBAA and ELISAs for most, but not all, of the cytokines tested, but the degree of correlation has varied widely. These variations are likely the result of how these comparisons were made as well as the antibodies used in each of the assays. For example, in an early study from the Luminex Corporation, Oliver et al (20) compared MBAA to ELISA but did not state the manufacturer of the ELISA kit, or the specific antibody pairs used in ELISA. Nonetheless, a correlation coefficient (it was not stated if this was a Pearson's or Spearman's value) of 0.912 was reported for GM-CSF.

Chen et al (21) used reagents from same company (Pharmingen) to make both MBAA and ELISA cytokine assays. A six-plex MBAA for IL-2, IL-4, IL-10, IL:-12, IFN-gamma, and TNF-alpha revealed correlation coefficients ranging from 0.92 (IL-2) to 1.0 (IL-10). By using the same capture and reporting antibodies for both assays, these authors effectively eliminated this variable. Thus these comparisons must be regarded as highly reliable in assessing true ‘matrix effects’ of mutliplexing.

A much later study comparing multiplex kits from different vendors (LINCO Research, Bio-Rad Laboratories, R&D Systems, and BioSource International) by Khan and coworkers (22) measured the levels of IFN-gamma, IL-1beta, IL-6, IL-8, and tumor necrosis factor-alpha and compared the results with those from ELISA techniques. In this study, MBAA kis from different vendors yielded different absolute concentration of cytokines, although the cytokine levels followed similar qualitative patterns. However, absolute values for interleukin-8 were similar in ELISA and Luminex using kits from the same manufacturer (R&D Systems) to make the determinations. Again, this is suggesting that the antibody pairs used in the various ELISA and MBAA kits are a major determinant in the absolute cytokine values measured in both types of assays. Clearly, appropriate comparison of these two techniques requires that identical antibody pairs be used.

In a similar fashion to the study by Chen and colleagues, another research group (3) has prepared multiplex assays using a combination of antibodies from R&D Systems and Pharmingen, and compared the resulting data to ELISA kits from the corresponding vendors. Although it is unclear if the same clones of antibodies were used for the comparative assays, it is a reasonable assumption that they might have been. Although correlation coefficients between MBAA and ELISA were not reported, several interesting observations were reported. First, little, if any, cross-reactivity was observed between the cytokine antibodies. However, inappropriately high values were observed in the MBAA in the absence of a rat serum preincubation. The addition of this step reduced these values to close to the values observed by ELISA. As some of the antibodies used in the study were of rat origin, the presence of heterophile antibodies was strongly suspected as the reason for these artifacts.

MBAA technology was used by Prabhakar and coworkers (23) for quantifying, pro-inflammatory cytokines such as, tumor necrosis factor-alpha (TNF-alpha), interleukin-1 beta (IL-1beta), IL-6 and IL-8 levels in lipopolysaccharide (LPS)-stimulated human plasma samples. They described the validation and comparison of the multiplex assay with results from commercially available cytokine kits, although the vendor(s) of these kits were not specified and only bi-plex MBAAs were analyzed. This study found a generally good agreement between MBAA and ELISA, and ascribed differences observed between these assays to dissimilarity in the capture and reporter antibodies used.

Hildesheim and coworkers reported the validation of a multiplex cytokine assay from single vendor (Linco Corp). Eight cytokines were measured simultaneously in 720 specimens from 24 healthy volunteers. Due to inconsistent results IL-4, IL6 and IL-8 were excluded from further analysis. For the remaining five cytokines, excellent intra-assay reproducibility was observed, with Spearman correlation coefficients consistently above 0.90 and exact agreement rates ranging from 77.6-90.3%. However, only IFN-gamma data was correlated with conventional ELISA. The Spearman correlation coefficient was 0.93. Exact agreement and weighted kappa values were 67.3% and 0.73 (95% confidence interval, 0.67-0.80), respectively (24).

DuPont et al. (25) examined the correlation of ELISA and MBAA techniques for quantitating a variety of cytokines in culture supernatants using kits from two commercial vendors (either from Linco and Upstate). They demonstrated excellent correlations between ELISA and Luminex for seven cytokines (IL-1beta, IL-4, IL-5, IL-6, IL-10, IFN - gamma and TNF-alpha), fair correlations for IL-13, and a poor correlation in the case of IL-12. However, although the correlations were generally good, the authors reported a significant variation between the absolute cytokine concentrations determined by ELISA and either multiplex kit. These differences were generally more pronounced with the LINCOplex kit. The levels of IL-1beta, IL-5, IL-6, IFN-gamma, and TNF-alpha by LINCOplex kit, and TNF-alpha by Beadlyte Upstate yielded cytokine concentrations that were from 2- to 10-fold lower than the ELISA determinations. For IL-10 and IL-4, the opposite was true, with either LINCOplex or Beadlyte kits determinations being higher than ELISA. However, a high degree of correlation and concordance, was seen between these two multiplex kits for IL-1beta IL-4, IL-10, IL-13, IFN-gamma, and TNF-alpha. For sensitivity, the LINCOplex kit showed higher relative sensitivity for IL-4, IL-13, and IFN-gamma, equivalent sensitivity for IL-1beta, IL-6, IL-10, and TNF-alpha, and poor sensitivity in case of IL-5 in comparison to ELISA. Similar to other investigators, DuPont and colleagues point to differences in antibody pairs and sample diluents as likely causes of observed differences. They also stress that each antibody will have optimal binding affinities at specific pH and salt concentrations, which may impact on multiplex assays (25).

In a recent, very large study of more than 2000 serum specimens, Ray et al (26) carefully examined the validation and implementation of cytokine multiplex assays, as a replacement for ELISA techniques. Although a fairly good correlation (Lin's concordance correlation = 84.5%; Lin's corrected for bias = 94.5%) was found between these assays, the multiplex results were, on average, 2.36-fold higher than ELISA values. The authors conclude that, given the different analytic platforms, different antibody pairs, and different lots of standards, this level of correlation was “impressive”.

Summary

A multitude of studies have demonstrated good correlations, but often poor concurrence of quantitative values, between multiplex bead array assays and corresponding ELISA measurements. Enough evidence has been presented to indicate that a comparison of randomly selected bead array assays and ELISAs will likely demonstrate substantial differences. However, if comparisons are made between MBAA and ELISA which use identical capture and reporter antibodies, as well as similar diluents and serum blockers, variability will be minimized, correlations will be good, and similar quantitative values will be achieved. With this in mind, any study that involves sequential monitoring of patients, or other samples, should be performed using only a single technique, one platform, and one commercial vendor for all samples. As illustrated by Khan et al, although multiplex assays from different vendors will show similar trends in cytokine levels, the absolute levels of cytokines measured will vary (22). Therefore, even MBAA assays from different vendors should not be considered interchangeable. When transitioning from an existing assay to a new MBAA assay, it might be possible to establish the systematic bias between the two assays and at least establish some basis for comparing data.

MBAA tests have proven to be easy to perform and reproducible. Given the fact that these assays are also cost- and time-effective, and minimize the sample volume requirements, it is likely that these multiplex bead array assays will become increasingly commonplace.