stock pasteExtracts from Elispot proficiency panel
Reported by Werner Freber (founder of BIO-SYS Gmbh)

Like in the past years the CIP group organized a collaborative Elispot proficiency study. Additional to the procedure in the past, this year all Elispot-plates had been analysed central in addition to the labs reading. The Bioreader 5000 -E beta / BIO-SYS  had been chosen by the CIP group to perform  the central analyse under GAMP / GLP(Good automated manufacturing practice) and Quality management in the Lab of BIO-SYS GmbH / Karben / Germany

While this study has not been published CIP group allows now to publish the aspects of the study, that reflect the capabilities of the Bioreader 5000 Pro–E.

The study did have two objectives to find out:

1.) 5if serum can be successfully replaced in serum-free FREEZING MEDIA to cryopreserve PBMC for ELISPOT analysis.

2.) Does the central analyse lead to a higher detection rate and/or better signal noise ratio comparing to users individual analyse?

Citation5
IFN-g ELISPOT Proficiency panels organized by CIP, CRI-CIC and other groups showed that ELISPOT protocols with serum-free test media can support favourable test performance. As serum containing medium can be successfully replaced in ELISPOT by serum-free TEST MEDIA we now wanted to test if serum can be successfully replaced in serum-free FREEZING MEDIA to cryopreserve PBMC for ELISPOT analysis.
Group of participants
31 Labs from 8 European countries and the USA participated in this proficiency panel phase.
Five labs of the group were recruited from the CRI-CIC proficiency panel program that collaborated in this study.

Participants had to thaw all cells (3 vials per donor) using their preferred thawing procedure without any mandatory protocol step being enforced by the organizers. Immediately after thawing, and again after resting (a resting phase was highly recommended but not mandatory for labs that have SOPs that do not foresee a resting phase) the number (Mio. PBMC) and viability (%) of recovered PBMC had to be determined and documented. The recovered cells were tested in an IFN-gamma ELISPOT assay making use of the locally established operating procedure. In order to facilitate the analysis of data generated, participants had to adhere strictly to the predefined pipetting scheme that included six replicates of the MOCK control on the plate, three peptide antigens added as triplicates and 1 well of positive control of choice for each of the six donor-freezing medium conditions (slide 6).

slide-6

Central Analysis of ELISPOT plate

All ELISPOT filters were collected to perform a central analysis at a certified lab in Germany (Bio-Sys) that uses optimized and advanced detection algorithms.

Evaluable data sets

One participating lab analysed all 6 donors (instead of only two selected donors) and generated three completed independent data sets (Groups 1, 2, and 3). One participant observed an enormous background spot production in all tested donor-antigen combinations, which made the evaluation of the results impossible. This data set was therefore excluded from the final analysis. Consequently, we obtained 32 evaluable data sets from the 31 participating labs.

PART IV - CENTRAL ANALYSIS OF FILTER PLATES

31 of the 32 filter plates were centrally analysed in collaboration with W. Freber using a Bioreader 5000 –E beta brand (BIOSYS GmbH / Karben / Germany). Prior to central analysis it was agreed that CIP would not be involved with this central analysis. The only requirements made were that:

1. One and the same measure protocol had to be used to analyze each single well in a plate.

2. The number of measure protocols used to analyze all plates should be a little as possible.

3. The quality of the readout should be as high as possible

As expected it was not possible to analyse all 31 plates using exactly the same settings which may be partly due to the fact that all participants used different plates, reagents and that spots sizes and densities varied significantly among the different protocols. In order to analyze all 31 plates appropriately a total number of

  • one (1) protocol for 50% of all plates
  • two(2) protocols for 75% of all plates
  • for the remaining 25% of plates different (plate-specific) reader protocols had to be applied.

The CIP team collected the incoming raw spot counts and applied the same response determination criteria as for the data sets that were submitted by participants.

As can be seen on slide 34 the central analysis led to a similar (defined as equal or less than 10% discrepancy between the two detection rates) detection rate as the peripheral analysis for most tests (23/32). In 6 out of 31 cases did the central analysis lead to a higher detection rate (ID04, ID12, ID27, ID29, ID34 and D35). In two cases (ID07 and ID16) the central analysis led to a lower detection rate.

slide-34

Slide 35 shows two representative examples of experiments (ID27 and ID16) where the central analysis led to different results compared to the peripheral count. If you are interested in a more detailed feed-back, please contact the organizer. Please also note that the examples shown should not be seen as recommendations from CIP but merely reflect analysis performed by two independent investigators. The main message for readers of the report should be that ELISPOT reader settings and human auditing of results (plausibility check) can dramatically influence the signal to noise rations. Consequently it is recommended to spend sufficient time and consideration to optimize spot acquisition.

slide-35

Slide 36 displays 4 representative examples where lab count and central analysis led to similar results. It should be noted that both investigators (participant and central lab) decided to “gate out” smaller spots that are found in the medium control wells and in the wells with stimulated cells. It is clear that the decision to focus on the “more intense spots” across all wells of this plate will lead to a significant change in the number of reported spots per well as compared to data acquisition that would have been set to capture as many spots as possible (including weaker signals in medium control and stimulated wells). Although “gating” of events is routinely done in flow cytometry assays so far strategies to rationally control raw event/spot counts in ELISPOT assays have not been intensively discussed in the community. The main message of this report is not that any specific approach (count only the big spots or count all spots) should be recommended (this decision should be up to the individual investigator). The organizers would rather point out that different “styles” of spot acquisition exist which may contribute significantly largely to the difference of raw spot numbers reported across institutions.

slide-36

Slide 37 shows the overall background spot production that was observed in the medium control wells (spots per 100,000 seeded PBMC). The mean spot production in the medium control observed in the central analysis was lower for all three freezing media as compared to the background spot production reported by the participants. This is most probably due to the fact that the central lab used a detection algorithm that censored smaller and less intensive spots in the medium control and only count bigger and more intensive spots. This is possible due to the fact that the wells showed two different “spot categories” that differ in their specific appearance and size distribution and can therefore be differentiated by the reader.

slide-37

Slide 39 shows the signal to noise ratios for EBV and FLU responses tested in all participating labs. The signal to noise ratio was calculated by dividing the mean spot number found after adding the peptide stimulus by (the mean spot number in the medium control plus 2-times the standard deviation found in the medium control replicate). In cases where the denominator was a zero the mean spot number found after adding a stimulus was divided by 1. The figure shows that the signal to noise ratios was increased by the central analysis for most labs. This is probably due to the decision of the central lab to focus mainly on the bigger and more intensive spots as usually found after adding an antigenic stimulus and to censor the smaller and less intensive spots that are usually considered as being due to stimulus-unspecific spot production. The higher signal noise ratio on the central analyze may become important for detection of low frequencies of antigen-specific T cells.

slide-39

End of citation from CIP


  • CIP organized and successfully completed a large ELISPOT proficiency panel with a heterogeneous group of 31 labs using representative assay protocols.
  • Participants received a feed-back about their performance relative to other group members
  • The new datasets were included to data sets accumulated over the last 5 years to determine new (and dynamic) benchmarks for assay performance (background and replicate variability) expected from an average lab
  • The central analysis confirmed the unexpected high overall detection rate of this proficiency panel.
  • For most participants the spot counts between individual and central analysis did not differ greatly. However, in a small fraction of labs discrepant results were obtained
  • Objective ways to “gate” on spots that have defined properties (size, density, gradients etc.) is technically feasible and may be used to control signal to noise rations.
  • As no standards for the decision which spots should be accepted or excluded exists, spot acquisition will remain a significant source of variation for results acquired in and reported by different institutions.
  • It has to be noted that CIP does not recommend the use of a specific spot reader or a specific strategy to count spots. The main points made by CIP are (1) that different readers and approaches exist for data acquisition (2) it is known that spot population with different size distributions exists which might be differentiated by computer-based algorithms.


We thank the CIP group to give us the opportunity to analyses so many plates of a heterogenic nature.

We did not have the idea - contrary the users in its lab- to develop individual measure protocol for each specific lab. The aim anyway was to operate with as little measure protocols as possible (see citation on part IV of this document)

Detection rate

We could confirm the majority of lab users detection rate (see slide 34). But in the more we are glad that in 6 of 31 cases we found a higher detection rate. (appr 20% of all cases).This may be of great importance to exclude false negative results and if it comes to decisions how to evaluate the donors response.

I two cases Bioreader found a lower detecting rate comparing to the lab.

When we did have a look at the original plates we realized that these plates did have an extreme variance between the triple replicate - so statistically this is leading to a low detection rate. (see slide 35. Lower set of wells)

So, the Bioreader documented the ‘true’ variance between this triple, so once again: this result (lower detection rate) is the true result.

Signal noise ratio (slide 39)

On 23 of 31 plates Bioreader determined a higher signal noise rate automatically.

For our understanding this is because the Bioreader does have an excellent background compensation ,based on the patented, dual telecentric illumination. In addition of course the ‘auto-setup –finder ‘ and the expert rules made this possible.

I don’t need to explain how important a high signal noise ratio is , especially if it comes to samples with low signals.



1. Mander,A., Gouttefangeas,C., Ottensmeier,C., Welters,M.J., Low,L., van der Burg,S.H., & Britten,C.M.

Serum is not required for ex vivo IFN-gamma ELISPOT: a collaborative study of different protocols from the European CIMT Immunoguiding Program.

Cancer Immunol. Immunother. 59, 619-627 (2010).

2. Janetzki,S., Price,L., Britten,C.M., van der Burg,S.H., Caterini,J., Currier,J.R., Ferrari,G., Gouttefangeas,C., Hayes,P., Kaempgen,E., Lennerz,V., Nihlmark,K., Souza,V., & Hoos,A.

Performance of serum-supplemented and serum-free media in IFNgamma Elispot Assays for human T cells.

Cancer Immunol. Immunother. 59, 609-618 (2009).

3. Smith,S.G., Joosten,S.A., Verscheure,V., Pathan,A.A., McShane,H., Ottenhoff,T.H., Dockrell,H.M., & Mascart,F.

Identification of major factors influencing ELISpot-based monitoring of cellular responses to antigens from Mycobacterium tuberculosis.

PLoS. One. 4, e7972 (2009).

4. Moodie,Z., Price,L., Gouttefangeas,C., Mander,A., Janetzki,S., Lower,M., Welters,M.J., Ottensmeier,C., van der Burg,S.H., & Britten,C.M.

Response definition criteria for ELISPOT assays revisited.

Cancer Immunol. Immunother. (2010).

5. Report on the CIP ELISPOT proficiency panel ID09_2010_ELI

H. Filbert, B. Y. Renard, C.M. Britten

CIP committee: S.H. van der Burg, M.J.P. Welters, C. Gouttefangeas, C. Ottensmeier

CRI-CIC immunoassay Working Group: S. Janetzki’