10 JAAR SKML EXTERNE

KWALITEITSRONDZENDINGEN

VOOR MOLECULAIRE DIAGNOSTIEK

NAAR PARASITAIRE INFECTIES

WMDI 2 februari 2021

Jaap van Hellemond

Rob Koelewijn

Theo Schuurs

& SKML sectie parasitologie

Disclosure belangen spreker

Geen (potentiële) belangenverstrengeling

Voor bijeenkomst mogelijk relevante relaties1 Bedrijfsnamen

Sponsoring of onderzoeksgeld2

Honorarium of andere (financiële) vergoeding3

Aandeelhouder4

Andere relatie, namelijk …5

Geen

Stichting Kwaliteitsbewaking Medische Laboratoriumdiagnostiek (SKML)

Non-profit organisatie met als doel:

• bevorderen van de kwaliteit van medisch laboratoriumonderzoek

• kern van de werkzaamheden is externe kwaliteitsbewakings-rondzendingen voor alle laboratoriumspecialismen

• 15 secties (alg. chemie, haematologie, virologie, parasitologie, etc.)

• Sectie bestuursleden op voordracht van vakverenigingen

• Sectie parasitologie : WMDI

Detection of parasites

Classical method:- Microscopy ± concentration techniques - Limited sensitivity- Outcome observer & procedure dependent

Need for better diagnostics

Nucleic Acid Amplification Techniques (NAAT)- High sensitivity / specificity- Automation- Potential for harmonisation

- Real-time PCR methods are now frequently used

How well do NAATs function in practice?

External Quality Assessment Schemes (EQAS)

How well do NAATs function in practice?

External Quality Assessment Schemes (EQAS)=

Inter-laboratory comparison

Aim EQAS of SKML section parasitology

• True pathogens in real patient material • no lyophilized materials• no purified DNA in artificial matrix• no fixation

• high quality• reproducible and stabile materials• challenging loads

→ EQAS for entire procedure !!

Why is EQAS for entire procedure relevant for parasitic diseases?

Present in materials known to be difficult for DNA extraction- e.g. stool, blood

Parasites often present in low abundance- Acanthamoeba, Toxoplasma, Schistosoma

Well protected life cycle stages- Eggshells (e.g. Trichuris trichiura)- Cyst wall (e.g. Acanthamoeba spp.)

→ not only efficient DNA amplification requiredPre-treatment and DNA-isolation methods can be crucial!

SKML parasitology EQAS

2011: 1st pilot EQAS for detection of parasites by NAAT • Protozoa in stool

2021: 5 EQAS for Molecular diagnostics to detect parasites Protozoa in stool (Giardia, E. histolytica, Cryptosporidium spp, D. fragilis, E. dispar, Blastocystis spp.)

Helminths in stool (Strongyloides, Schistosoma, hookworm, Ascaris, Trichuris)

Sexual Transmitted Diseases (Trichomonas vaginalis, Chlamydia trachomatis, Neisseria gonorrhoea)

Malaria (part of “Blood parasites) (Plasmodium, both for LAMP & RT-PCR)

New: Acanthamoeba spp

Patient sample in expert lab(t=-18d tot -9d)

Sample preparationIn coordination lab for validation

Sample selection & distribution

Test by 5-7expert labs

Test in 5-fold by one expert lab

Workflow materials molecular QC panel

5-plo Reference laboratory 1

Reference laboratory 2

Reference laboratory 3

Reference laboratory 4

Reference laboratory 5

homogenizedstool

sample

Nummer

Materiaal

Hoeveelheid

Aangeleverd door

Klinische gegevens

Datum bereiding

Vries dooi cyclus

Verzonden naar ref. lab

PIN

Datum 1e bevinding

Uitslag Microscopie CR EH

Uitslag PCR GL CR CR EH

CT-waarde 19,1 31,7 onb onb

Uitgevoerd door:

Datum 5-plo

Uitslag 5-plo 27,6 33,6 34,6 36,7

27,7 34,0 34,0 37,3

27,9 34,5 35,6 37,3

27,4 34,1 34,4 36,9

28,2 34,6 34,1 36,4

Gem. 27,8 34,2 34,5 36,9

SD. 0,30 0,40 0,64 0,39

Uitslag PCR GL

Uitslag ref. lab GL CR CR EH

Gem. 24,9 31,8 28,6 33,2

SD. 2,48 2,4 5,01 3,07

CT-waarde max. 27,8 34,2 34,5 37,0

CT-waarde min. 21,1 28,5 20,2 28,4

Geschikt voor rondzending

Opmerking

Nummer rondzending

Datum rondzending

Casus

Verdiepingsvragen

Aantal deelnemers

Aantal inzendingen op target

%

Aantal correct

%

CT-waarde gem

SD.

CT-waarde max.

CT-waarde min.

Overige bevindingen,

al dan niet in combinatie

HZH/EMC

2012.4.E

2012.4

2012.42012.4

13-11-2012

2012.4

13-11-2012

ja ja

26-7-2012 24-10-2012

Nee

31-7-2012

2012-3F

10-7-2012

25-jul

HZH/EMC HZH/EMC

2012-3E

89

17-7-2012

16-7-2012

Nee

34

89

ja

38% 37%

33

ja

FMB-17

feces PBS 1:3

Izore

feces PBS

Izore

FMB-20

24-10-2012

HZH/EMC

feces PBS

HZH/Erasmus

24-10-2012

Haarlem

FMB-23 FMB-24

feces PBS

32 34

2012.4.F

88 88

24-10-2012

30 29 32 33

36% 39%

34,5

88,2% 87,9%

32,3

100% 97,1%

27,7 29,3

2,9

37

21 31

38 41

25

3,2 2,8

negGLCR

Rondzending feces PCR

SKML 3e en 4e rondzending 2012

40

27

2,9

Validation control 1: homogeneity

-3,0

-2,0

-1,0

0,0

1,0

2,0

3,0

t=0 t=1 t=2

Δ Cq

2013.2A Cryptosporidium spp.

2013.2A D. fragilis

2013.2B G. lamblia

Validation control 2: stability materials (after 14 days at room temperature)

-3,0

-2,0

-1,0

0,0

1,0

2,0

3,0

t=0 t=1 t=2

Δ Cq

2013.2A Cryptosporidium spp.

2013.2A D. fragilis

2013.2B G. lamblia

Methods developed to prepare• homogeneous and stable materials • suitable for molecular diagnostics

Validation control 2: stability materials (after 14 days at room temperature)

2013 99 85 92 38 77 26 n.d. -

2014 96 84 100 54 83 52 98 48

2015 99 85 96 57 97 29 100 73

2016 97 119 100 30 85 61 99 80

2017 97 216 100 61 100 32 100 121

2018 100 97 92 64 97 100 96 52

2019 96 97 98 63 97 69 93 102

Average 98 97 91 98

N N

Dientamoeba

fragilis

Entamoeba

histolytica N

Cryptosporidium

spp

Giardia

lambliaN

Sensitivity (%) 2013-2019 protozoa in stool

N = number of reported results (participants on samples)

False – negative results 2013-2015

relative load

12 116

363

2977

3040

3121

3164

3282

3431

3617

1148

3131

3239

3622 77 22

540

048

9

3197 62

3097

3199

3261 12

9

1688

1101

3253

3279

1184

3280

2171 85

2938

-15

-10

-5

0

5

10

15

Giardia lamblia

Participant number

Dif

fere

nce r

ep

ort

ed

Cq

valu

e t

o m

ed

ian

all p

art

icip

an

ts

Results for 1 material with Giardia lamblia

12 116

363

2977

3040

3121

3164

3282

3431

3617

1148

3131

3239

3622 77 22

540

048

9

3197 62

3097

3199

3261 12

9

1688

1101

3253

3279

1184

3280

2171 85

2938

-15

-10

-5

0

5

10

15

Giardia lamblia

Participant number

Dif

fere

nce r

ep

ort

ed

Cq

valu

e t

o m

ed

ian

all p

art

icip

an

ts Large variation in Cq values (>10 cycles)= 210 = > 1000 fold load difference!!How come?

Results for 1 material with Giardia lamblia

Participant number

False result (administrative)

False result (true)

Cq values G. lamblia per participant, 2013-2015

Participant number

False result (administrative)

False result (true)

Cq values G. lamblia per participant, 2013-2015

Variation = not random distributed

Workflow molecular diagnostics intestinal parasites

pre-treatment stool

DNAisolation

targetamplification

G. lamblia: distribution of DNA and stool

G. lamblia: distribution of DNA and stool

G. lamblia: distribution of DNA and stool

G. lamblia: distribution of DNA and stool

PCR?

pretreatment/DNA-isolation?

Theoretical questions on methods

Pre-treatment stool:- dilution in lysis buffer before DNA isolation?- freeze – thaw cycle?- other pre-treatment steps?- volume of stool used for DNA extraction

Equipment

DNA Target

Etc.

Correlation Cq with stool ‘input volume’ for PCR

- EQAS results NAAT better than for microscopy methods- Enormous variation in reported Cq values- Laboratories reporting low Cq values report less false negative results - High Cq values are associated with small ‘stool volume’ as input for PCR

Conclusions EMQAS Protozoa stool

STD External Mol. Qual. Ass. scheme

T. vaginalis(vanaf 2015)

STD External Mol. Qual. Ass. scheme

T. vaginalis(vanaf 2015)

STD External Mol. Qual. Ass. scheme

No correlation DNA input vs reported Cq

STD External Mol. Qual. Ass. scheme

Distribution of real helminth-pathogens in stool• Field samples (& non-endemic controls)

• No lyophilized material nor DNA in artificial matrix

• Ethanol preservative for long stability => global participation

Panel:

• Hookworm (N. americanus & Ancylostoma spp.)

• Ascaris spp.

• Trichuris trichiura,

• Strongyloides stercoralis

• Schistosoma spp.

Helminth External Molecular Quality Assessment Scheme (HEMQAS)

Helminth External Molecular Quality Assessment Scheme (HEMQAS)

HEMQAS pilot: participants outcome

Necator americanusStool sample DNA sample

1 3 7 8 10 9 11 12 6 5 2 4 a c d f g h b eNegative Positive Educational

Lab Micr 0 0 0 0 50 30 60 60 320 450 0 0 N N N N N N P P Negative/Positive/DNA samplesA 1 2 2 0 1 N 1 5 5 = 2 points (Cq low)B = 2 points (Cq high)C N = 0 pointsD 0 1 4 3 3 N N 5 6 P = 0 pointsE -5 -4 -6 -4 8 N -4 -3 -3

F N 6 N N 3 N N 6 6 Educational samplesG 1 -5 -5 -5 0 N N -8 -8 = not scored (Cq low)H 0 3 0 2 -2 N 0 -4 -4 = not scored (Cq high)I 5 4 4 6 0 N 7 7 6 N = not scoredJ -5 -4 -4 -1 -6 N -3 -4 -5

K 1 -1 -1 3 0 N N 3 2 x = ∆ Cq to median Cq of allL -4 -3 5 -4 -7 N N -6 -6 N = false negative

M N N N N N N N N N P = false positiveN P 5 1 5 7 6 I N I NO -1 -4 -3 -3 -5 N N 0 0 = not performed

median Cq 35 30 29 30 28 29 - 33 32 29 I = inhibition

Exp 1/5 3/5

HEMQAS pilot: participants outcomeΔ

Cq

to m

edia

n

Variation in reported Cq values even bigger.

• Proof of concept: validated, homogeneous samples

• Feasibility: world-wide EQAS distribution and reporting

• More variation in performance HEMQAS than GI protozoa

• Species-related performance differences, challenging samples

• Quantitative: reported Cq-values differ > 15 cycles

Need for quality control (and harmonisation) for NAAT methods

Conclusions HEMQAS pilot

Acknowledgements –HEMQAS pilot37

• Proof of concept: validated, homogeneous samples

• Feasibility: world-wide EQAS distribution and reporting

• More variation in performance HEMQAS than GI protozoa

• Species-related performance differences, challenging samples

• Quantitative: reported Cq-values differ > 15 cycles

Need for quality control (and harmonisation) for NAAT methods

Cools et al., 2020 PLoS Negl Trop Dis 14(6): e0008231.

Conclusions HEMQAS pilot

Malaria

• Validated in 2020

• Now part of scheme “Blood parasites”

• 4 EDTA-blood lysates per year

• Suitable for RT-PCR & rapid LAMP assays

Acanthamoeba species

• Pilot scheme March 2021

• Free of charge

• Subscribe www.skml.nl

New schemes

• NAAT perform relative well compared to microscopic methods

• False positive results occur (1-2%, half of which is administrative error)

• False negative results are associated with relatively high Cq values

• Very large variation among laboratories in reported Cq values (> 10 cycles !!)

• EQAS with true specimens allows control of entire procedure

• detection of flaws in pre-treatment & DNA isolation

• crucial for optimisation of methodology

• Pre-treatment & DNA isolation are crucial for optimal detection of parasites

Overall conclusions

Advantages SKML EQAS MDx Parasites

• Distribution of real parasites!

• Distributed materials are real patient materials (or closely similar)

• Allows control of entire MDx procedure

• Price competitive (cheaper than e.g. QCMD)

• Informative and detailed reports

Informative SKML result reports

Informative SKML result reports:

Detailed information per target

Acknowledgments

Erasmus MC• Rob Koelewijn• Nicolette v.d. Ham• Corné Klaassen• All MDx & Parasitology technicians

SKML section parasitology• Theo Schuurs• Lisette van Lieshout• Theo Mank• Welmoed Silvis

• Eric Brienen• Bert Mulder• Foekje Stelma• Titia Kortbeek

HEMQAS expert team• Lisette van Lieshout• Jaco Verweij• Sitara Rao• Steven Williams• Rojelio Mejia• James McCarthy• Bruno Levecke

All SKML participants