7/24/2019 ROC Curves

1/3

ROC Curve Homework

There were 600 cases from a defined population that had been subjected to some diagnostic Test

A, Test B and Test C.

In Test A, 120 people test result was positive and 480 people test result was negative. Supposethat 100 actually positive cases and 500 actually negative cases were ultimately found in the

population studied and that the diagnostic test to be evaluated yielded 70 true positive (TP)

decisions, 30 false negative (FN) decision, 450 true negative (TN) decisions and 50 false positive

(FP) decisions.

In Test B, 60 people test result was positive and 540 people test result was negative. Suppose

that 100 actually positive cases and 500 actually negative cases were ultimately found in the

population studied and that the diagnostic test to be evaluated yielded 40 TP decisions, 60 FN

decision, 480 TN decisions and 20 FP decisions.

In Test C, 70 people test result was positive and 530 people test result was negative. Suppose

that 100 actually positive cases and 500 actually negative cases were ultimately found in the

population studied and that the diagnostic test to be evaluated yielded 45 TP decisions, 55 FN

decision, 475 TN decisions and 25 FP decisions.

Based on the materials learned in the class,

1) Please first summarize these data in separated decision matrix and calculate Accuracy

for Test A, Test B and Test C.

2) Please compare the accuracy of Test A, B and C and tell which test is best and which test

is worst?

3) If accuracy measure cant help completely answer question 2), please draw simple ROC

curves to tell.

Answers:

1)

TestAresult

Positive(T+)

Negative(T

)

TotalActual

States

Actualresult

Positive(D+) 70 30 100

Negative(D) 50 450 500

TotalAtestresults 120 480

Calculated indices

7/24/2019 ROC Curves

2/3

TPF=70/100=0.7

FNF=1-TPF=0.3

FPF=50/500=0.1

TNF-1-FPF=0.9

P(D+)=100/600=0.17, P(D-)=1-P(D+)=0.83

Accuracy=TPF*P(D+)+TNF*P(D-)=0.7*0.17+0.9*0.83=0.866

TestBresult

Positive(T+) Negative(T) TotalActualStates

Actualresult

Positive(D+) 40 60 100

Negative(D) 20 480 500

TotalB

test

results

60

540

Calculated indices

TPF=40/100=0.4

FNF=1-TPF=0.6

FPF=20/500=0.04

TNF-1-FPF=0.96

P(D+)=100/600=0.17, P(D-)=1-P(D+)=0.83

Accuracy=TPF*P(D+)+TNF*P(D-)=0.4*0.17+0.96*0.83=0.865

TestCresult

Positive(T+) Negative(T) TotalActualStates

Actualresult

Positive(D+) 45 55 100

Negative(D) 25 475 500

TotalBtestresults 70 530

TPF=45/100=0.45FNF=1-TPF=0.55

FPF=25/500=0.05

TNF-1-FPF=0.95

P(D+)=100/600=0.17, P(D-)=1-P(D+)=0.83

Accuracy=TPF*P(D+)+TNF*P(D-)=0.45*0.17+0.95*0.83=0.8930.865

7/24/2019 ROC Curves

3/3

2) Based on the calculated indices, accuracy of Test C is highest, 0.893, so Test C is best among

the three tests; however, it is very hard to which is better between Test A and Test B, because the

accuracy of Test A, 0.866 is very close to the accuracy of Test B, 0.865.

3)

Based on the ROC curve, the area under Test A is much larger than the one under Test B. So,

Test A is better than Test B.

0.04,0.4

1,1

0.1,0.7

1,1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

TPF

FPF

ROCCurve

TestB

TestA

2) As all the accuracy values are very close to each other, it is very hard to tell whether which test is better.