Validity of screening test
Validity of screening test
P
r
o
f
.
J
a
s
i
m
A
l
-
A
s
a
d
i
Learning objectives
•
At the end of this lecture, students will be
able to:
1.
describe elements of screening tests
2.
calculate sensitivity, specificity and positive
predictive value
3.
explain the implications of false positives
and false negatives
4.
understand predictive values
What is validity?
•
The term validity refers to
what extent
the test
accurately measures
what it is
supposed to
measure.
•
In other words, validity expresses
the ability
of
a test to separate or
distinguish
those who
have the disease from those who do not have
it.
•
Validity has two components :
sensitivity
and
specificity
•
Sensitivity
is the ability of a screening procedure to
correctly identify
those who
have the disease
.
The proportion of persons
with
the disease who have a
positive
test
result is “
True positive
”
It is 100% sensitive if it is positive in every case.
Cases
which are missed are called “
False negative
”.
•
Specificity
is the ability of a screening procedure to
correctly identify
those who
do not have
the disease.
The proportion of persons
without
the disease who have a
negative
test
result is “
True negative
”.
Non-diseased
individuals who are tested positive are called “
False
positive
”
S
e
n
s
i
t
i
v
i
t
y
a
n
d
S
p
e
c
i
f
i
c
i
t
y
Calculation of sensitivity & Specificity
Measures of the Validity of Screening
Tests
(cont’d)
•
Positive Predictive value
: The proportion
(percentage) of
true diseased (positive)
individuals correctly identified by the screening
test out of the total positives by the screening
test (a/a+b x100).
•
Negative Predictive value
: The proportion
(percentage ) of
true negative
people correctly
detected by the screening test out of the total
negatives by the screening test (d/c+d x100).
Predictive values
Positive predictive value
= A / (A+B)x100
Negative predictive value=
D / (C+D)X100
2
X
2
t
a
b
l
e
Other Measures from the 2 x 2 Table
•
Accuracy of a screening test (Agreement rate
):
determined by the formula: (A+D)/(A+B+C+D) x 100.
•
Disagreement (misclassification) rate
:
=
B+C/(A+B+C+D) x100
•
Prevalence (True)--determined by the formula:
(A+C)
/(A+B+C+D) x1000
•
Prevalence according to the screening results:
(A+B)/(A+B+C+D) x1000
H
i
g
h
l
y
s
e
n
s
i
t
i
v
e
t
e
s
t
•
Highly sensitive test
is needed in screening for
a rare and fatal disease
, because we don’t
want to miss any case of the disease.
•
e.g. in myocardial infarction and cancer we
need a highly sensitive screening test.
Implication or significance of high
false negative rate
1. Severe cases of the disease will be deprived
from appropriate treatment. This will lead to
more complications and death.
2. False reassurance of patients. This disagrees
with characteristics of the screening test
which should diagnose the disease at early
stage.
Highly specific test
•
A highly specific test is required for screening
of a fairly common and non fatal disease,
because we don’t want high false positive rate
to avoid excessive over-diagnosis and its
consequences (treatment, side effects of
drugs)
•
e.g. in Diabetes mellitus, we need a highly
specific screening test.
Implication or significance of high
false positive rate
1. Treatment may be given to non-diseased
persons with its sequences:
a. Side effects of drugs
b. High cost of drugs
2. The test may be harmful and invasive
3. Psychologically harmful to healthy person.
Effects of Disease Prevalence on the
Predictive Value of a Screening Test
•
When the prevalence of a disease increases,
the positive predictive value rises, and the
negative predictive value falls.
Effect of Prevalence on Predictive Value Positive
with Constant Sensitivity and Specificity
Prevalence
PV+ (%) Sensitivity
Specificity
(%)
(%)
(%)
0.1
1.8
90
95
1.0
15.4
90
95
5.0
48.6
90
95
50.0
94.7
90
95
Prevalence & Predictive Value
Relationship Between cut-off point of measure
& sensitivity and specificity
•
The sensitivity and specificity are generally
constant and don’t change with change of
prevalence of the disease, but they change in
relation to each other if the
cut-off point
of
reading is changed
Relationship Between Sensitivity and
Specificity
•
To improve sensitivity, the cut-off point used
to classify individuals as diseased should be
moved farther in the range of the non-
diseased (normal).
•
To improve specificity, the cut-off point should
be moved farther in the range typically
associated with the disease.
X= Cut-off value
Relationship Between cut-off point of measure & sensitivity
and specificity
N
o
n
-
d
i
s
e
a
s
e
d
c
a
s
e
s
D
i
s
e
a
s
e
d
c
a
s
e
s
F
P
r
a
t
e
S
p
e
c
i
f
i
c
i
t
y
High cut-off point
Low cut-off point
Lessons Learned
•
Different cut-points yield different sensitivities and
specificities
•
The cut-point determines how many subjects will be
considered as having the disease
•
The cut-point that identifies more true negatives will also
identify more false negatives
•
The cut-point that identifies more true positives will also
identify more false positives
Where to Draw the Cut-Point
•
If the diagnostic (confirmatory) test is expensive or invasive:
− Minimize false positives or
− Use a cut-point with high specificity
•
If the penalty for missing a case is high (e.g., the disease is
fatal and treatment exists, or disease easily spreads):
− Maximize true positives X
That is, use a cut-point with high sensitivity
•
Balance severity of false positives against false negatives
Multiple Testing
Use of Multiple Tests
•
Commonly done in medical practice
•
Choices depend on cost, invasiveness, volume
of test, presence and capability of lab
infrastructure, urgency, etc.
•
Can be done sequentially or simultaneously
Sequential Testing
(Two-Stage Screening)
•
After the first (screening) test was conducted,
those who tested positive were brought back
for the second test to further reduce false
positives
•
Consequently, the overall process will increase
specificity but with reduced sensitivity
Example of a Two-Stage Screening Program: Test
1 (Blood Sugar)
•
Test 1 (blood sugar), assume:
− Disease prevalence = 5%, population = 10,000
− Sensitivity = 70%, specificity = 80%
− Screen positives from the first test
Example of a Two-Stage Screening Program: Test 2
(Glucose Tolerance Test)
Example of a Two-Stage Screening Program: Test 2
(Glucose Tolerance Test)
T
w
o
-
S
t
a
g
e
S
c
r
e
e
n
i
n
g
:
R
e
-
S
c
r
e
e
n
t
h
e
P
o
s
i
t
i
v
e
s
f
r
o
m
t
h
e
F
i
r
s
t
T
e
s
t
Net Sensitivity & Net specificity in a Two-Stage
Screening when Test + in the First Test Are Re-Screened
•
Net sensitivity = Sensitivity 1 x Sensitivity 2
•
Net specificity = Spec1 + [Spec2 – (Spec1 x Spec2)]
Other Two-Stage Screening
Simultaneous Testing
•
When two (or more) tests are conducted in
parallel
•
The goal is to maximize the probability that
subjects with the disease (true positives) are
identified (increase sensitivity)
•
Consequently, more false positives are also
identified (decrease specificity)
Simultaneous Testing: Calculate Net Sensitivity
•
When two tests are used simultaneously, disease
positives are defined as those who test positive by
either one test or by both tests
•
We use the addition rule of probability to calculate
the net sensitivity
Net
Sensitivity
= Sens 1 + [Sens 2 – (Sens 1 x Sens 2)]
Simultaneous Testing: Calculate Net Specificity
•
When two tests are used simultaneously, disease
negatives are defined as those who test negative by
both tests
•
We use the multiplication rule of probability to
calculate the net specificity
Net specificity = specificity test 1 x specificity test 2
Example of a Simultaneous Testing
Net Gain and Net Loss
•
In simultaneous testing, there is a net gain in
sensitivity but a net loss in specificity, when
compared to either of the tests used
•
In sequential testing when positives from the
first test are retested, there is a net loss in
sensitivity but a net gain in specificity,
compared to either of the tests used
Review
•
Test A is known to have the following characteristics:
− Sensitivity of 80%
− Specificity of 90%
− Cost of $15 per test
Suppose the following:
− Test A is used in a population of 10,000 to identify
individuals who have the disease
− The prevalence of the disease is 5%
What are the net sensitivity, net specificity, and cost per
positive case when:
(1)
Test A is used twice simultaneously and when
(2)
a single Test A is used first, and individuals who test
positive with Test A are tested again with Test A
(sequentially
Screening tests are essential in healthcare to identify diseases accurately. Learn about sensitivity, specificity, and predictive values to evaluate test validity. Discover how to calculate these measures and understand the implications of false results.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Validity of screening test Prof. Jasim Al-Asadi
Learning objectives At the end of this lecture, students will be able to: 1. describe elements of screening tests 2. calculate sensitivity, specificity and positive predictive value 3. explain the implications of false positives and false negatives 4. understand predictive values
What is validity? The term validity refers to what extent the test accurately measures what it is supposed to measure. In other words, validity expresses the ability of a test to separate or distinguish those who have the disease from those who do not have it. Validity has two components : sensitivity and specificity
Sensitivity is the ability of a screening procedure to correctly identify those who have the disease. The proportion of persons with the disease who have a positive test result is True positive It is 100% sensitive if it is positive in every case. Cases which are missed are called False negative . Specificity is the ability of a screening procedure to correctly identify those who do not have the disease. The proportion of persons without the disease who have a negative test result is True negative . Non-diseased individuals who are tested positive are called False positive
Sensitivity and Specificity Standard test Results/ Disease Total Present Absent True Positive (A) False Positive (B) Present A + B Screening test Results False negative (C) True Negative (D) Absent C + D Total A + C B + D A+B+C+D
Calculation of sensitivity & Specificity TP TN Specificity= Sensitivity= TP + FN TN + FP
False negative and False positive Standard test/ Disease Present Absent Total True positive A False negative C A + C False positive B True negative D B + D Positive Test Results A + B Negative C + D Total A+B +C+D False negative = C / (A+C)x100 Or = 100% - Sensitivity False positive = B / (B+D)x100 Or = 100% - Specificity
Measures of the Validity of Screening Tests (cont d) Positive Predictive value: The proportion (percentage) of true diseased (positive) individuals correctly identified by the screening test out of the total positives by the screening test (a/a+b x100). Negative Predictive value: The proportion (percentage ) of true negative people correctly detected by the screening test out of the total negatives by the screening test (d/c+d x100).
Predictive values Standard test Results/ Disease Total Present Absent True Positive (A) False Positive (B) Present A + B Screening test Results False negative (C) True Negative (D) Absent C + D Total A + C B + D A+B+C+D Positive predictive value = A / (A+B)x100 Negative predictive value= D / (C+D)X100
2 X 2 table Disease + - Positive predictive value + Test Negative predictive value - Sensitivity Specificity
Other Measures from the 2 x 2 Table Accuracy of a screening test (Agreement rate): determined by the formula: (A+D)/(A+B+C+D) x 100. Disagreement (misclassification) rate: = B+C/(A+B+C+D) x100 Prevalence (True)--determined by the formula: (A+C) /(A+B+C+D) x1000 Prevalence according to the screening results: (A+B)/(A+B+C+D) x1000
Highly sensitive test Highly sensitive test is needed in screening for a rare and fatal disease, because we don t want to miss any case of the disease. e.g. in myocardial infarction and cancer we need a highly sensitive screening test.
Implication or significance of high false negative rate 1. Severe cases of the disease will be deprived from appropriate treatment. This will lead to more complications and death. 2. False reassurance of patients. This disagrees with characteristics of the screening test which should diagnose the disease at early stage.
Highly specific test A highly specific test is required for screening of a fairly common and non fatal disease, because we don t want high false positive rate to avoid excessive over-diagnosis and its consequences (treatment, side effects of drugs) e.g. in Diabetes mellitus, we need a highly specific screening test.
Implication or significance of high false positive rate 1. Treatment may be given to non-diseased persons with its sequences: a. Side effects of drugs b. High cost of drugs 2. The test may be harmful and invasive 3. Psychologically harmful to healthy person.
Effects of Disease Prevalence on the Predictive Value of a Screening Test When the prevalence of a disease increases, the positive predictive value rises, and the negative predictive value falls.
Effect of Prevalence on Predictive Value Positive with Constant Sensitivity and Specificity Prevalence (%) PV+ (%) Sensitivity Specificity (%) (%) 0.1 1.0 5.0 50.0 1.8 15.4 48.6 94.7 90 90 90 90 95 95 95 95
Prevalence & Predictive Value Note: Test has 95% sensitivity and 95% specificity
Relationship Between cut-off point of measure & sensitivity and specificity The sensitivity and specificity are generally constant and don t change with change of prevalence of the disease, but they change in relation to each other if the cut-off point of reading is changed
Relationship Between Sensitivity and Specificity To improve sensitivity, the cut-off point used to classify individuals as diseased should be moved farther in the range of the non- diseased (normal). To improve specificity, the cut-off point should be moved farther in the range typically associated with the disease.
Relationship Between cut-off point of measure & sensitivity and specificity Non-diseased (TN) specificity Diseased (TP) sensitivity Cutoff point
High cut-off point Diseased cases Non-diseased cases FP rate Specificity
Low cut-off point Diseased Centers Non-diseased Centers TP rate Sensitivity
Lessons Learned Different cut-points yield different sensitivities and specificities The cut-point determines how many subjects will be considered as having the disease The cut-point that identifies more true negatives will also identify more false negatives The cut-point that identifies more true positives will also identify more false positives
Where to Draw the Cut-Point If the diagnostic (confirmatory) test is expensive or invasive: Minimize false positives or Use a cut-point with high specificity If the penalty for missing a case is high (e.g., the disease is fatal and treatment exists, or disease easily spreads): Maximize true positives X That is, use a cut-point with high sensitivity Balance severity of false positives against false negatives
Use of Multiple Tests Commonly done in medical practice Choices depend on cost, invasiveness, volume of test, presence and capability of lab infrastructure, urgency, etc. Can be done sequentially or simultaneously
Sequential Testing (Two-Stage Screening) After the first (screening) test was conducted, those who tested positive were brought back for the second test to further reduce false positives Consequently, the overall process will increase specificity but with reduced sensitivity
Example of a Two-Stage Screening Program: Test 1 (Blood Sugar) Test 1 (blood sugar), assume: Disease prevalence = 5%, population = 10,000 Sensitivity = 70%, specificity = 80% Screen positives from the first test
Example of a Two-Stage Screening Program: Test 2 (Glucose Tolerance Test)
Example of a Two-Stage Screening Program: Test 2 (Glucose Tolerance Test)
Two-Stage Screening: Re-Screen the Positives from the First Test
Net Sensitivity & Net specificity in a Two-Stage Screening when Test + in the First Test Are Re-Screened Net sensitivity = Sensitivity 1 x Sensitivity 2 Net specificity = Spec1 + [Spec2 (Spec1 x Spec2)]
Simultaneous Testing When two (or more) tests are conducted in parallel The goal is to maximize the probability that subjects with the disease (true positives) are identified (increase sensitivity) Consequently, more false positives are also identified (decrease specificity)
Simultaneous Testing: Calculate Net Sensitivity When two tests are used simultaneously, disease positives are defined as those who test positive by either one test or by both tests We use the addition rule of probability to calculate the net sensitivity Net Sensitivity= Sens 1 + [Sens 2 (Sens 1 x Sens 2)]
Simultaneous Testing: Calculate Net Specificity When two tests are used simultaneously, disease negatives are defined as those who test negative by both tests We use the multiplication rule of probability to calculate the net specificity Net specificity = specificity test 1 x specificity test 2
Net Gain and Net Loss In simultaneous testing, there is a net gain in sensitivity but a net loss in specificity, when compared to either of the tests used In sequential testing when positives from the first test are retested, there is a net loss in sensitivity but a net gain in specificity, compared to either of the tests used
Review Test A is known to have the following characteristics: Sensitivity of 80% Specificity of 90% Cost of $15 per test Suppose the following: Test A is used in a population of 10,000 to identify individuals who have the disease The prevalence of the disease is 5% What are the net sensitivity, net specificity, and cost per positive case when: (1) Test A is used twice simultaneously and when (2) a single Test A is used first, and individuals who test positive with Test A are tested again with Test A (sequentially
