Hypothesis Testing
Summer Institutes
Module 1, Session 6
1
Hypothesis Testing
Summer Institutes
Module 1, Session 6
2
Hypothesis Testing
Motivation
1.
Is the chance of getting a cold different when
participants take vitamin C than when they take
placebo? (Pauling 1971 data).
2.
Suppose that 6 out of 15 students in a grade-school
class develop influenza, whereas 20% of grade-school
children nationwide develop influenza. Is there
evidence of an excessive number of cases in the class?
Summer Institutes
Module 1, Session 6
3
Hypothesis Testing Motivation cont’d
3.
In a study of 49 HIV+ pregnant women, we find
a mean arterial pressure of 79.4 mm Hg. In the
general population of pregnant women, the mean
arterial pressure is 78.8 mm Hg with standard
deviation of 1.4 mm Hg.
•
Are the data from our study consistent with the data
from the general population?
•
What if
= 80.1 mm Hg?
•
What if
= 81.3 mm Hg?
•
What if the sample was of size n=100 instead of
49?
Summer Institutes
Module 1, Session 6
4
Hypothesis Testing
Define:
=
population
mean arterial pressure for
HIV+ pregnant women at 36 weeks
gestation
Hypotheses:
1.
Null Hypothesis
: Generally, the hypothesis that
the unknown parameter equals a fixed value.
H
0
:
= 78.8 mm Hg
2.
Alternative Hypothesis
: contradicts the null
hypothesis.
H
A
:
78.8 mm Hg
Summer Institutes
Module 1, Session 6
5
Hypothesis Testing
Decision and Truth:
We assume that either H
0
or H
A
is true. Based
on the data we will choose one of these
hypotheses.
=
“size”
1 -
= “
power”
Fail to
reject H
0
Reject H
0 ,
Accept H
A
Truth
Decision
Summer Institutes
Module 1, Session 6
6
Hypothesis Testing
Let’s fix
, for example,
= 0.05.
0.05
=
=
P[ choose H
A
| H
0
true ]
=
P[ reject H
0
| H
0
true ]
Q: How to construct a procedure that makes this error with only 0.05 probability?
A: Suppose we assume H
0
is true and suppose that, using that assumption, the data should
give us a standard normal, Z.
If
= 0 then |Z| is rarely “large”. A “large” |Z| would make me question whether
= 0 .
Summer Institutes
Module 1, Session 6
7
Hypothesis Testing
Therefore,
we reject
H
0
if
|Z| > 1.96.
= P[reject
H
0
| H
0
true] = 0.05
Then if we do find a large value of |Z| we can claim that:
Either
H
0
is true and something unusual happened (with probability
)
or
H
0
is not true.
But note that we operate under the assumption that H
0
is true and look for
evidence suggesting it is false. We can reject the null but we can’t prove it
is true. Therefore, we say:
•
If |Z| > 1.96, then we
reject
H
0
and
accept
H
A
.
•
If |Z| < 1.96, then we
fail to reject
H
0
.
Summer Institutes
Module 1, Session 6
8
Hypothesis Testing Example
Mean Arterial Pressure Example:
Let
be the mean arterial pressure for HIV+ pregnant women at 36 weeks. In our sample of
49 women:
=
79.4 mm Hg
Also, we are told that for the general population of pregnant women at 36 weeks...
0
= mean arterial pressure = 78.8 mm Hg
= std. dev. of mean arterial pressure = 1.4 mm Hg
Null hypothesis (H
0
)
: mean for HIV+ pregnant women (
) is the same as the mean for the
general population of pregnant women (
0
).
Alternative hypothesis (H
A
)
: mean for HIV+ pregnant women (
) is different than the mean
for the general population of pregnant women (
0
).
H
0
:
=
0
= 78.8 mm Hg
H
A
:
0
(
78.8 mm Hg)
Summer Institutes
Module 1, Session 6
9
Hypothesis Testing Example
Mean Arterial Pressure Example:
Test H
0
with significance level
= 0.05.
Under
H
0
we know:
Therefore,
•
Reject
H
0
if
> 1.96 gives an
= 0.05 test.
•
Reject
H
0
if
Summer Institutes
Module 1, Session 6
10
Hypothesis Testing Example cont’d
Test:
Reject
H
0
if
= 79.4 therefore we reject the null and accept the alternative
hypothesis.
In terms of Z:
We
reject
H
0
if Z < -1.96 or Z > 1.96
or
or
Summer Institutes
Module 1, Session 6
11
Reject Ho
if
Summer Institutes
Module 1, Session 6
12
Hypothesis Testing and the P-value
p-value
:
•
a measure of how well the data fits the null
hypothesis
•
probability of obtaining a result as extreme or
more extreme than the actual sample (when H
0
is
true)
•
not
the probability that the null (or alternative)
is true
Summer Institutes
Module 1, Session 6
13
Hypothesis Testing and the P-value Example
p-value
: Mean arterial pressure example
= 79.4 mm Hg
n = 49
= 1.4 mm Hg
H
0
:
= 78.8 mm Hg
H
A
:
78.8 mm Hg
p-value is given by: 2 * P[ > 79.4] =0.0027
Summer Institutes
Module 1, Session 6
14
(Somewhat Outdated) Guidelines for Judging the
Significance of p-value
If .05 ≤ p < .10, than the results are
marginally
significant
.
If .01
≤ p < .05, then the results are
significant
.
If .001 ≤ p < .01, then the results are
highly
significant
.
If p < .001, then the results are
very
highly
significant
.
If p > .1, then the results are considered
not statistically
significant
(sometimes denoted by NS).
Significance is not everything!
Summer Institutes
Module 1, Session 6
15
In March 2016, the ASA provided these guidelines:
1.
P-values can indicate how incompatible the data are with a specified statistical
model.
2.
P-values do not measure the probability that the studied hypothesis is true, or the
probability that the data were produced by random chance alone.
3.
Scientific conclusions and business or policy decisions should not be based only
on whether a p-value passes a specific threshold.
4.
Proper inference requires full reporting and transparency.
5.
A p-value, or statistical significance, does not measure the size of an effect or the
importance of a result.
6.
By itself, a p-value does not provide a good measure of evidence regarding a
model or hypothesis.
American Statistical Association Guidelines for the Use of P-values
Summer Institutes
Module 1, Session 6
16
Hypothesis Testing and Confidence Intervals
Hypothesis Test
:
Fail to reject
H
0
if:
Confidence Interval
: Plausible values for
are given by:
Summer Institutes
Module 1, Session 6
17
Hypothesis Testing
“How many sides?”
Depending on the alternative hypothesis, a test may have a
one-sided
alternative
or a
two-sided alternative
. Consider
H
0
:
=
0
We can envision (at least) three possible alternatives
H
A
:
0
(1)
H
A
:
<
0
(2)
H
A
:
>
0
(3)
(1)
is an example of a “two-sided alternative”
(2) and (3) are examples of “one-sided alternatives”
The distinction impacts:
•
Rejection regions
•
p-value calculation
Summer Institutes
Module 1, Session 6
18
1-sided Hypothesis Testing
Mean Arterial Pressure Example
: Instead of the two-sided alternative considered earlier
we may have only been interested in the alternative that HIV+ pregnant women have higher
mean arterial pressure.
H
0
:
= 78.8
H
A
:
> 78.8
Given this, an
= 0.05 test would reject when
We put all the probability on “one-side”.
The p-value would be half of the previous,
p-value = P[ > 79.4]
= 0.0014
1.64
Summer Institutes
Module 1, Session 6
19
Take Home Points
Hypothesis Testing Steps
1.
Identify H
0
and H
A
2.
Identify a test statistic
3.
Determine a significance level,
= 0.05,
=
0.01
4.
Critical value determines rejection / acceptance region
5.
Calculate p-value
6.
Interpret the result
Hypothesis testing in various medical scenarios such as the impact of vitamin C on colds, influenza cases in schools, and arterial pressure in pregnant women. Learn about constructing procedures, making errors, and interpreting results.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Hypothesis Testing Summer Institutes Module 1, Session 6 1
Hypothesis Testing Motivation 1. Is the chance of getting a cold different when participants take vitamin C than when they take placebo? (Pauling 1971 data). 2. Suppose that 6 out of 15 students in a grade-school class develop influenza, whereas 20% of grade-school children nationwide develop influenza. Is there evidence of an excessive number of cases in the class? Summer Institutes Module 1, Session 6 2
Hypothesis Testing Motivation contd 3. In a study of 49 HIV+ pregnant women, we find a mean arterial pressure of 79.4 mm Hg. In the general population of pregnant women, the mean arterial pressure is 78.8 mm Hg with standard deviation of 1.4 mm Hg. Are the data from our study consistent with the data from the general population? X What if = 80.1 mm Hg? What if = 81.3 mm Hg? X What if the sample was of size n=100 instead of 49? Summer Institutes Module 1, Session 6 3
Hypothesis Testing Define: = population mean arterial pressure for HIV+ pregnant women at 36 weeks gestation Hypotheses: 1. Null Hypothesis: Generally, the hypothesis that the unknown parameter equals a fixed value. H0: = 78.8 mm Hg 2. Alternative Hypothesis: contradicts the null hypothesis. HA: 78.8 mm Hg Summer Institutes Module 1, Session 6 4
Hypothesis Testing = size 1 - = power Decision and Truth: We assume that either H0 or HA is true. Based on the data we will choose one of these hypotheses. Truth H0 Correct HA Correct Fail to 1- Decide H0 reject H0 Decision Reject H0 , 1- Decide HA Accept HA Summer Institutes Module 1, Session 6 5
Hypothesis Testing Let s fix , for example, = 0.05. 0.05 = = P[ choose HA | H0 true ] = P[ reject H0 | H0 true ] Q: How to construct a procedure that makes this error with only 0.05 probability? A: Suppose we assume H0 is true and suppose that, using that assumption, the data should give us a standard normal, Z. If = 0 then |Z| is rarely large . A large |Z| would make me question whether = 0 . Summer Institutes Module 1, Session 6 6
Hypothesis Testing Therefore, we reject H0if |Z| > 1.96. = P[reject H0 | H0 true] = 0.05 Then if we do find a large value of |Z| we can claim that: Either H0 is true and something unusual happened (with probability ) or H0 is not true. But note that we operate under the assumption that H0 is true and look for evidence suggesting it is false. We can reject the null but we can t prove it is true. Therefore, we say: If |Z| > 1.96, then we reject H0 and accept HA. If |Z| < 1.96, then we fail to reject H0. Summer Institutes Module 1, Session 6 7
Hypothesis Testing Example Mean Arterial Pressure Example: Let be the mean arterial pressure for HIV+ pregnant women at 36 weeks. In our sample of 49 women: = 79.4 mm Hg X Also, we are told that for the general population of pregnant women at 36 weeks... 0 = mean arterial pressure = 78.8 mm Hg = std. dev. of mean arterial pressure = 1.4 mm Hg Null hypothesis (H0): mean for HIV+ pregnant women ( ) is the same as the mean for the general population of pregnant women ( 0). Alternative hypothesis (HA): mean for HIV+ pregnant women ( ) is different than the mean for the general population of pregnant women ( 0). H0 : = 0 = 78.8 mm Hg HA : 0 ( 78.8 mm Hg) Summer Institutes Module 1, Session 6 8
Hypothesis Testing Example Mean Arterial Pressure Example: Test H0 with significance level = 0.05. Under H0 we know: X ( ) 1 , 0 ~ o N / n Therefore, X > 1.96 gives an = 0.05 test. n / 96 . 1 0 n Reject H0 if Reject H0 if 0 + or X . 1 96 X 0 n Summer Institutes Module 1, Session 6 9
Hypothesis Testing Example contd Test: Reject H0 if or or X = 79.4 therefore we reject the null and accept the alternative hypothesis. X = 0 Z We reject H0 if Z < -1.96 or Z > 1.96 In terms of Z: / n Summer Institutes Module 1, Session 6 10
Reject Ho if X Summer Institutes Module 1, Session 6 11
Hypothesis Testing and the P-value p-value: a measure of how well the data fits the null hypothesis probability of obtaining a result as extreme or more extreme than the actual sample (when H0 is true) not the probability that the null (or alternative) is true Summer Institutes Module 1, Session 6 12
Hypothesis Testing and the P-value Example p-value: Mean arterial pressure example X = 79.4 mm Hg n = 49 = 1.4 mm Hg H0 : = 78.8 mm Hg HA : 78.8 mm Hg X p-value is given by: 2 * P[ > 79.4] =0.0027 78.2 79.4 Mean arterial pressure Summer Institutes Module 1, Session 6 13
(Somewhat Outdated) Guidelines for Judging the Significance of p-value If .05 p < .10, than the results are marginally significant. If .01 p < .05, then the results are significant. If .001 p < .01, then the results are highlysignificant. If p < .001, then the results are veryhighlysignificant. If p > .1, then the results are considered not statistically significant (sometimes denoted by NS). Significance is not everything! Summer Institutes Module 1, Session 6 14
American Statistical Association Guidelines for the Use of P-values In March 2016, the ASA provided these guidelines: 1. P-values can indicate how incompatible the data are with a specified statistical model. 2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone. 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold. 4. Proper inference requires full reporting and transparency. 5. A p-value, or statistical significance, does not measure the size of an effect or the importance of a result. 6. By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis. Summer Institutes Module 1, Session 6 15
Hypothesis Testing and Confidence Intervals Hypothesis Test: Fail to reject H0 if: 2 1 + X Q 0 Z n 2 1 and X Q 0 Z n Confidence Interval: Plausible values for are given by: 2 1 + X Q Z n 2 1 and X Q Z n Summer Institutes Module 1, Session 6 16
Hypothesis Testing How many sides? Depending on the alternative hypothesis, a test may have a one-sided alternative or a two-sided alternative. Consider H0 : = 0 We can envision (at least) three possible alternatives HA : 0 (1) HA : < 0 (2) HA : > 0 (3) (1) is an example of a two-sided alternative (2) and (3) are examples of one-sided alternatives The distinction impacts: Rejection regions p-value calculation Summer Institutes Module 1, Session 6 17
1-sided Hypothesis Testing Mean Arterial Pressure Example: Instead of the two-sided alternative considered earlier we may have only been interested in the alternative that HIV+ pregnant women have higher mean arterial pressure. H0 : = 78.8 HA : > 78.8 Given this, an = 0.05 test would reject when X ( Z ) . 0 = . 1 = 1 05 0 65 1.64 Z Q / n We put all the probability on one-side . The p-value would be half of the previous, X p-value = P[ > 79.4] = 0.0014 Summer Institutes Module 1, Session 6 18
Take Home Points Hypothesis Testing Steps 1. Identify H0 and HA 2. Identify a test statistic 3. Determine a significance level, = 0.05, = 0.01 4. Critical value determines rejection / acceptance region 5. Calculate p-value 6. Interpret the result Summer Institutes Module 1, Session 6 19
