Hypothesis Testing in Statistics
undefined
Hypothesis Testing:
Hypothesis Testing:
Minding your Ps and Qs and Error
Minding your Ps and Qs and Error
Rates
Rates
Bonnie Halpern-Felsher, PhD
Bonnie Halpern-Felsher, PhD
H
y
p
o
t
h
e
s
i
s
T
e
s
t
i
n
g
•
Hypothesis testing is the key to our scientific
inquiry.
•
I
n
a
d
d
i
t
i
o
n
a
l
t
o
r
e
s
e
a
r
c
h
h
y
p
o
t
h
e
s
e
s
,
n
e
e
d
s
t
a
t
i
s
t
i
c
a
l
h
y
p
o
t
h
e
s
e
s
.
•
Involves the statement of a null hypothesis, an
alternative hypothesis, and the selection of a
level of significance.
S
t
a
t
i
s
t
i
c
a
l
H
y
p
o
t
h
e
s
e
s
•
S
t
a
t
e
m
e
n
t
s
o
f
c
i
r
c
u
m
s
t
a
n
c
e
s
i
n
t
h
e
p
o
p
u
l
a
t
i
o
n
t
h
a
t
t
h
e
s
t
a
t
i
s
t
i
c
a
l
p
r
o
c
e
s
s
w
i
l
l
e
x
a
m
i
n
e
a
n
d
d
e
c
i
d
e
t
h
e
l
i
k
e
l
y
t
r
u
t
h
o
r
v
a
l
i
d
i
t
y
•
Statistical hypotheses are discussed in terms of
the population, not the sample, yet tested on
samples
•
Based on the mathematical concept of
probability
•
Null Hypothesis
•
Alternative Hypothesis
N
u
l
l
H
y
p
o
t
h
e
s
i
s
•
What is the Null Hypothesis?
N
u
l
l
H
y
p
o
t
h
e
s
i
s
•
The case when the two groups are equal;
population means are the same
•
Null Hypothesis = H
0
•
This is the hypothesis actually being tested
•
H
0
is assumed to be true
A
l
t
e
r
n
a
t
i
v
e
H
y
p
o
t
h
e
s
i
s
•
What is the Alternative Hypothesis?
A
l
t
e
r
n
a
t
i
v
e
H
y
p
o
t
h
e
s
i
s
•
The case when the two groups are not equal;
when there is some treatment difference; when
other possibilities exist
•
Alternative Hypothesis = H
1
Or
H
a
•
H
1
is
assumed
to be true when the H
0
is false.
S
t
a
t
i
s
t
i
c
a
l
H
y
p
o
t
h
e
s
e
s
•
The H
0
and H
1
must be mutually exclusive
•
The H
0
and H
1
must be exhaustive; that is, no
other possibilities can exist
•
T
h
e
H
1
c
o
n
t
a
i
n
s
o
u
r
r
e
s
e
a
r
c
h
h
y
p
o
t
h
e
s
e
s
S
t
a
t
i
s
t
i
c
a
l
H
y
p
o
t
h
e
s
e
s
•
Can you give an example of a Null and
Alternative Hypothesis?
N
u
l
l
H
y
p
o
t
h
e
s
i
s
•
H
0
•
There is no treatment effect
•
The drug has no effect
•
H
1
•
There is a treatment effect
•
The drug had an (some, any) effect
E
v
a
l
u
a
t
i
o
n
o
f
t
h
e
N
u
l
l
•
In order to gain support for our research
hypothesis, we must reject the Null Hypothesis
•
Thereby concluding that the alternative
hypothesis (likely) reflects what is going on in
the population.
•
You can never “prove” the Alternative
Hypothesis!
S
i
g
n
i
f
i
c
a
n
c
e
L
e
v
e
l
•
Need to decide on a Significance Level:
•
The probability that the test statistic will reject
the null hypothesis when the null hypothesis
is true
•
Significance is a property of the distribution
of a test statistic, not of any particular draw of
the statistic
•
Determines the Region of Rejection
•
Generally 5% or 1%
A
l
p
h
a
L
e
v
e
l
•
The value of alpha (
α)
is associated with the
confidence level of our test; significance level.
•
For results with a 90% level of confidence, the
value of
α
is 1 - 0.90 = 0.10.
•
For results with a 95% level of confidence, the
value of alpha is 1 - 0.95 = 0.05.
•
Typically set at 5% (.05) or 1% (.01)
p
-
v
a
l
u
e
•
T
h
e
p
-
v
a
l
u
e
,
o
r
c
a
l
c
u
l
a
t
e
d
p
r
o
b
a
b
i
l
i
t
y
,
i
s
t
h
e
e
s
t
i
m
a
t
e
d
p
r
o
b
a
b
i
l
i
t
y
o
f
r
e
j
e
c
t
i
n
g
t
h
e
n
u
l
l
h
y
p
o
t
h
e
s
i
s
(
H
0
)
o
f
a
s
t
u
d
y
q
u
e
s
t
i
o
n
w
h
e
n
t
h
a
t
h
y
p
o
t
h
e
s
i
s
i
s
t
r
u
e
•
Probability that the observed statistic occurred
by chance alone
Obtaining Significance
•
Compare the values of alpha and the
p
-value. There
are two possibilities that emerge:
–
The
p
-value is less than or equal to alpha
(e.g.,
p
<
.05). In this case we reject the null hypothesis. When
this happens we say that the result is statistically
significant. In other words, we are reasonably sure
that there is something besides chance alone that
gave us an observed sample.
–
The
p
-value is greater than alpha
(e.g., p > .05). In
this case we fail to reject the H
0
. Therefore, not
statistically significant. Observed data are likely due
to chance alone.
Error
•
In an ideal world we would always reject the null
hypothesis when it is false, and we would not reject
the null hypothesis when it is indeed true. But there
are two other scenarios that are possible, each of
which will result in an error
•
Type I and Type II Errors
Type I Error
•
Rejection of the null hypothesis that is actually true
•
Same as a “false positive”
•
The alpha value gives us the probability of a Type I
error.
–
For example,
α
= .05 = 5% chance of rejecting a true
null hypothesis
Type I Error
•
Example:
–
H
0
= drug has no effect
–
H
1
= drug has an effect
–
Reject H
0
and instead claim H
1
is correct, so claim
that the drug has an effect when indeed it does n
o
t.
–
Therefore the drug is falsely claimed to have an
effect.
Controlling Type I Error
•
Alpha is the maximum probability of having a Type I
error.
–
E.g., 95% CI, chance of having a Type I is 5%
–
Therefore, a 5% chance of rejecting H
0
when H
0
is true
–
That is, 1 out of 20 hypotheses tested will result in Type I error
•
We can control Type I error by setting a different
α
level.
Controlling Type I Error
•
Particularly important to change α level to be more
conservative if calculating several statistical tests and
comparisons.
•
We have a 5% chance of getting a significant result just
by chance. So, if running 10 comparisons, should set
a more conservative α level to control for Type I error
–
Bonferroni correction: .05/10 = .005
α
Type II Error
•
We do not reject a null hypothesis that is false.
•
Like a false negative
•
E.g., thought the drug had no effect, when it
actually did
•
The probability of a Type II error is given by the
Greek letter beta (
β)
. This number is related to the
power or sensitivity of the hypothesis test, denoted
by 1 –
β
Controlling Type II Error:
Power
•
Power:
The power of a test sometimes, less formally,
refers to the probability of rejecting the null when it is
not correct.
•
Power = P(reject H
0
|H
1
is true) = P(accept H
1
|H
1
is
true)
•
As the power increases, the chances of a Type II
error (false negative; β) decreases.
•
Power = 1-β
•
Power increases with sample size
Type I and II Errors
True/Reality
Decision
Type I and II Errors
True/Reality
Decision
The probability of making an error where you “
decide H
1
but H
0
is
true”
is the α, then the probability of being correct, given that H
0
is
true, is 1 – α. Similarly, the probability of making an error where you
“decide H
0
yet H
1
is true”
is β, therefore the probability of making a
correct decision given that H
1
is true is 1 - β
Questions????
Hypothesis testing is essential in scientific inquiry, involving the formulation of null and alternative hypotheses at a chosen level of significance. Statistical hypotheses focus on population characteristics and are tested on samples using probability concepts. The null hypothesis assumes no effect, while the alternative hypothesis considers possible treatment effects or differences. These hypotheses must be mutually exclusive and exhaustive, with the alternative hypothesis containing the research predictions.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Hypothesis Testing: Minding your Ps and Qs and Error Rates Bonnie Halpern-Felsher, PhD
Hypothesis Testing Hypothesis testing is the key to our scientific inquiry. In additional to research hypotheses, need statistical hypotheses. Involves the statement of a null hypothesis, an alternative hypothesis, and the selection of a level of significance. 2
Statistical Hypotheses Statements of circumstances in the population that the statistical process will examine and decide the likely truth or validity Statistical hypotheses are discussed in terms of the population, not the sample, yet tested on samples Based on the mathematical concept of probability Null Hypothesis Alternative Hypothesis 3
Null Hypothesis What is the Null Hypothesis? 4
Null Hypothesis The case when the two groups are equal; population means are the same Null Hypothesis = H0 This is the hypothesis actually being tested H0 is assumed to be true 5
Alternative Hypothesis What is the Alternative Hypothesis? 6
Alternative Hypothesis The case when the two groups are not equal; when there is some treatment difference; when other possibilities exist Alternative Hypothesis = H1Or Ha H1 is assumed to be true when the H0 is false. 7
Statistical Hypotheses The H0 and H1 must be mutually exclusive The H0 and H1 must be exhaustive; that is, no other possibilities can exist The H1 contains our research hypotheses 8
Statistical Hypotheses Can you give an example of a Null and Alternative Hypothesis? 9
Null Hypothesis H0 There is no treatment effect The drug has no effect H1 There is a treatment effect The drug had an (some, any) effect 10
Evaluation of the Null In order to gain support for our research hypothesis, we must reject the Null Hypothesis Thereby concluding that the alternative hypothesis (likely) reflects what is going on in the population. You can never prove the Alternative Hypothesis! 11
Significance Level Need to decide on a Significance Level: The probability that the test statistic will reject the null hypothesis when the null hypothesis is true Significance is a property of the distribution of a test statistic, not of any particular draw of the statistic Determines the Region of Rejection Generally 5% or 1% 12
Alpha Level The value of alpha ( ) is associated with the confidence level of our test; significance level. For results with a 90% level of confidence, the value of is 1 - 0.90 = 0.10. For results with a 95% level of confidence, the value of alpha is 1 - 0.95 = 0.05. Typically set at 5% (.05) or 1% (.01) 13
p-value The p-value, or calculated probability, is the estimated probability of rejecting the null hypothesis (H0) of a study question when that hypothesis is true Probability that the observed statistic occurred by chance alone 14
Obtaining Significance Compare the values of alpha and the p-value. There are two possibilities that emerge: The p-value is less than or equal to alpha (e.g., p < .05). In this case we reject the null hypothesis. When this happens we say that the result is statistically significant. In other words, we are reasonably sure that there is something besides chance alone that gave us an observed sample. The p-value is greater than alpha (e.g., p > .05). In this case we fail to reject the H0. Therefore, not statistically significant. Observed data are likely due to chance alone. 16
Error In an ideal world we would always reject the null hypothesis when it is false, and we would not reject the null hypothesis when it is indeed true. But there are two other scenarios that are possible, each of which will result in an error Type I and Type II Errors 17
Type I Error Rejection of the null hypothesis that is actually true Same as a false positive The alpha value gives us the probability of a Type I error. For example, = .05 = 5% chance of rejecting a true null hypothesis 18
Type I Error Example: H0 = drug has no effect H1 = drug has an effect Reject H0 and instead claim H1 is correct, so claim that the drug has an effect when indeed it does not. Therefore the drug is falsely claimed to have an effect. 19
Controlling Type I Error Alpha is the maximum probability of having a Type I error. E.g., 95% CI, chance of having a Type I is 5% Therefore, a 5% chance of rejecting H0 when H0 is true That is, 1 out of 20 hypotheses tested will result in Type I error We can control Type I error by setting a different level. 20
Controlling Type I Error Particularly important to change level to be more conservative if calculating several statistical tests and comparisons. We have a 5% chance of getting a significant result just by chance. So, if running 10 comparisons, should set a more conservative level to control for Type I error Bonferroni correction: .05/10 = .005 21
Type II Error We do not reject a null hypothesis that is false. Like a false negative E.g., thought the drug had no effect, when it actually did The probability of a Type II error is given by the Greek letter beta ( ). This number is related to the power or sensitivity of the hypothesis test, denoted by 1 22
Controlling Type II Error: Power Power: The power of a test sometimes, less formally, refers to the probability of rejecting the null when it is not correct. Power = P(reject H0|H1 is true) = P(accept H1|H1 is true) As the power increases, the chances of a Type II error (false negative; ) decreases. Power = 1- Power increases with sample size 23
Type I and II Errors True/Reality H0 1- H1 Decision H0 H1 1- 24
Type I and II Errors True/Reality H0 1- H1 Decision H0 H1 1- The probability of making an error where you decide H1 but H0 is true is the , then the probability of being correct, given that H0 is true, is 1 . Similarly, the probability of making an error where you decide H0 yet H1is true is , therefore the probability of making a correct decision given that H1 is true is 1 - 25
