FACTORIAL ANOVA
undefined
FACTORIAL ANOVA
Chapter 14
Adapted from Jamison Fargo, PhD EDUC 6600 Slides
undefinedO
l
i
v
e
r
W
e
n
d
e
l
l
H
o
l
m
e
s
•
A
m
e
r
i
c
a
n
P
h
y
s
i
c
i
a
n
,
W
r
i
t
e
r
,
H
u
m
o
r
i
s
t
,
H
a
r
v
a
r
d
P
r
o
f
e
s
s
o
r
•
1
8
0
9
-
1
8
9
4
‘People can be divided into two classes:
Those who go ahead and do something,
and those who sit still and inquire,
'Why wasn't it done the other way?’’
Dr. Petrov is interested in
conducting an experiment where:
30 high school students are randomly assigned
to a new
computer simulation tool
for learning
geometry and
30 other students are randomly assigned to the
standard
lecture and paper/pencil problem
solving format.
However, Dr. Petrov is
also
interested in the
effect
of sex
differences on learning outcomes.
3
•
Independent ANOVA types…
1-Way ANOVA = 1 factor
(previously covered)
2-Way ANOVA = 2 factors
(focus of lecture)
3-Way ANOVA = 3 factors
4-Way ANOVA = 4 factors
4
The
# levels
of each factor determines
ANOVA
design
Row factor =
2
levels
Column factor =
3
levels
2-way ANOVA
with a
2
X
3
factorial design
Row factor =
4
levels
Column factor =
2
levels
2-way ANOVA
with a
4
X
2
factorial design
MEANS TABLE
Summary of data
6
Means Table:
2-way ANOVA:
ROW FACTOR
“Sex”
r = 3
2-way ANOVA:
COLUMN FACTOR
“Teaching method”
c = 2
7
Means Table:
2-way ANOVA:
ROW FACTOR
“Sex”
r = 3
Row Means
Marginal Means
for Rows
2-way ANOVA:
COLUMN FACTOR
“Teaching method”
c = 2
8
Means Table:
2-way ANOVA:
ROW FACTOR
“Sex”
r = 3
Row Means
Marginal Means
for Rows
Column Means
Marginal Means for
Coumns
2-way ANOVA:
COLUMN FACTOR
“Teaching method”
c = 2
9
Means Table:
2-way ANOVA:
ROW FACTOR
“Sex”
r = 3
Row Means
Marginal Means
for Rows
Column Means
Marginal Means for
Coumns
2-way ANOVA:
COLUMN FACTOR
“Teaching method”
c = 2
THE BASICS
Statistical significance of effects
Simultaneously
evaluate effect of
2 or
more
factors’ effects on a continuous
outcome
AND
investigate a potential 3
rd
effect: an
interaction
between the two factors.
Cross-classification
:
Participants only belong to 1
mutually
exclusive
‘cell’
Belong to
1 level of
row
factor
Belong to 1 level of
columns
factor
11
11
Typical
2-way
ANOVA
3
x
2
design
Row factor (“A”): 3 levels
Column factor (“B”): 2 levels
Do
r
ow
marginal
means differ?
Do population means differ across levels of row factor,
averaging across levels of column factor?
H
0
:
μ
1
•
=
μ
2
•
= … =
μ
r
•
H
1
: Not
H
0
12
12
Do
column
marginal
means differ?
Do population means differ across levels of column factor
,
averaging across levels of row factor
?
H
0
:
μ
•
1
=
μ
•
2
= … =
μ
•
c
H
1
: Not
H
0
13
13
Does
pattern of cell
means differ?
Are
differences
among population
means
across row
factor
similar
across
all levels of column factor
(
and vice
versa
)?
H
0
:
Differences among levels for 1
factor
do not vary
across levels of
other factor
H
1
:
Not
H
0
14
14
1.
Significant Interaction
and…
Always check for FIRST
Always interpret FIRST
Exercise EXTREME caution interpreting significance and effect of main effects
2.
No significant interaction, but…
Significant main effects for
both
rows
and
columns
Significant of
only
main effect for rows
,
but not for columns
Significant of
only
main effect for columns
,
but not for rows
3.
No significance
… main effects or interaction(s)
15
15
Treat
each cell
as a separate group
(e.g., M/Rep, M/Dem, F/Rep, F/Dem) and
run analysis as 1-Way ANOVA with R*C groups?
Results in same
SS
Between
as factorial design (
SS
R
+ SS
C
+ SS
RC
; when study is balanced)
Cannot see
patterns
in data, as all levels of all factors are
blended together
in each
group
Cannot as easily observe interaction effects
Limits identification of characteristics that uniquely differentiate participants
More
cumbersome
when many factors included
Less powerful
16
16
17
17
Subject-to-subject variability
contributes to
increased
MS
W
=> Less power
Adding factors that
explain subject-to-subject variability
in outcome
reduces
MS
W
& increases power
Variance within (and thus across) individual cells
is reduced
as cases become more
homogeneous
in terms of their characteristics
Factors that
do not
have this effect
may slightly decrease power
df
W
=
N – rc
decreases as # cells increases,
increasing
MS
W
& decreasing
F
-ratios
Alternatives:
Restriction (subjects from 1-level only –
reduced generalizability)
Repeated-measures (matched) designs
Similar to 1-Way ANOVA…
Independence
(observations)
Outcome is
normally
distributed in EACH population
Homogeneity of variance
in EACH population
18
18
OMNIBUS F-TESTS
Statistical significance of effects
20
20
1-Way ANOVA
2-Way ANOVA
4-SUM OF SQUARES
21
21
SS
Total
=
SS
(R)ows
+
SS
(C)olumns
+
SS
RC
+
SS
Within-Cell
SS
Total
=
SS
Between-Groups
+
SS
Within-Groups
For
balanced
designs (all cells are the same size)…
GROUPING Effect
k = # groups
COLUMN Main Effect
c = # columns
1-Way ANOVA
2-Way ANOVA
ROW Main Effect
r = # rows
INTERACTION Effect
r x c = # cells
SS
R
Computed with
row means:
all scores in a given
row
are averaged,
regardless of column
n
row
= # participants
per row
22
22
SS
C
Computed with
column means:
all scores in a given
column
are averaged
regardless of row
n
col
= # participants
per column
23
23
SS
RC
24
24
Variability among
cell means
AFTER REMOVING variability due to
individual
row
and
column
effects
n
cell
= # participants
per cell
SS
W
25
25
SS
within
each cell added together
SS
W
= SS
11
+ SS
12
+
…
+ SS
rc
F
o
r
e
a
c
h
c
e
l
l
,
a
l
l
s
c
o
r
e
s
w
i
t
h
i
n
t
h
a
t
c
e
l
l
a
r
e
s
u
b
t
r
a
c
t
e
d
f
r
o
m
c
e
l
l
m
e
a
n
,
s
q
u
a
r
e
d
,
a
n
d
s
u
m
m
e
d
26
26
1-Way ANOVA
2-Way ANOVA
DEGREES OF FREEDOM
27
27
1-way or independent groups (Ch 12)
n = # obs per group
k = # groups
n
T
= total # observations
2-way or factorial (Ch 14)
n = # obs per cell
r = # rows
c = # columns
n
T
= total # observations
28
28
1-Way ANOVA
2-Way ANOVA
VARIANCE ESTIMATES
29
29
30
30
1-Way ANOVA
2-Way ANOVA
OMNIBUS F-TESTS
31
31
32
EFFECT SIZE
How big is the effect?
Proportion of variation
in
outcome
accounted for
by a
particular factor or interaction term
Interpretation:
Range: 0 to 1
Small: .01 to .06
Medium: .06 to .14
Large: > .14
34
34
•
E
t
a
-
s
q
u
a
r
e
d
(
η
2
)
–
1
-
w
a
y
A
N
O
V
A
•
SS
Between
/ SS
Total
–
2
-
w
a
y
A
N
O
V
A
•
R
o
w
f
a
c
t
o
r
:
S
S
R
/
S
S
T
o
t
a
l
•
C
o
l
u
m
n
f
a
c
t
o
r
:
S
S
C
/
S
S
T
o
t
a
l
•
I
n
t
e
r
a
c
t
i
o
n
:
S
S
R
C
/
S
S
T
o
t
a
l
35
35
η
2
are biased parameter estimates
Should estimate omega squared (
ω
2
)
Substitute
SS
and
df
values
Same interpretation as
η
2
When
all
factors are
experimental
or when
many
factors are included in
analysis,
SS
due to a factor or interaction will be small relative to
SS
Total
Partial effect size estimates are often reported
Proportion of variation in outcome accounted for by a particular factor or interaction term,
excluding other main effects or interaction sources of variation
36
36
INTERACTIONS
Moderation of effects
Interaction between …
2 factors is called a “2-way interaction”
3 factors is called a “3-way interaction”
Quite rare, be skeptical BUT I have found/published them ;)
Significance
indicates that the
effect of 1 factor is not same at
all levels of another factor
i.e.
the effect of 1 factor
depends
on the level of the other
Effect of variables combined is different than would be predicted by
either variable alone
Most interesting results, but more difficult to explain or interpret
than main effects
38
38
Ordinal
Direction or order of effects is similar for
different subgroups
Disordinal
Direction or order of effects is reversed for
different subgroups
39
39
Significance of interaction always
evaluated 1
st
If significant, interpret interaction, not main effects
If non-significant, interpret main effects
Once we know effects of 1 factor are
tempered by
or contingent on
levels of another factor (as in an
interaction), interpretation of either factor (main
effect) alone is problematic
Best interpreted through
visualization
Cell means plot
Interactions exist if lines cross or will cross (non-
parallel)
Design graph to best illustrate
Outcome on y-axis
Select one factor for x-axis
Other factor(s) represented by separate lines, colors,
panels, ect.
Selection guides interpretation, can dictate whether plot is
ordinal/disordinal
It is Recommend to only interpreting significant
main
effects
(Keppel & Wickens, 2004) IF…
1.
there is
NO significant interaction
2.
there is a significant interaction WITH EXTREME CAUTION, IF…
interaction
effect size is small
relative to that of main effects and
there is an
ordinal
pattern to the means
However, must
report
all main and interaction effects regardless of statistical
significance
41
41
Results may be distorted if additional factors are not included in
analysis so that interactions are not tested
E.g., If experimental effects of a drug had opposite effects in men and
women, the variable representing drug effects may appear to be ineffective
(non-significant main effect) without including the variable for sex
differences
If interaction terms are non-significant, increased
confidence
that
effect of key factor (e.g., drug treatment) is
generalizable
to all
levels of other factors (e.g., sex)
42
42
FOLLOW-UP TESTS
Prob Interactions, Post Hoc, a prior’
Factorial ANOVA produces omnibus results
It does NOT indication of specific level (group) differences within or across factor(s)
Multiple comparisons
elucidate differences within significant main effects or interactions
Pattern of results dictates approach (eg.
Significant main effects, but no interaction)
Each of the 3
F
-tests in a 2-Way ANOVA represents a ‘planned comparison’
No adjustment to
α
EW
necessary
However, within each
main-effect
and
interaction
a separate family of possible multiple
comparisons may be conducted (
α
EW
must be controlled within each ‘family’)
44
44
Simple main effects
generally tested within
each level of stratifying
factor
2-levels
Simple, pairwise
comparisons: Tukey HSD or
t
-tests with Bonferroni
correction
> 2 levels
Modified 1-way ANOVA
followed by simple or
complex comparisons
45
45
Modified 1-Way ANOVA tests of simple main effects often done ‘by hand’
Obtain
MS
Between
from standard 1-Way ANOVA
Comparing means across 1 level of 1 factor within 1 level of another factor
Obtain
MS
Within
from original 2-Way ANOVA
Ensure homogeneity of variance assumption is reasonably satisfied
46
46
An alternative is to perform
‘interaction contrasts’
, rather than
immediately testing simple effects
With a 2x2 design, only tests of
simple main effects
are possible
With a 2x3 design,
3 separate 2x2 ANOVAs
may be conducted
Interaction magnitude (and significance) can differ from one subset to
another
Simple effects can be used following significant interaction subsets
MS
B
for overall interaction = ‘average’ of
MS
Interactions
for separate interaction
subsets
47
47
The you may evaluate possible significant main effect…
Factors with
2 levels
No multiple comparisons required
Factors with
> 2 levels
2-way ANOVA is reduced to two 1-Way ANOVAs
Simple (pairwise) or complex (linear) contrasts are computed within
individual significant main-effect(s) (ignoring others)
48
48
49
49
N
o
f
u
r
t
h
e
r
t
e
s
t
s
i
f
F
-
t
e
s
t
o
f
m
a
i
n
-
e
f
f
e
c
t
i
n
d
i
c
a
t
e
s
d
i
f
f
e
r
e
n
c
e
S
i
m
p
l
e
o
r
c
o
m
p
l
e
x
c
o
m
p
a
r
i
s
o
n
s
a
m
o
n
g
m
a
r
g
i
n
a
l
m
e
a
n
s
(
l
e
v
e
l
s
)
S
i
g
n
i
f
i
c
a
n
t
m
a
i
n
-
e
f
f
e
c
t
s
RESULTS
How to present findings, APA style
Marginal
M
s for main effects, cell
M
s for interactions and their
SD
s (or
SE
s) and
CI
s
No need to report
MS
W
For each significant effect
F
(
df
Effect
,
df
Within
) =
F
stat
,
p
-value, effect size (
η
2
or
ω
2
)
Results of post-hoc or planned comparisons
Figures are *extremely* helpful!
51
51
With a non-significant interaction
# of follow-up tests on main-effects needs to be kept low so as to not inflate
α
EW
, where
each main-effect can contain a family of tests
With a significant interaction
# of tests of simple effects or interaction contrasts should not exceed
df
Interaction
For 2x2 ANOVA: #
≤
(
r
-1)*(
c
-1)
In
Conformity
data example = 2*1 = 2 tests
Some forgo tests of simple main effects and compute all possible pairwise comparisons at cell level
Results in many, many tests
Following a significant interaction and significant simple main effects
Not necessary to conduct all possible pairwise comparisons
Planned comparisons should be derived from theory or previous research and flow from research
questions
Significant unplanned interactions that do not conform to theory should be swallowed with a
HIGH DEGREE OF SKEPTICISM
52
52
UNBALANCED
Sub-sample sizes are not equal
Equal
n
s in each cell
= Orthogonal design
Factors are independent/uncorrelated so that significance of any effect is independent of
significance of other effects (including interaction)
Most research consists of unbalanced data
As
n
s across cells become more unequal, factors become more dependent/correlated
Unbalanced:
SS
Between
≠ SS
R
+ SS
C
+ SS
RC
More difficult to determine independent effects of each factor
Previous equations and R commands will not work correctly for unbalanced
designs
54
54
Balanced
Sum of areas where factors
overlap with DV =
SS
B
Remaining portion of DV =
SS
W
Unbalanced
Sum of areas where factors
overlap
with DV
≠
SS
B
Some areas counted twice
Remaining portion of DV =
SS
W
55
55
56
56
Reason
for unequal
n
s should be
random
, not related to factor(s) themselves (more
difficult with non-experimental studies)
If not so, validity of results is questionable when regular ANOVA procedures are
employed
Adjustments made to ANOVA to correct for unequal
n
s
1.
Analysis of weighted means:
Non-recommended
, but common, approach where
imbalance is slight and imbalance is random
1.
Harmonic mean
of cell
n
s is used in computation of various
MS
2.
Total
N
is adjusted = Harmonic mean of all cell sizes x # cells
3.
MS
Within
=
Weighted
average of cell variances
4.
Each row and column mean computed = Simple (non-weighted) average of cell means in a
given row or column
2.
Alternate
SS
calculations
to handle overlapping variation accounted for in outcome
(next 2 slides)
3.
Regression analysis
(Take EDUC/PSY 7610! And think about PSY 7650 “MLM”)
Several methods for partitioning or allocating variation between
outcome and factor(s) to account for unbalanced designs
Commonly used
Type I SS: Sequential or Hierarchical
Type II SS: Partially Sequential
Type III SS: Simultaneous or Regression
Specialized and less commonly used
Type IV SS: Don’t use
Type V SS: Used for fractional factorial designs
Type VI SS: Effective hypothesis tests though sigma-restricted coding
57
57
Type II or III SS recommended in most cases
Results should be fairly consistent
Type III is most commonly used
Nothing wrong with Type II
Considered by some to be more powerful, especially when testing main effects
Uncertainty of results when
n
are vastly unbalanced
Not an issue when design is balanced
Type I-III yield same results
Even when unbalanced, interaction result same
58
58
Dr. Petrov plans an experiment to study the impact of teaching methods and sex differences on learning outcomes using factorial ANOVA. The design includes row and column factors, with different levels for each, to analyze the data effectively. The means tables provide a summary of the experiment's data, showing results for male, female, and other groups across different teaching methods. This experiment encompasses various statistical concepts, such as 1-way ANOVA, 2-way ANOVA with a 2X3 factorial design, and more. Understanding these concepts is essential for conducting meaningful research in education.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Chapter 14 FACTORIAL ANOVA Adapted from Jamison Fargo, PhD EDUC 6600 Slides
People can be divided into two classes: Those who go ahead and do something, and those who sit still and inquire, 'Why wasn't it done the other way? Oliver Wendell Holmes American Physician, Writer, Humorist, HarvardProfessor 1809-1894
Dr. Petrov is interested in conducting an experiment where: r = # rows c = # columns r x c 1-way ANOVA: GROUPING FACTOR Teaching method k = 2 30 high school students are randomly assigned to a new computer simulation tool for learning geometry and 2-way ANOVA: COLUMN FACTOR Teaching method c = 2 30 other students are randomly assigned to the standard lecture and paper/pencil problem solving format. However, Dr. Petrov is also interested in the effect of sex differences on learning outcomes. 2-way ANOVA: ROW FACTOR Sex r = 3 3
r = # rows c = # columns ANALYSIS OF VARIANCE r x c Independent ANOVA types 1-Way ANOVA = 1 factor (previously covered) The # levels of each factor determines ANOVA design Row factor = 2 levels Column factor = 3 levels 2-way ANOVA with a 2X3 factorial design 2-Way ANOVA = 2 factors (focus of lecture) 3-Way ANOVA = 3 factors Row factor = 4 levels Column factor = 2 levels 2-way ANOVA with a 4X2 factorial design 4-Way ANOVA = 4 factors 4
MEANS TABLE Summary of data
Means Table: r = 3 rows c = 2 columns 3 x 2 lecture and paper/pencil problem computer simulation tool Margin 2-way ANOVA: ROW FACTOR Sex r = 3 Male Female Other Grand Mean Mean of ENTIRE sample s Outcome (DV) MG Margin 2-way ANOVA: COLUMN FACTOR Teaching method c = 2 6
Means Table: r = # rows c = # columns 3 x 2 lecture and paper/pencil problem computer simulation tool Margin 2-way ANOVA: ROW FACTOR Sex r = 3 M1 Male M2 Female Row Means Marginal Means for Rows M3 Other Grand Mean Mean of ENTIRE sample s Outcome (DV) MG Margin 2-way ANOVA: COLUMN FACTOR Teaching method c = 2 7
Means Table: r = # rows c = # columns 3 x 2 lecture and paper/pencil problem computer simulation tool Margin 2-way ANOVA: ROW FACTOR Sex r = 3 M1 Male M2 Female Row Means Marginal Means for Rows M3 Other Grand Mean Mean of ENTIRE sample s Outcome (DV) 1 2 MG Margin Column Means Marginal Means for Coumns 2-way ANOVA: COLUMN FACTOR Teaching method c = 2 8
Means Table: r = # rows c = # columns 3 x 2 lecture and paper/pencil problem computer simulation tool Margin 2-way ANOVA: ROW FACTOR Sex r = 3 11 12 M1 Male 21 22 M2 Female Row Means Marginal Means for Rows M31 32 M3 Other Grand Mean Mean of ENTIRE sample s Outcome (DV) 1 2 MG Margin Column Means Marginal Means for Coumns Cell Means Interior Means for Each Cell 2-way ANOVA: COLUMN FACTOR Teaching method c = 2 9
THE BASICS Statistical significance of effects
FACTORIAL 2-WAY ANOVA r = # rows c = # columns r x c Simultaneously evaluate effect of 2 or morefactors effects on a continuous outcome AND investigate a potential 3rd effect: an interaction between the two factors. Typical 2-way ANOVA 3x2 design Row factor ( A ): 3 levels Column factor ( B ): 2 levels B B1 11 21 31 B2 12 22 32 Cross-classification: A1 A2 A3 Participants only belong to 1 mutually exclusive cell A Belong to 1 level of row factor Belong to 1 level of columns factor 11
r = # rows c = # columns r x c TEST OF ROW MAIN EFFECT Do row marginal means differ? Do population means differ across levels of row factor, averaging across levels of column factor? B B1 M11 M21 M31 B2 M12 M22 M32 MB2 Marginals MA1 MA2 MA3 H0: 1 = 2 = = r A1 A2 A3 H1: Not H0 A Marginals MB1 12
r = # rows c = # columns r x c TEST OF COLUMN MAIN EFFECT Do column marginal means differ? Do population means differ across levels of column factor, averaging across levels of row factor? B B1 M11 M21 M31 B2 M12 M22 M32 MB2 Marginals MA1 MA2 MA3 H0: 1 = 2= = c A1 A2 A3 A H1: Not H0 Marginals MB1 13
TEST OF INTERACTION EFFECT B B1 M11 M21 M31 B2 M12 M22 M32 MB2 Marginals MA1 MA2 MA3 Does pattern of cell means differ? A1 A2 A3 A Are differences among population means across row factor similar across all levels of column factor (and vice versa)? Marginals MB1 B B1 M11 M21 M31 B2 M12 M22 M32 MB2 Marginals MA1 MA2 MA3 H0:Differences among levels for 1 factor do not vary across levels of other factor A1 A2 A3 A H1: Not H0 Marginals MB1 14
POSSIBLE OUTCOMES: ORDER TO INVESTIGATE 1. Significant Interaction and Always check for FIRST Always interpret FIRST Exercise EXTREME caution interpreting significance and effect of main effects 2. No significant interaction, but Significant main effects for bothrows and columns Significant of onlymain effect for rows, but not for columns Significant of onlymain effect for columns, but not for rows 3. No significance main effects or interaction(s) 15
IGNORING FACTORIAL DESIGN Treat each cell as a separate group (e.g., M/Rep, M/Dem, F/Rep, F/Dem) and run analysis as 1-Way ANOVA with R*C groups? Results in same SSBetween as factorial design (SSR + SSC + SSRC ; when study is balanced) Cannot see patterns in data, as all levels of all factors are blended together in each group Cannot as easily observe interaction effects Limits identification of characteristics that uniquely differentiate participants More cumbersome when many factors included Less powerful 16
REDUCED ERROR Adding factors that explain subject-to-subject variability in outcome reduces MSW& increases power Subject-to-subject variability contributes to increased MSW => Less power Variance within (and thus across) individual cells is reduced as cases become more homogeneous in terms of their characteristics Factors that do not have this effect may slightly decrease power dfW=N rc Alternatives: Restriction (subjects from 1-level only reduced generalizability) Repeated-measures (matched) designs decreases as # cells increases, increasing MSW& decreasing F-ratios 17
ASSUMPTIONS Can NOT test or fix this You can only can PLAN well Look at the PLOT {histogram, QQ plot, boxplot} of the OUTCOME, for each cell s subsample Similar to 1-Way ANOVA Independence (observations) Outcome is normally distributed in EACH population Homogeneity of variance in EACH population Run Levene s Test of HOV for all cell s subsample 18
OMNIBUS F-TESTS Statistical significance of effects
ANOVA SUMMARY TABLE: 4-SUM OF SQUARES 1-Way ANOVA 2-Way ANOVA SS df MS F p Source SS df MS F p Source Row Group Column Within R x C Total Within Total 20
PARTITIONING TOTAL VARIANCE (SSTOTAL) For balanceddesigns (all cells are the same size) GROUPING Effect k = # groups SSTotal = SSBetween-Groups + SSWithin-Groups 1-Way ANOVA 2-Way ANOVA SSTotal = SS(R)ows + SS(C)olumns + SSRC + SSWithin-Cell INTERACTION Effect r x c = # cells COLUMN Main Effect c = # columns ROW Main Effect r = # rows 21
SSR Computed with row means: all scores in a given row are averaged, regardless of column Grand Mean Mean of ENTIRE sample s Outcome (DV) Row Means Marginal Means for Rows nrow= # participants per row = + + ... ( + 2 2 2 [( ) ( ) ) ] SS n X X X X X X 2 Alternative Forms, Give the same value 1 2 R row row GM row GM row r GM 2 2 2 2 n n n n + + + ... X X X X 1 2 row row row r = = = = = 1 1 1 1 i i i i SS R n N row 22
SSC Computed with column means: all scores in a given column are averaged regardless of row Column Means Marginal Means for Columns Grand Mean Mean of ENTIRE sample s Outcome (DV) ncol= # participants per column = + + ... ( + 2 2 2 [( ) ( ) ) ] SS column n X X X X X X 2 Alternative Forms, Give the same value 1 2 C col GM col GM col r GM 2 2 2 2 n n n n + + + ... X X X X 1 2 col col col r = = = = = 1 1 1 1 i i i i SS C n N col 23
SSRC Variability among cell means AFTER REMOVING variability due to individual row and column effects Grand Mean Mean of ENTIRE sample s Outcome (DV) Cell Means Interior Means for Each Cell ncell= # participants per cell 2 Alternative Forms, Give the same value = + + ... ( + 2 2 2 [( ) ( ) ) ] SS n X X X X X X SS SS 11 12 RC cell cell GM cell GM cell rc GM R C 2 2 2 2 n n n n + + + ... X X X X 11 12 cell cell cell rc = = = = = 1 1 1 1 i i i i SS SS SS RC R C n N cell 24
SSW SS within each cell added together SSW = SS11 + SS12 + + SSrc For each cell, all scores within that cell are subtracted from cell mean, squared, and summed 3 Alternative Forms, Give the n rc ( ) rc 2 = SS X X W irc rc 1,1 = = 1 j i same value 2 2 2 n n n + + + ... X X X 11 12 cell cell cell rc n = = = = 1 1 1 i i i 2 i SS X W n = 1 i cell = SS SS SS SS SS W T R C RC 25
ANOVA SUMMARY TABLE: DEGREES OF FREEDOM 1-Way ANOVA 2-Way ANOVA SS df MS F p Source SS df MS F p Source Row Group Column Within R x C Total Within Total 26
2-way or factorial (Ch 14) n = # obs per cell r = # rows c = # columns nT = total # observations DEGREES OF FREEDOM: df 1-way or independent groups (Ch 12) n = # obs per group k = # groups nT = total # observations 27
ANOVA SUMMARY TABLE: VARIANCE ESTIMATES 1-Way ANOVA 2-Way ANOVA SS df MS F p Source SS df MS F p Source Row Group Column Within R x C Total Within Total 28
4X VARIANCE ESTIMATES: MS ????????=???????? ???????? Variance Estimate MSR MSC MSRxC MSW Effect of Sensitive to effect of Row s Factor Name Row-wise Factor Column s Factor Name Column-wise Factor Interaction between the Row-wise & Column-wise Factors Interaction Within-Cell, Residual, or Error None of the Factors (noise) 29 Note MSWithin is also called MSW , MSError , MSResidual , or MSE
ANOVA SUMMARY TABLE: OMNIBUS F-TESTS 1-Way ANOVA 2-Way ANOVA SS df MS F p Source SS df MS F p Source Row Group Column Within R x C Total Within Total 30
3X F-STATISTICS (F-ratio or F-test) P-VALUES ???????=???????? ????? ?? The Denominator (bottom) of each F-ratio and df s are the same for all 3 F-tests Degrees of Freedom numerator F-stats Effect of denominator Row s Factor Name Column s Factor Name FR FC FRxC dfR = r - 1 dfWithin = nT (r x c) dfC = c - 1 dfWithin = nT (r x c) dfR x C = (r 1) x (c - 1) dfWithin = nT (r x c) Interaction NOTE Since each F-test CAN have a different numerator df, the critical values (FCV) may all be different 31
EFFECT SIZE How big is the effect?
Proportion of variation in outcome accounted for by a particular factor or interaction term EFFECT SIZE ???????? ?2=???????? ??????? Interpretation: Range: 0 to 1 Small: .01 to .06 Medium: .06 to .14 Large: > .14 Eta-squared ( 2) 1-way ANOVA SSBetween / SSTotal 2-way ANOVA Row factor: SSR / SSTotal Column factor: SSC / SSTotal Interaction: SSRC / SSTotal 34
EFFECT SIZE should but people dont 2 are biased parameter estimates Should estimate omega squared ( 2) Substitute SSand df values * SS df MS Effect SS Effect MS + Within = 2 Total Within Same interpretation as 2 35
EFFECT SIZE PARTIAL When all factors are experimental or when many factors are included in analysis, SS due to a factor or interaction will be small relative to SSTotal Partial effect size estimates are often reported Proportion of variation in outcome accounted for by a particular factor or interaction term, excluding other main effects or interaction sources of variation SS Effect SS + 2 Partial = SS Effect Within ( )/ ] + df MS MS N MS Effect MS Effect MS Within N = 2 Partial [ ( )/ df Effect Effect Within Within 36
INTERACTIONS Moderation of effects
INTERACTIONS: aka Moderation Interaction between 2 factors is called a 2-way interaction 3 factors is called a 3-way interaction Quite rare, be skeptical BUT I have found/published them ;) Significance indicates that the effect of 1 factor is not same at all levels of another factor i.e. the effect of 1 factor depends on the level of the other Effect of variables combined is different than would be predicted by either variable alone Most interesting results, but more difficult to explain or interpret than main effects 38
INTERACTIONS Ordinal Disordinal Direction or order of effects is similar for different subgroups Direction or order of effects is reversed for different subgroups 6 100 Male Female M Phonemic Errors Mean Test Score 90 5 80 4 70 3 High SES Low SES 60 2 50 1 40 0 Control Treatment LTLE RTLE Study Group Hemisphere 39
INTERACTIONS Significance of interaction always evaluated 1st If significant, interpret interaction, not main effects If non-significant, interpret main effects Once we know effects of 1 factor are tempered by or contingent on levels of another factor (as in an interaction), interpretation of either factor (main effect) alone is problematic Best interpreted through visualization Cell means plot Interactions exist if lines cross or will cross (non- parallel) Design graph to best illustrate Outcome on y-axis Select one factor for x-axis Other factor(s) represented by separate lines, colors, panels, ect. Selection guides interpretation, can dictate whether plot is ordinal/disordinal
INTERACTIONS It is Recommend to only interpreting significant main effects (Keppel & Wickens, 2004) IF 1. there is NO significant interaction 2. there is a significant interaction WITH EXTREME CAUTION, IF interaction effect size is small relative to that of main effects and there is an ordinal pattern to the means However, must report all main and interaction effects regardless of statistical significance 41
NEED FOR TESTING INTERACTIONS Results may be distorted if additional factors are not included in analysis so that interactions are not tested E.g., If experimental effects of a drug had opposite effects in men and women, the variable representing drug effects may appear to be ineffective (non-significant main effect) without including the variable for sex differences If interaction terms are non-significant, increased confidence that effect of key factor (e.g., drug treatment) is generalizable to all levels of other factors (e.g., sex) 42
FOLLOW-UP TESTS Prob Interactions, Post Hoc, a prior
MULTIPLE COMPARISONS FOLLOW IT UP Factorial ANOVA produces omnibus results It does NOT indication of specific level (group) differences within or across factor(s) Multiple comparisons elucidate differences within significant main effects or interactions Pattern of results dictates approach (eg. Significant main effects, but no interaction) Each of the 3 F-tests in a 2-Way ANOVA represents a planned comparison No adjustment to EWnecessary However, within each main-effect and interaction a separate family of possible multiple comparisons may be conducted ( EWmust be controlled within each family ) 44
SIGNIFICANT INTERACTION Simple main effects generally tested within each level of stratifying factor 2-levels Simple, pairwise comparisons: Tukey HSD or t-tests with Bonferroni correction > 2 levels Modified 1-way ANOVA followed by simple or complex comparisons B B1 M11 M21 M31 B2 M12 M22 M32 MB2 Marginals MA1 MA2 MA3 A1 A2 A3 A Marginals MB1 B B1 M11 M21 M31 B2 M12 M22 M32 MB2 Marginals MA1 MA2 MA3 A1 A2 A3 A Marginals MB1 45
SIGNIFICANT INTERACTION Modified 1-Way ANOVA tests of simple main effects often done by hand Obtain MSBetween from standard 1-Way ANOVA Comparing means across 1 level of 1 factor within 1 level of another factor Obtain MSWithin from original 2-Way ANOVA Ensure homogeneity of variance assumption is reasonably satisfied MS MS (1-way ANOVA) Between = Simple Effect F (2-way ANOVA) ) from 1-way ANOVA Within df Within ( , Critical F df Between 46
INTERACTION CONTRASTS An alternative is to perform interaction contrasts , rather than immediately testing simple effects With a 2x2 design, only tests of simple main effects are possible With a 2x3 design, 3 separate 2x2 ANOVAs may be conducted Interaction magnitude (and significance) can differ from one subset to another Simple effects can be used following significant interaction subsets MSB for overall interaction = average of MSInteractionsfor separate interaction subsets 47
NON-SIGNIFICANT INTERACTION The you may evaluate possible significant main effect Factors with 2 levels No multiple comparisons required Factors with > 2 levels 2-way ANOVA is reduced to two 1-Way ANOVAs Simple (pairwise) or complex (linear) contrasts are computed within individual significant main-effect(s) (ignoring others) 48
NON-SIGNIFICANT INTERACTION Significant main-effects B B1 M11 M21 M31 B2 M12 M22 M32 MB2 Marginals MA1 MA2 MA3 A1 A2 A3 Simple or complex comparisons among marginal means (levels) A Marginals MB1 No further tests if F-test of main-effect indicates difference 49
RESULTS How to present findings, APA style
