The Impact of Board Settings and Instructor Interventions on Student Participation and Engagement
T
h
e
E
f
f
e
c
t
o
f
B
o
a
r
d
S
e
t
t
i
n
g
s
a
n
d
I
n
s
t
r
u
c
t
o
r
I
n
t
e
r
v
e
n
t
i
o
n
s
o
n
S
t
u
d
e
n
t
P
a
r
t
i
c
i
p
a
t
i
o
n
a
n
d
E
n
g
a
g
e
m
e
n
t
Dr. Linda A
&
Dr. Andrew Feldstein
2019 October
Yellowdig:
Shifting the
Paradigm
Discussion and the Learning Process
“By examining participation we see our
relationship to “content”…as part of a
shared practice and cultural belonging, not
as a process of individual “internalization”.”
(Jenkins &
Ito
, 2015, P. 4)
Yellowdig Board
Settings
*
Maximum Weekly Points
*Comment/Pin Control
*Points Settings
*Allow Upvote Points
*Instructor Badges
I
n
f
l
u
e
n
c
e
o
f
P
o
i
n
t
s
U
n
l
i
k
e
g
r
a
d
e
s
,
p
o
i
n
t
s
y
s
t
e
m
s
a
r
e
b
u
i
l
t
o
n
a
g
r
o
w
t
h
m
o
d
e
l
f
o
r
t
h
e
u
s
e
r
,
n
o
t
a
d
e
f
i
c
i
t
,
w
h
i
c
h
i
s
o
n
e
r
e
a
s
o
n
t
h
a
t
e
d
u
c
a
t
o
r
s
w
i
t
h
e
x
p
e
r
i
e
n
c
e
i
n
g
a
m
e
d
e
s
i
g
n
h
a
v
e
g
r
a
v
i
t
a
t
e
d
t
o
w
a
r
d
e
x
p
e
r
i
e
n
c
e
p
o
i
n
t
s
i
n
l
i
e
u
o
f
g
r
a
d
e
s
.
(Holman, Aguilar, and Fishman 2013;
Sheldon 2012)
D
e
s
i
g
n
F
r
a
m
e
w
o
r
k
Measurement
Model
Student Actions
(5 DVs)
=
Board
Structural Setting
(4 IV main effects, 1
interaction effect)
+
Instructor
Interventions
(3 IV main effects, 1
interaction effect)
Board Data
•
383 Boards (Aug. ‘17- Aug. ‘19)
•
80 Instructors
•
8000+ Students
•
Aggregated Board Data
Student
Actions
Observed
Average Based on Board Settings
Instructor
Decisions On
Board
Settings
•
Comment/Pin Control
•
Upvotes
•
Weekly Max Points
•
Comment on Pin Value
Instructor
Decisions on
Board
Settings
Comment/Pin Control
•
40- Encouraged comments (>1:2)
•
228- 1:2 Default
•
115– Discourage Comments (<1:2)
Instructor
Decisions on
Board
Settings
Upvote Value
•
341- Allows Points for Upvotes
•
42- No Points for Upvotes
Instructor
Decisions on
Board
Settings
Weekly Max Points
•
142- Enabled
•
241- Not Enabled
Results:
Overall
Model
Note
. Significance Level = .05. *** p < .001. ** p < .01. * p < .05.
Multivariate Tests on Discussion Structure Setting
and Instructor Interventions over Student Behaviors
Results: The
Interaction
Effect
Note
. Significance Level = .05. *** p < .001. ** p < .01. * p < .05.
Multivariate Tests on Discussion Structure Setting
and Instructor Interventions over Student Behaviors
Results: The
Interaction
Effect –
Conversation
Ratio (
F
(13,
347) = 1.88,
p
<
.05,
η
2
= .07
)
√
Weekly Max
√
Upvote Value
√
Encouraging Comments
Results: The
Interaction
Effect –
Posts per
Student (
F
(13,
347) = 2.24,
p
<
.01,
η
2
= .08
)
√
Weekly Max
X
Upvote Value
√
Suppressing Comments
Results: The
Interaction
Effect –
Reads per
Post (
F
(13, 347)
= 2.49,
p
< .01,
η
2
= .09
)
X
Weekly Max
X
Upvote Value
√
Encouraging Comments
Discussion:
The
Interaction
Effect of
Board
Settings
Results: The
Main Effect
of Upvote –
Posts
Exceeding
Word Min
(
F
(1, 347) = 4.16,
p
< .05,
η
2
= .01
)
Instructor:
Decisions on
Board
Participation
•
Contributing Posts
•
Contributing Comments
•
Awarding Badges
Instructor
Decisions on
Participation
Instructor Posts
•
171- No Posts
•
87- ~A Pin per Week
•
125- Multiple Posts
Instructor
Decisions on
Participation
Instructor Comments
•
191- No Comments
•
86- ~ Comment per Week
•
106- Multiple Comments
Instructor
Decisions on
Participation
Instructor Badges
•
291- No Badges Awarded
•
92- Awarded Badges
Results: The
Interaction
Effect –
Comments
Exceeding
Word Min
√
One Instructor Comment
√
One Instructor Post
X
Instructor Badge
Results:
Instructor
Posts
Results:
Instructor
Comments
Results:
Instructor
Badges
Discussion:
Instructor
Participation
•
Board settings suppress the
influence of instructor
participation
•
Participate judiciously
•
Positive reinforcement improves
student interactions
More
Questions?
•
Deng, L., & Tavares, N. J. (2013). From Moodle to Facebook: Exploring students'
motivation and experiences in online communities.
Computers & Education
,
68
, 167-176.
•
Giannetto, D., Chao, J., & Fontana, A. (2013, July). Gamification in a social learning
environment. In
Proceedings of the Informing Science and Information Technology
Education Conference
(pp. 195-207). Informing Science Institute.
•
Hew, K. F., Cheung, W. S., & Ng, C. S. L. (2010). Student contribution in asynchronous
online discussion: A review of the research and empirical exploration.
Instructional
science
,
38
(6), 571-606.
•
Hewitt, J. (2005). Toward an understanding of how threads die in asynchronous computer
conferences.
The journal of the learning sciences
,
14
(4), 567-589.
•
Holman, C., Aguilar, S., & Fishman, B. (2013, April). GradeCraft: What can we learn from a
game-inspired learning management system?. In
Proceedings of the third international
conference on learning analytics and knowledge
(pp. 260-264). ACM.
•
Jenkins, H., & Ito, M. (2015).
Participatory culture in a networked era: A conversation on
youth, learning, commerce, and politics
. John Wiley & Sons.
•
Sheldon, P. (2012). Profiling the non-users: Examination of life-position indicators,
sensation seeking, shyness, and loneliness among users and non-users of social network
sites.
Computers in Human Behavior
,
28
(5), 1960-1965.
•
Vonderwell, S., & Zachariah, S. (2005). Factors that influence participation in online
learning.
Journal of Research on Technology in education
,
38
(2), 213-230.
This study delves into the effects of board settings and instructor interventions on student engagement and participation. Insights are drawn from Yellowdig's paradigm shift in discussions, the influence of point systems, a design framework for fostering participation, and student actions measured against board structural settings. The vast aggregated data from 383 boards, 80 instructors, and over 8000 students provide valuable insights into enhancing student actions through effective instructional strategies and board configurations.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Dr. Linda A The Effect of Board Settings and The Effect of Board Settings and Instructor Interventions on Student Instructor Interventions on Student Participation and Engagement Participation and Engagement & Dr. Andrew Feldstein 2019 October
Yellowdig: Shifting the Paradigm Discussion and the Learning Process By examining participation we see our relationship to content as part of a shared practice and cultural belonging, not as a process of individual internalization . (Jenkins & Ito, 2015, P. 4)
Yellowdig Board Settings *Maximum Weekly Points *Comment/Pin Control *Points Settings *Allow Upvote Points *Instructor Badges
Ex.1 Ex. 2 Ex. 3 Influence of Influence of Points Points Original Pin/Post (10) Sending a Comment (3) Upvotes (1) Instructor Badge (10) Receiving a comment (5) 10 10 10 Unlike grades, point systems are built on a growth model for the user, not a deficit, which is one reason that educators with experience in game design have gravitated toward experience points in lieu of grades. 6 0 6 2 6 5 0 0 10 0 10 28 20 51 16 (Holman, Aguilar, and Fishman 2013; Sheldon 2012)
Design Framework Design Framework Limited discussion participation Guideline dilemma Learners perspective Instructors perspective Technical aspects Use of grades Use of number of posting guidelines Instructor-facilitation Best Practices? (Deng & Tavares, 2013; Hew, Cheung, & Ng, 2010; Hewitt, 2005; Vonderwell & Zachariah, 2005) (Hew et al., 2010)
Student Actions (5 DVs) = Board Structural Setting (4 IV main effects, 1 interaction effect) + Instructor Interventions (3 IV main effects, 1 interaction effect) Measurement Model
383 Boards (Aug. 17- Aug. 19) 80 Instructors 8000+ Students Aggregated Board Data Board Data
Average Based on Board Settings 95% Confidence Interval Lower Bound Student Actions Mean Student Actions Observed Upper Bound Conversation Ratio* 2.76 2.45 3.07 Posts Per Student 6.03 5.02 7.04 Length of Posts Exceed Min 138.52 116.90 160.15 Length of Comments Exceed Min 22.79 18.09 27.48 Average Reads Per Post* 7.83 6.95 8.72
Comment/Pin Control Upvotes Weekly Max Points Comment on Pin Value Instructor Decisions On Board Settings
Comment/Pin Control 40- Encouraged comments (>1:2) 228- 1:2 Default 115 Discourage Comments (<1:2) Instructor Decisions on Board Settings
Upvote Value Instructor Decisions on Board Settings 341- Allows Points for Upvotes 42- No Points for Upvotes
Weekly Max Points 142- Enabled 241- Not Enabled Instructor Decisions on Board Settings
Multivariate Tests on Discussion Structure Setting and Instructor Interventions over Student Behaviors 2 .05 IV Effect F df1 df2 686 Sig. .000 Results: Overall Model Comment/Pin Control .91 3.25*** 10 Upvote Value .96 2.56* 5 343 .027 .04 Weekly Max Points .94 4.08** 5 343 .001 .06 Interaction Effect of Structure Setting .71 1.89*** 65 1625 .000 .07 Note. Significance Level = .05. *** p < .001. ** p < .01. * p < .05.
Multivariate Tests on Discussion Structure Setting and Instructor Interventions over Student Behaviors 2 .05 IV Effect F df1 df2 686 Sig. .000 Results: The Interaction Effect Comment/Pin Control .91 3.25*** 10 Upvote Value .96 2.56* 5 343 .027 .04 Weekly Max Points .94 4.08** 5 343 .001 .06 Interaction Effect of Structure Setting .71 1.89*** 65 1625 .000 .07 Note. Significance Level = .05. *** p < .001. ** p < .01. * p < .05.
Comment/Pin Control * Upvote Value * Weekly Max Points Enabled Results: The Interaction Effect Conversation Ratio (F(13, 347) = 1.88, p < .05, 2= .07) Weekly Max Upvote Value Encouraging Comments
Comment/Pin Control * Upvote Value * Weekly Max Points Enabled Results: The Interaction Effect Posts per Student (F(13, 347) = 2.24, p < .01, 2= .08) Weekly Max X Upvote Value Suppressing Comments
Comment/Pin Control * Upvote Value * Weekly Max Points Enabled Results: The Interaction Effect Reads per Post (F(13, 347) = 2.49, p < .01, 2= .09) X Weekly Max X Upvote Value Encouraging Comments
Discussion: The Interaction Effect of Board Settings Weekly Max Points Default Comment Value Effect Upvote Value Suppressing Comments Encouraging Comments Conversation Ratio X X X X X Posts/Student X X X X Reads/Post
Results: The Main Effect of Upvote Posts Exceeding Word Min (F(1, 347) = 4.16, p < .05, 2= .01)
Contributing Posts Contributing Comments Awarding Badges Instructor: Decisions on Board Participation
Instructor Posts 171- No Posts 87- ~A Pin per Week 125- Multiple Posts Instructor Decisions on Participation
Instructor Comments 191- No Comments 86- ~ Comment per Week 106- Multiple Comments Instructor Decisions on Participation
Instructor Badges Instructor Decisions on Participation 291- No Badges Awarded 92- Awarded Badges
Results: The Interaction Effect Comments Exceeding Word Min One Instructor Comment One Instructor Post X Instructor Badge
4 10 Converssation Ratio 8 3 Read per Post Results: Instructor Posts 6 2 4 1 2 0 0 No Post One Post More Post No Post One Post More Posts
8 6 Posts / Student Results: Instructor Comments 4 2 0 No Comment One or Fewer Comment / 16 Weeks More Comment
4 Conversation Ratio 3 Results: Instructor Badges 2 1 0 No Yes
Board settings suppress the influence of instructor participation Participate judiciously Positive reinforcement improves student interactions Discussion: Instructor Participation
Deng, L., & Tavares, N. J. (2013). From Moodle to Facebook: Exploring students' motivation and experiences in online communities. Computers & Education, 68, 167-176. Giannetto, D., Chao, J., & Fontana, A. (2013, July). Gamification in a social learning environment. In Proceedings of the Informing Science and Information Technology Education Conference (pp. 195-207). Informing Science Institute. Hew, K. F., Cheung, W. S., & Ng, C. S. L. (2010). Student contribution in asynchronous online discussion: A review of the research and empirical exploration. Instructional science, 38(6), 571-606. Hewitt, J. (2005). Toward an understanding of how threads die in asynchronous computer conferences. The journal of the learning sciences, 14(4), 567-589. Holman, C., Aguilar, S., & Fishman, B. (2013, April). GradeCraft: What can we learn from a game-inspired learning management system?. In Proceedings of the third international conference on learning analytics and knowledge (pp. 260-264). ACM. Jenkins, H., & Ito, M. (2015). Participatory culture in a networked era: A conversation on youth, learning, commerce, and politics. John Wiley & Sons. Sheldon, P. (2012). Profiling the non-users: Examination of life-position indicators, sensation seeking, shyness, and loneliness among users and non-users of social network sites. Computers in Human Behavior, 28(5), 1960-1965. Vonderwell, S., & Zachariah, S. (2005). Factors that influence participation in online learning. Journal of Research on Technology in education, 38(2), 213-230. More Questions?
