Analysis of Online Student Success Outcomes

Slide Note

This study delves into evaluating online student success outcomes, highlighting actionable research findings. It explores the differences between online and face-to-face education, emphasizing best practices for effective online learning. The research also discusses continuous quality improvement strategies in the context of online education at UCI.

zain_360 Follow

Uploaded on Oct 07, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript

Evaluation of Online Student Success: Actionable Research Outcomes Preston Reed Principal Research Analyst, Office of Institutional Research preston.reed@uci.edu 10/7/2024 1

Online Education 10/7/2024 2

National Online Enrollments (IPEDS) 10/7/2024 3

UC Online Enrollments (IPEDS) 10/7/2024 4

UCI Strategic Plan Online education found a home in our strategic plan 10/7/2024 5

Online vs Face-To-Face 10/7/2024 6

Online vs. Face-to-Face: (Dis)parity Mixed findings Dept. of Ed large meta-analysis (Means et al., 2010) All experimental or quasi-experimental Found outcomes were *better* for online compared to face-to-face (F2F) Online penalty found in other studies Large observational CC studies have found performance gaps (e.g., Xu & Jaggars, 2014) 10/7/2024 7

Online Ed: Best Practices So what are some of the best practices ? Organization and Presentation Learning objectives and assessments presented clearly Interpersonal interaction Appropriate use of technology 10/7/2024 8

What does that mean for UCI? 10/7/2024 9

Continuous Quality Improvement (CQI) Integrating quality improvement into the daily ongoings of the system (Park et al., 2013) CQI personified by 3 characteristics Frequency Depth Context within the system 10/7/2024 10

CQI at UCI 10/7/2024 11

The present study Goal to examine student outcomes in success for online relative to F2F 10/7/2024 12

Key outcome Student success Successful: Earned a grade of Pass or C or better Unsuccessful: Earned a grade of No Pass, C- or lower Dropped after census Withdrew from the course 10/7/2024 13

Selection Criteria Data collected from 14 lower-division courses over 14 terms 10/7/2024 14

Analyses Propensity Score Matching (PSM) used to match enrollments on several key demographic and academic characteristics 10/7/2024 15

In Brief: What is PSM? Looks at set of predictor variables to predict likelihood of being in treatment or control group Matches people in both groups on predicted probability This incorporates confounding variables Can be easier to communicate findings 10/7/2024 16

Matching Characteristics Grouped into Demographics E.g., age, gender, low-income Academics E.g., SAT Scores, GPA, Enrolled units 10/7/2024 17

Analyses Resulted in 8,374 matched enrollments 58 course offerings 4,187 Face to face 4,187 Online Post-matching, no significant differences between groups on any matched variable 10/7/2024 18

Results Across all courses: Students in online courses were less likely to succeed than those in face-to-face courses 84.4% vs. 90.8%, p < .001 Small effect size r2 = .01 Success Rates by Instruction Method 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Face to Face Online 10/7/2024 19

Results by Course Success rates between online and face to face methodologies varied between courses. 10/7/2024 20

Results by Course 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Public Health Psychology Economics Statistics Physics Biology Classics Soc Sci - Computer Research Writing - Argument and Rsrch Astronomy Music Soc Sci Research Methods Anthropology Writing - Crit Read and Rhetoric Face to Face Online 10/7/2024 21

Courses with significant success discrepancies 96.0% 100.0% 98.5% 94.2% 91.9% 87.3% 90.0% 81.9% 78.4% 80.0% 74.5% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Classics Economics Public Health Psychology Face to Face Online 10/7/2024 22

Bringing Findings to Key Stakeholders Findings supported exactly what administrators would have thought Four courses with high discrepancy Not using current best practices Mostly passive online reading assignments. 10/7/2024 23

Limitations Findings based on a specific subset of enrollments and online course offerings e.g., offered online and face to face same term, more than 100 enrolled in each medium; students who entered UCI as freshmen Need to go beyond Success or Failure as a measure of instruction method efficacy For example, how students perform in next course in series (e.g., Bio 93 to Bio 94) 10/7/2024 24

Moving forward: How UCI is using this information Used analyses as leverage to approach instructors to retool courses to incorporate best practices. Using class with very little discrepancy as model for other classes Using funding from UCOP and other initiative funding to retool courses 10/7/2024 25

Moving forward: How UCI is using this information? Will follow up with classes in the future 10/7/2024 26

Takeaways Can t simply assume prior findings applicable at your campus Even if you have studied online ed at your campus, it may not be the same for all offerings 10/7/2024 27

Takeaways Should continuously evaluate and improve offerings at your campus If things go right, may result in real changes that improve students experiences at your school Be sure to know your stakeholders More likely to enact change 10/7/2024 28

Thanks! Colleagues at UCI OIR Sarah Eichhorn, Associate Vice Provost for Teaching and Learning UCI Teaching and Learning Research Center Di Xu, Assistant Professor Education UCI Digital Learning Lab 10/7/2024 29