Impact of PLTW Program in Massachusetts: Insights and Progress
A five-year evaluation of the PLTW program in Massachusetts reveals significant growth in student representation, positive impacts on MCAS scores, and increased student engagement. The study highlights areas for improvement such as recruitment, support for teachers, and enhancing student feedback. Overall, PLTW has expanded substantially, benefiting both students and educators in the state.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
PLTW in Massachusetts Implementation and Impact 1 John Papay October 2019 1
PLTW Scaling Impact Study Five-year evaluation designed to understand the impact and implementation of scaling PLTW offerings in MA Key data sources Administrative records from the Massachusetts DESE Student and teacher surveys PLTW roster data and EOC data Critical that all schools roster all kids in all classes 2
Key Lessons (1) Progress continues from last year PLTW has again grown substantially Students in PLTW have become more representative of the state as a whole as a result of the grant program Teachers and students are quite satisfied with PLTW (2) New results shed light on broader impacts of PLTW Participating in PLTW courses appears to increase MCAS scores, particularly in high school Impacts are equally large for a range of students PLTW seems to promote student engagement (3) Areas for growth Recruitment and representativeness, particularly in CS and Engineering Time in PLTW classes Support and additional programming for teachers Feedback for students 3
Key Lessons (1) Progress continues from last year (2) New results shed light on broader impacts of PLTW (3) Areas for growth 4
PLTW has grown substantially in recent years as a result of the grant program Number of unique PLTW students, over time 5
PLTW has become much more representative of students in MA 6
PLTW has become much more representative of students in MA 7
66% of Gateway students report having positive experiences with PLTW courses (vs. 16% negative) 8
79% of HS students report having positive experiences with PLTW courses (vs. 12% negative) 9
53% of Launch students report being quite or extremely excited about going to their PLTW class 10
HS students (but not Gateway students) are more interested in PLTW courses than other classes 35% of Gateway students are more interested in PLTW than other courses (vs. 44% less interested) 57% of HS students are more interested in PLTW than other courses (vs. 22% less interested) 11
59% of Gateway students and 77% of high school students say PLTW is valuable because it will help them succeed in other subjects 12
High school students say PLTW courses are valuable for future careers 77% say they learn things that will help them get a job 72% say PLTW classes will help their career more than other courses 13
74% of teachers report that students are more satisfied in PLTW than other classes How satisfied are your students in PLTW courses compared to other courses? 14
69% of teachers are quite or extremely satisfied with PLTW courses Based on your overall experience with PLTW, how satisfied are you with the program? 15
61% of teachers report being more satisfied with PLTW than with other classes (vs. 16% less satisfied) How satisfied are you with your PLTW classes relative to your non-PLTW classes? 16
63% of teachers report that PLTW increased their satisfaction with teaching (vs. 3% decreased) 17
63% of teachers report that PLTW increased their teaching efficacy (vs. 2% decreased) 18
Key Lessons (1) Progress continues from last year (2) New results shed light on broader impacts of PLTW (3) Areas for growth 19
Participating in PLTW courses appears to increase MCAS scores We compare students in PLTW courses to other, similar students who are not taking PLTW courses Similar prior-year test scores Similar demographics Similar schools We find: Mixed evidence in Launch (expected given implementation) Small positive impacts of Gateway courses Large positive impacts of High School courses 20
Participating in PLTW high school courses has large impacts on MCAS scores 1 SD 15 scale points on science & legacy MCAS (23 points on new MCAS) 21
Participating in PLTW high school courses has large impacts on MCAS scores even if we compare students in the same school 22
The benefits of PLTW are potentially even larger for students with disabilities, students who are economically disadvantaged, and English learners Participating in PLTW increases math test scores by 0.21 SD for economically disadvantaged students (blue dot) and 0.14 SD for other students (red square) 23
PLTW high school students report being more interested and challenged in PLTW courses than other US teenagers Gallup asked 13-17 years olds to pick 3 words from a list of adjectives that describe how they usually feel in school Most common words: Bored (50%) and tired (42%) Many fewer students listed challenged (31%) or interested (22%) We asked the same question of PLTW high school students in MA. By far the most common words were challenged (55%) and interested (54%) Only 19% said bored 24
Key Lessons (1) Progress continues from last year (2) New results shed light on broader impacts of PLTW (3) Areas for growth 25
Potential areas for growth (1) Recruiting is important: High school courses particularly Engineering and CS remain less diverse (2) Time appears to be a critical factor for PLTW success (3) Teachers find supports and complementary programming valuable, but they are not well used (4) Students are eager for more feedback and find it valuable 26
(1) High school courses particularly Engineering and CS remain less diverse Girls are underrepresented in CS and ENG courses, while economically disadvantaged students are underrepresented in ENG. Patterns hold when we compare students in the same school. 27
(1) High school courses particularly Engineering and CS remain less diverse These courses underserve English learners and students with disabilities, despite evidence that PLTW courses may have larger impacts for these students. 28
(1) There may be opportunities to improve guidance in the course selection process 29
(1) Students report interest in subject, career, and hands-on work as top reasons for taking PTLW class 30
(2) Students at all levels report wanting more time 40% of Gateway students and 29% of high school students said they never, rarely, or occasionally had enough time. We asked students what would make their course more valuable 33% of Gateway students and 25% of high school students listed more time for each class period The 2nd most popular answer for Gateway 31
(2) Teachers report that the schedule is not fully conducive to PLTW programming 32
(2) In schools where students report having more time, they tend to be more satisfied We plot average student satisfaction against the average student rating for amount of time they have in class at the school level. Particularly in Gateway courses, students are much more satisfied in schools where they have more time. 33
(3) PLTW teachers were much more likely to use some supports in 2018-19 than they were in 2017-18 and often turned to peers for advice In general, teachers used more supports than in 2017-18 24% reported using a MA mentor teacher (vs. 15% last year) 38% reported using a PLTW master teacher from training (vs. 30% last year) The most common supports were peer teachers: 72% reported turning to a PLTW teacher in the school/district 52% reported turning to teachers they met at an event 51% reported turning to classmates from PLTW training 34
(3) Teachers who used Mass Stem Hub supports generally found them quite valuable 35
(3) A smaller percentage of teachers engaged in complementary programming in 2018-19 than 2017-18 36
(3) Teachers continued to find complementary programming quite valuable 37
(4) Students are generally satisfied with the feedback they receive from their teachers 60% of Gateway students say feedback was quite or extremely helpful. 73% of HS students say feedback was quite or extremely helpful. 38
(4) In classes where students report getting better feedback, they are much more satisfied with the overall experience We plot student satisfaction against the student rating for how valuable the feedback received was, on average in each class. Students are much more satisfied in schools where get better feedback. 39
Conclusion (1) Progress continues from last year PLTW has again grown substantially Students in PLTW have become more representative of the state as a whole as a result of the grant program Teachers and students are quite satisfied with PLTW (2) New results shed light on broader impacts of PLTW Participating in PLTW courses appears to increase MCAS scores, particularly in high school Impacts are equally large for a range of students PLTW seems to promote student engagement (3) Areas for growth Recruitment and representativeness, particularly in CS and Engineering Time in PLTW classes Support and additional programming for teachers Feedback for students 40
Questions? 41
