Massachusetts Next-Generation Accountability & Assistance System Overview
Presented to the Board of Elementary & Secondary Education in May 2018, this overview covers the timeline, process, accountability discussions, system highlights, and other related discussions. It includes key events such as discussions on accountability indicators, weighting, normative and criterion-referenced components, categorization of schools, and reporting. The system aims to provide actionable information for schools and districts to focus on continuous improvement and support the performance of all students.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Massachusetts Next-Generation Accountability & Assistance System Presentation to the Board of Elementary & Secondary Education May 2018 Russell Johnston, Senior Associate Commissioner Rob Curtin, Associate Commissioner Ventura Rodriguez, Associate Commissioner
1. 2. 3. 4. 5. 6. 7. 8. Timeline & process System highlights Accountability indicators Weighting of accountability indicators Normative component Criterion-referenced component Categorization of schools & districts Reporting AGENDA
1 Timeline & process
Accountability discussions with the Board December 2017 January 2018 March 2018 May 2018 June 2018 September 2018 BESE discussion about weighting of accountability indicators BESE discussion & vote to solicit public comment on proposed amendments to state accountability regulations (603 CMR 2.00) BESE discussion about accountability & assistance system BESE discussion & vote on proposed amendments to state accountability regulations BESE vote on new accountability system BESE discussion about accountability system design ESE publishes 2018 accountability results for all public schools & districts 4 Massachusetts Department of Elementary and Secondary Education
Other accountability discussions Accountability information sessions Accountability discussions Approximately 500 district & school leaders attended in-person information sessions in: o Marlborough o Malden o Holyoke o Taunton o Lynn Public Schools o Boston Public Schools Accountability & Assistance Advisory Council Racial Imbalance Advisory Council American Federation of Teachers & Massachusetts Teachers Association Massachusetts Business Alliance for Education Massachusetts Association of School Committees Urban Superintendents Network 5 Massachusetts Department of Elementary and Secondary Education
2 System highlights
System highlights Additional accountability indicators o Provide information about school performance & student opportunities beyond test scores Provide actionable information to districts and schools focused on continuous improvement Normative & criterion-referenced components Focus on raising the performance of each school's lowest performing students o In addition to the performance of the school as a whole Discontinuation of accountability & assistance levels 1-5 o Replaced with accountability categories that define the progress that schools are making & the type of support they may receive from the Department Districts classified based on district-level data o No longer based on the performance of a districts lowest performing school 7 Massachusetts Department of Elementary and Secondary Education
3 Accountability indicators
Massachusetts accountability indicators non-high schools Indicator Measure English language arts (ELA) average scaled score Mathematics average scaled score Science achievement (Composite Performance Index (CPI)) Achievement ELA mean student growth percentile (SGP) Mathematics mean SGP Student Growth English Language Proficiency Progress made by students towards attaining English language proficiency (percentage of students meeting annual targets required in order to attain English proficiency in six years) Chronic absenteeism (percentage of students missing 10 percent or more of their days in membership) Additional Indicator(s) 9 Massachusetts Department of Elementary and Secondary Education
Massachusetts accountability indicators high schools Indicator Measure English language arts (ELA) achievement (Composite Performance Index (CPI)) Mathematics achievement (CPI) Science achievement (CPI) Achievement ELA mean student growth percentile (SGP) Mathematics mean SGP Student Growth Four-year cohort graduation rate Extended engagement rate (five-year cohort graduation rate plus the percentage of students still enrolled) Annual dropout rate High School Completion English Language Proficiency Progress made by students towards attaining English language proficiency (percentage of students meeting annual targets required in order to attain English proficiency in six years) Chronic absenteeism (percentage of students missing 10 percent or more of their days in membership) Percentage of 11th& 12thgraders completing advanced coursework (Advanced Placement, International Baccalaureate, postsecondary courses, &/or other selected rigorous courses. Note: MassCore may be included in the future) Additional Indicator(s) 10 Massachusetts Department of Elementary and Secondary Education
4 Weighting of accountability indicators
Considerations for weighting achievement & growth The current ratio of achievement & growth is 3 (achievement) to 1 (growth) Impact of increasing weight of growth in system: o Could increase differentiation between similarly-achieving schools o Increases the value of a normative measure (there will always be a 1st percentile & a 99thpercentile) o Decreases the value of grade 3 assessment results (no SGP for students in grade 3) o Decreases value of science assessment in system (no SGP for science) 12 Massachusetts Department of Elementary and Secondary Education
Considerations for weighting achievement & growth All indicators need to be included in the weighting Progress towards English language proficiency only applies to a subset of schools, & weighting needs to be flexible Ratio between achievement & growth can be held constant between non-high schools & high schools but actual weightings will differ ESE intends to apply the same weighting rules to both the normative & criterion-referenced components of the system Recommendation is to maintain current ratio (3:1) 13 Massachusetts Department of Elementary and Secondary Education
Proposed weighting of indicators in non-high schools Current Weighting 3:1 With ELL Indicator Measures No ELL ELA, math, & science achievement values (based on scaled score) Achievement 60% 67% Student Growth ELA/Math Student Growth Percentile (SGP) 20% 23% English Language Proficiency Progress made by students towards attaining English language proficiency 10% Additional Indicators Chronic absenteeism 10% 10% 14 Massachusetts Department of Elementary and Secondary Education
Proposed weighting of indicators in high schools & middle/high/K-12 schools Current Weighting 3:1 With ELL Indicator Measures No ELL Achievement ELA, math, & science achievement 40% 48% Student Growth ELA/Math Student Growth Percentile (SGP) 20% 22% Four-year cohort graduation rate Extended engagement rate Annual dropout rate High School Completion 20% 20% English Language Proficiency Progress made by students towards attaining English language proficiency 10% Chronic absenteeism Percentage of students completing advanced coursework Additional Indicators 10% 10% 15 Massachusetts Department of Elementary and Secondary Education
5 Normative component
Normative component Accountability percentile 1-99, calculated using all available indicators for a school Compares schools administering similar statewide assessments Used to identify the lowest performing schools in the state Same calculation used at the subgroup level to identify low- performing subgroups 17 Massachusetts Department of Elementary and Secondary Education
Comparisons Schools will be grouped & compared based on the assessment(s) administered in 2018 o Non-high schools Serving only a combination of grades 3-8 Administering only Next-Generation MCAS tests in ELA & math o Middle/high/K-12 schools Serving grade 10 & at least one other grade 3-8 Administering a combination of Next-Generation & legacy MCAS tests in grades 3-8 & 10 o High schools Schools in which the only tested grade is grade 10 Administering only legacy MCAS tests Separate comparison categories will not be necessary once all grades/tests have transitioned to Next-Generation MCAS 18 Massachusetts Department of Elementary and Secondary Education
6 Criterion-referenced component
Criterion-referenced component Focus on closing the achievement gap by raising the achievement floor o Gap-closing can occur as a result of a decline in performance by the high- performing group In addition to meeting targets for the school as a whole, the performance of the lowest performing students in each school will be measured o Every school has a group of lowest performers o Identified from cohort of students who were enrolled in the school for more than one year 20 Massachusetts Department of Elementary and Secondary Education
Lowest performing students cohort model For most schools serving grades 3-8, these students were: o Officially enrolled in current school for two consecutive years; October 1, 2016 through October 1, 2017 (SIMS) o Tested in the enrolled school in 2017 & 2018; & o Not a first- or second-year English learner in 2018 In schools where a legitimate cohort cannot be identified (fewer than 20 students), accountability results will be based on the performance of the all students group only 21 Massachusetts Department of Elementary and Secondary Education
Lowest performing students year-to-year approach In high schools, the cohort model cannot be used Improvement will be measured using a year-to-year approach based on students who were: o Officially enrolled in current school for two consecutive years; October 1, 2016 through October 1, 2017 (SIMS) o Tested in grade 10 in enrolled school in 2018, & attended grade 9 in the same school or district in 2017; & o Not a first- or second-year English learner in 2018 In schools where a legitimate group cannot be identified (fewer than 20 students), accountability results will be based on the performance of the all students group only 22 Massachusetts Department of Elementary and Secondary Education
Criterion-referenced component Targets set for each accountability indicator, for the school as a whole & for the lowest performing students in each school Non-high schools High schools & middle/high/K-12 schools Lowest performing students Lowest performing students Indicator All students All students ELA scaled score Math scaled score Science achievement ELA SGP Math SGP Four-year cohort graduation rate Extended engagement rate Annual dropout rate EL progress Chronic absenteeism Advanced coursework completion N/A N/A N/A N/A N/A N/A N/A N/A *Minimum group size for each indicator = 20 students 23 Massachusetts Department of Elementary and Secondary Education
Setting targets For 2018 reporting, targets will only be set for one year oLong-term targets will be set in the future Targets will be set based on historical improvement of like- performing schools oLike-performing schools defined as within the same quartile of schools based on historical school percentiles 24 Massachusetts Department of Elementary and Secondary Education
Criterion-referenced component Points assigned based on progress toward target for each indicator, for both the aggregate & the lowest performing students Declined No change Improved Met target Exceeded target 0 1 2 3 4 25 Massachusetts Department of Elementary and Secondary Education
Criterion-referenced component calculation non-high school All students (50%) Lowest performing students (50%) Indicator Points earned Total possible points Points earned Total possible points Weight Weight ELA scaled score Math scaled score Science achievement Achievement total ELA SGP Math SGP Growth total EL progress Chronic absenteeism 3 2 2 7 4 3 7 2 3 4 4 4 - - - 2 2 - 4 4 4 8 - 4 4 4 - 8 4 4 8 - 4 - - - 12 4 4 8 4 4 60% - - 20% 10% 10% 67% - - 23% - 10% Weighted total 6.1 9.6 - 4.9 7.6 - Percentage of possible points 63.5% - 64.7% - Criterion-referenced target percentage 64% 26 Massachusetts Department of Elementary and Secondary Education
Criterion-referenced component calculation high school All students (50%) Total possible points 4 4 4 12 4 4 8 4 4 4 12 4 4 4 8 10.0 70.0% Lowest performing students (50%) Points earned points 2 2 1 5 12 4 4 8 - - - - - 4 - 4 5.6 10.3 54.4% Indicator Points earned 3 2 2 7 4 3 7 3 4 3 10 2 3 3 6 7.0 Total possible Weight Weight ELA achievement Math achievement Science achievement Achievement total ELA SGP Math SGP Growth total Four-year cohort graduation rate Extended engagement rate Annual dropout rate High school completion total EL progress Chronic absenteeism Advanced coursework completion Additional indicators total Weighted total Percentage of possible points Criterion-referenced target percentage - - - 4 4 4 - - - 40% - - 20% - - - 20% 10% - - 10% - - 67% - - 23% - - - - - - - 10% - - 4 4 8 - - - - - 4 - 4 62% 27 Massachusetts Department of Elementary and Secondary Education
7 Categorization of schools & districts
Categorization of schools Schools will no longer be placed in a vertical hierarchy of levels 1-5 Number of schools that will be placed into a category based upon a relative standing will be cut in half from previous system o Approximately 90 percent of schools could be categorized based on their own performance against targets Most schools will have 50 percent of its categorization based on students that have been in the school for at least two years Category labels are primarily tied to the level of required assistance or intervention Stronger emphasis on schools commended for success 29 Massachusetts Department of Elementary and Secondary Education
Categorization of schools Accountability determinations for schools will be based on 5 factors: 1. Accountability percentile (1-99) 2. Criterion-referenced target percentage (0-100%) 3. Subgroup performance (subgroup percentile 1-99) 4. Graduation rate 5. Assessment participation 30 Massachusetts Department of Elementary and Secondary Education
Categorization of schools Schools without required assistance or intervention (approx. 85%) Schools requiring assistance or intervention (approx. 15%) Schools of recognition Meeting targets Partially meeting targets Focused/targeted support Broad/ comprehensive support Schools demonstrating high achievement, significant improvement, or high growth Criterion-referenced target percentage 75-100 Criterion-referenced target percentage 0-74 Non-comprehensive support schools with percentiles 1-10 Schools with low graduation rate Schools with low performing subgroups Schools with low participation Underperforming schools Chronically underperforming schools 2018: Performance against targets reported in 2 categories (meeting & partially meeting 2019: Performance against targets reported in 3 categories (meeting, partially meeting, & not meeting) Notes: School percentiles & performance against targets will be reported for all schools 31 Massachusetts Department of Elementary and Secondary Education
Categorization of districts Districts without required assistance or intervention Districts requiring assistance or intervention Meeting targets Partially meeting targets Focused/targeted support Broad/ comprehensive support Criterion-referenced target percentage 75-100 Criterion-referenced target percentage 0-74 Districts with low graduation rate Districts with low participation Underperforming districts Chronically underperforming districts 2018: Performance against targets reported in 2 categories (meeting & partially meeting 2019: Performance against targets reported in 3 categories (meeting, partially meeting, & not meeting) Notes: Performance against targets will be reported for all districts 32 Massachusetts Department of Elementary and Secondary Education
8 Reporting
Accountability reports Accountability reports published for each district & school (fall 2018) Reports will include: o Overall classification Including reason(s) for classification (e.g., low graduation rate, low performing subgroup) o Criterion-referenced target percentage o Accountability percentile (schools only) o Data related to performance on each accountability indicator for each subgroup meeting the minimum group size (20 students) All students Lowest performing students High needs students English learners Students with disabilities Economically disadvantaged students Major racial/ethnic subgroups 34 Massachusetts Department of Elementary and Secondary Education
ESE Turnaround Assistance Redesign: New Opportunities 2018 and beyond
Current Assistance Model View from Schools and Districts Level 3 District Level 4 District Level 5 District 36 Massachusetts Department of Elementary and Secondary Education
Current Assistance Model Massachusetts will turn around the lowest performing schools and districts by supporting schools in their implementation of improvement strategies with a variety of proven and context-specific interventions. (ESE Strategic Plan Goal #3 May 2015) Statewide System of Support Strategic Transformation Level 5 schools/districts District & School Turnaround 8 large urban districts DSACs 60 Small and Medium Sized Districts 37 Massachusetts Department of Elementary and Secondary Education
Statewide System of Support Redesign Why Restructure Now? Design Principles New Accountability Focus Reduced resources Inequitable distribution of support Added responsibilities for schools and districts in receivership Build on current successful practices Focus on schools in the 10th percentile Greater consistency and equity across the SSoS Identify efficiencies 38 Massachusetts Department of Elementary and Secondary Education
New Structure for the Statewide System of Support Reconfigured Regional Assistance Delivery Regional Leadership Dedicated Assistance Leads Continuing Focus on Successful Turnaround Practices Codify, prioritize, and train staff on deliver high quality assistance offerings with a track record of success Data-Driven Reflection and Improvement 39 Massachusetts Department of Elementary and Secondary Education
Current Assistance Model View from Schools and Districts Chronically underperforming district Focused/targeted support district Underperforming district 40 Massachusetts Department of Elementary and Secondary Education
