Impacts of Advanced Placement (AP) Policy at Ohio Public Universities
The study examines the impacts of the AP policy at Ohio public four-year university main campuses. Annual data comparisons show no significant effects on academic outcomes, indicating equivalency between AP credits and on-campus courses. The policy, implemented in 2009, guarantees college credit for AP test scores of 3, 4, and 5, leading to resource savings. Uniform standards ensure credit transfer and meeting graduation requirements. Overall, the AP policy facilitates credit transfers, reduces on-campus hours, and increases AP credit hours for graduates.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Impacts of the Advanced Placement (AP) Policy at Ohio Public Four-Year University Main Campuses: A Summary of Findings Shoumi Mustafa December 8, 2017
The Advanced Placement (AP) policy: What do we know of its impacts? Annual updating of the data Comparisons of academic outcomes between AP policy beneficiary & control groups with 12 years of data also confirm findings from studies on AP policy impacts. The AP policy helps save resources & does not affect academic outcomes. The AP Policy Guarantees college credit for AP test scores of 3, 4, 5. Replaces on-campus hours with AP credit & saves resources. Equivalency of learning outcomes between on- campus courses and AP credit ensures that academic outcomes remain unchanged. Equivalent learning outcomes? Multiple studies show that AP policy does not affect short or long-term academic outcomes. As such, the equivalency of learning outcomes is confirmed. Also, studies show that for the graduates, AP policy increases the number of AP credit hours, and reduces the number of completed on-campus hours. 2
AP policy: A brief history In 2007, the state legislature mandated adoption of standards by Ohio public institutions of higher education in awarding credit for AP tests. Committee comprising representatives from the institutions and the Ohio Department of Higher Education (Board of Regents) developed principles for the AP policy. Ohio Articulation and Transfer Advisory Council approved the resultant AP policy, and the Chancellor of the Ohio Department of Higher Education endorsed it. Directive 2008-010 was issued in Summer 2009; AP policy was implemented in FY2009-10 (Autumn 2009). Some institutions may have made their FY2009-10 AP credit decisions prior to receiving the directive; for them, AP policy was implemented in FY2010-11 (Autumn 2010). 3
AP policy: Components Uniform standards: College credit for AP test scores of 3, 4, or 5. Satisfaction of at least the first year of foreign language requirement for a score of 3 in an AP foreign language test. Availability of information from institutions: Complete lists of equivalent courses and corresponding hours of credits for test scores of 3, 4 and 5 in each AP test. Equivalency of AP and college credit: Credits for AP tests meet graduation requirements & transfer among institutions subject to transfer policy rules. 4
AP policy: Before-and-after scenarios Before AP policy, institutions decided: Whether to grant credit for AP tests. How many hours to grant in which courses. Whether AP credits would meet graduation requirements. Whether AP credits would transfer. After policy implementation, uniform standards require: AP test scores of 3, 4 and 5 receive college credit. Institutions inform in advance the number of hours granted in specific courses for test scores of 3, 4 and 5 in each AP test. AP credits meet graduation requirements & transfer among institutions. 5
AP policy: Desired outcomes and key assumption Desired outcomes: Students accumulate early college credit. Face shorter required time for graduation at any USO institution. Have opportunity for reducing college costs. Head-start could improve subsequent academic success. Key assumption: Learning outcomes of AP test scores of 3, 4 and 5 are equivalent to learning outcomes associated with successful completions of corresponding college courses. 6
On-campus hours and AP credit: Are the learning outcomes equivalent? What if the learning outcomes are not equivalent? If AP tests scores of 3, 4 and 5 do not prepare students adequately for college level work, subsequent academic success of AP policy beneficiaries would be adversely affected. An investigation of the equivalency is in order. Given the expected benefits of the AP policy, and the potential for compromised academic standards, it is necessary to estimate the impact of the AP policy on academic outcomes of policy beneficiaries. 7
Interpretation of estimated impacts The AP policy is beneficial if, Academic outcomes of AP policy beneficiaries improve as a result of the policy. Academic outcomes of AP policy beneficiaries do not change as a result of the policy. In both cases, learning outcomes associated with AP test scores of 3,4, and 5 are at least equivalent to the same from completing successfully corresponding college courses. A rethinking of the AP policy is warranted only if the policy has an adverse impact on academic outcomes of policy beneficiaries. 8
Indicators of academic outcomes Short-term indicators (from the first year of attendance): Grade Point Average (on a scale of 0 to 4) Number of attempted hours (converted to semester hours) Course completion rates Retention rates (first-to-second year) Long-term indicators: Four-year graduation rates Number of completed hours at graduation 9
Difference-in-difference estimator: Methodology Step 1: Calculate the difference in academic outcome for students eligible to receive AP credit: Before and after policy. What does the difference represent? Policy impact + impact of changing non-policy determinants of academic outcome. Step 2: Calculate the difference in academic outcome for students ineligible for AP credit: Before and after policy. What does the difference represent? Impact of changing non-policy determinants of academic outcome. Step 3: Calculate the difference of the differences from Step 1 & Step 2. The difference-in-difference estimator represents policy impact. 10
The influence of other factors The estimation of AP policy impact takes into account also of the influence of: Gender, Age, and Ethnicity. ACT scores. Family income. High school characteristics. Overall influence of the college of attendance. Additionally, estimation takes into account that standard errors are correlated among clusters of students attending the same institution. 11
First study for estimating short-term impacts Short-term study 1: Completed in 2013. Data: Sample of first-time freshman students enrolled in Ohio public four-year university main campuses FY2007-08 and FY2010-11. FY2007-08 and FY2008-09 represent the before policy period. FY2009-10 and FY2010-11 represent the after policy period. Who are the policy beneficiaries? Students with AP test scores of 3, 4 and 5 after policy implementation. Who are in the comparison group? Students without AP credits. dual enrollment credit recipients are excluded from this group. Indicators of academic outcome: GPA, attempted hours, course completion rate all from the first year of college attendance. 12
Second study for estimating short-term impacts Second short-term study: Completed in 2016. Data: Sample of first-time freshman students enrolled in Ohio public four-year university main campuses FY2005-06 and FY2012-13. 2005-06, 2006-07, 2007-08, and 2009-10 cohorts from before AP policy 2009-10, 2010-11, 2011-12, and 2012-13 cohorts from after AP policy Indicators of academic outcome: GPA, attempted hours, completion ratio all from the first year of college attendance, and first-to-second year rates of retention. Definitions of beneficiary and control groups are the same as the first study on short-term impacts. 13
Study for estimating long-term impacts Long-term study: Completed in 2016. Data: Sample of first-time freshman students enrolled in Ohio public four-year university main campuses FY2005-06 and FY2012-13. 2007-08, and 2009-10 cohorts from before AP policy 2009-10, 2010-11 cohorts from after AP policy Indicators of academic outcome: Four-year graduation rates Number of AP credit hours granted Number of on-campus completed hours at graduation. Definitions of beneficiary and control groups are the same as the first and the second studies on short-term impacts. 14
First short-term study results: AP policy impacts Sample includes 2007-08 through 2010-11 cohorts Indicators of academic outcomes Academic outcome indicator measured for AP credit received for Results Grade Point Average (GPA) No impact. Corresponding college course for any AP test for which students received scores of 3,4, or 5. Number of attempted hours No impact. All on-campus courses. Overall course completion rates No impact. English Language or Literature, a pre- requisite for English Composition II. English Composition II No impact. Completion rates in sequential courses when students received AP credit for pre or co-requisite courses. Calculus I, a co-requisite for Physics II Physics I No impact. Calculus I, pre-requisite for Calculus II Calculus II No impact. 15
Second short-term study results: AP policy impacts Sample includes 2005-06 through 2012-13 cohorts Indicators of academic outcomes Results Grade Point Average (GPA) No impact. Attempted hours No impact. Overall course completion rates No impact. An increase of 1.1 percentage points (1.2% relative to the pre-policy rate of 94.1%). Retention rates, from first to second year 16
Long-term study results: AP policy impacts Indicators of academic outcome Results Four-year graduation rate No impact. An increase of 2.5 hours. Number of AP credit hours granted on average An increase of 29.4%, relative to pre-policy average value of 8.5 hours. A decline of 1.7 hours. For graduates, the number of completed on-campus hours A decline of 1.3%, relative to the pre-policy average value of 128.3 hours. 17
What do the new data show? Comparison of AP policy beneficiary and control group averages by year: between 2005-06 and 2016-17 freshman cohorts. Indicators: GPA Number of attempted hours Course completion rates An additional comparison of the number of students in the AP policy beneficiary and control groups is also presented. 18
Comparison of beneficiary and comparison groups: The number of policy beneficiaries on the rise Number of Students: AP Policy Beneficiary & Control Groups First-Time Freshman Students: Ohio Public Four-Year University Main Campuses 25,000 23,667 23,11823,38322,54122,793 22,243 20,888 20,000 15,000 12,477 10,26710,75611,203 10,000 8,601 7,850 7,055 5,000 0 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 Control Group AP Policy Beneficiary Group Students with AP test scores of 3,4,5 Students without AP or dual credit 19
GPA comparison - policy beneficiary and control groups: Difference in average GPA scores remains constant Comparison of GPA: AP Policy Beneficiary and Control Groups First-Time Freshman Cohorts: Ohio Public Four-Year University Main Campuses 4.000 3.261 3.243 3.241 3.243 3.229 3.224 3.220 3.171 3.165 3.143 3.089 3.065 3.000 2.512 2.485 2.482 2.474 2.450 2.454 2.417 2.406 2.392 2.388 2.371 2.356 2.000 1.000 0.000 AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 20
Attempted hours comparison - beneficiary and control groups: Difference in average attempted hours remains constant Comparison of Attempted Hours: AP Policy Beneficiary and Control Groups First-Time Freshman Cohorts: Ohio Public University Main Campuses 35.0 32.3 32.2 32.2 32.2 32.1 32.0 32.0 31.2 31.0 30.9 30.9 30.9 30.0 28.6 28.4 28.3 28.0 27.9 27.9 27.9 27.8 27.8 27.7 27.6 27.5 25.0 20.0 15.0 10.0 5.0 0.0 AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 21
Course completion rates - beneficiary and control groups: Difference in course completion rates remains constant Comparison of Completion Rates: AP Policy Beneficiary and Control Groups First-Time Freshman Cohorts: Ohio Public Four-Year University Main Campuses 100.0% 94.8% 94.5% 94.6% 94.5% 94.6% 94.5% 94.3% 94.3% 94.3% 94.3% 93.9% 93.7% 90.0% 83% 82.8% 82.8% 82.6% 81.4% 81.2% 80.8% 79.8% 79.9% 79.2% 78.8% 78.2% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0 AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control AP Control 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 22
What did we learn? Impact on short-term indicators: No impact on GPA, number of attempted hours, course completion rates. No impact on course completion rates in sequential courses when students received AP credit for pre or co-requisite courses. A small positive impact of 1.1 percentage points on first-to-second year retention rates. Impact on long-term indicators: No impact on graduation rates. An increase in the number of AP credit hours granted to students: of 2.7 hours. For graduates, a decline in the number of completed on-campus hours, of 1.7 hours. Recent trends in the data: Comparison of 12 years of data reveals that average GPA, number of attempted hours, and course-completion rates differ little between AP credit recipients and the control group before and after the AP policy implementations. 23
