Understanding Workforce Innovation & Opportunity Act Performance
Today's discussion delves into the performance accountability indicators of the Workforce Innovation and Opportunity Act, focusing on employment rates, median earnings, credential attainment, and measurable skills gain for various programs. Key aspects such as negotiated levels of performance, statistical adjustment, technical assistance, and corrective action are highlighted to determine program success.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Understanding Workforce Innovation & Opportunity Act Performance
Today we will talk about: Performance Accountability Indicators Exclusions Negotiated Levels of Performance Statistical Adjustment Determining Success Technical Assistance & Corrective Action Questions & Discussion
Performance Accountability Indicators Employment in the 2nd and 4th quarters after exiting the program Median earning in the 2nd quarter after exiting the program Credential Attainment Measurable Skills Gain These indicators apply to Title 1A: Ponca Tribe of Nebraska, Title 1B: Adult, Dislocated Worker, & Youth Programs, Title 2: Nebraska VR & Nebraska Commission for the Blind and Visually Impaired, Title 3: Nebraska Department of Labor Employment Services, Title 4: Southeast Community College Adult Education & Family Literacy, Proetus, Pine Ridge Job Corps, National Able Network, & Nebraska Department of Labor Programs: Jobs for Veterans State Grant, National Dislocated Worker Grants, and Trade Adjustment Assistance.
Employment Rate- 2nd Quarter After Exit The percentage of participants who are in unsubsidized employment during the second quarter after exit from the program. Methodology: The number of participants who exited during the reporting period who are found to be employed through a records search in the second quarter after the exit quarter. divided by The number of participants who exited during the reporting period. (for the title I Youth program, the indicator is the percentage of program participants in education or training activities, or unsubsidized employment, during the second quarter after exit)
Employment Rate- 4th Quarter After Exit The percentage of participants who are in unsubsidized employment during the fourth quarter after exit from the program. Methodology: The number of participants who exited during the reporting period who are found to be employed through a records search in the fourth quarter after the exit quarter. divided by The number of participants who exited during the reporting period. (for the title I Youth program, the indicator is the percentage of program participants in education or training activities, or unsubsidized employment, during the second quarter after exit)
Median Earnings- 2nd Quarter After Exit The median earnings of participants who are in unsubsidized employment during the second quarter after exit from the program. Methodology: Total quarterly earnings for all participants employed in the second quarter after exit are collected either by direct wage record match or supplemental wage information. The collected quarterly wage information values are listed in order, from the lowest to the highest value. The value in the middle in this list is the median earnings value.
Credential Attainment The percentage of participants enrolled in an education or training program (excluding those in on-the-job training and customized training) who attain a recognized postsecondary credential or a secondary school diploma, or its recognized equivalent, during participation or within one year after exit from the program. Methodology: The number of participants who exited during the reporting period who obtained a recognized postsecondary credential during the program or within one year after exit or those who were in a secondary education program and obtained a secondary school diploma or its recognized equivalent during the program or within one year of exit and were also employed, or in an education program leading to a recognized postsecondary credential within one year after exit. divided by The number of participants enrolled in an education or training program (excluding those in OJT and customized training) who exited during the reporting period.
Measurable Skill Gains The percentage of program participants who, during a program year, are in an education or training program that leads to a recognized postsecondary credential or employment and who are achieving measurable skills gains, defined as documented academic, technical, occupational, or other forms of progress, towards such a credential or employment. Depending on the type of education or training program in which a participant is enrolled, documented progress is defined as one of the following: Educational functioning level gain (below post-secondary level); Diploma or recognized equivalent (secondary school); Secondary or postsecondary transcript or report card; Satisfactory or better progress report, towards established milestones; or Successful passage of a knowledge-based exam that is required for a particular occupation or progress in attaining technical or occupational skills.
Measurable Skill Gains Methodology : The number of program participants during the reporting period who are in an education or training program that leads to a recognized postsecondary credential or employment and are achieving measurable skill gains based on attainment of at least one type of gain. divided by The number of program participants during the reporting period who are in an education or training program that leads to a recognized postsecondary credential or employment. Participants who, during any point in the program year, are placed in an education or training program that leads to a recognized postsecondary credential or employment are included in the denominator. This includes participants who continue to receive services and ones who have exited the program. Note: A participant may have achieved more than one type of gain in a reporting period. However, only one gain per participant in a reporting period may be used to calculate success on the measurable skill gains indicator.
Exclusions Participants who exit from services for reasons identified below are excluded from USDOL s Common Measures. Any participant considered excluded must have documentation in the file which verifies the reason for exclusion. Institutionalized - The participant exits the program because he or she has become incarcerated in a correctional institution or has become a resident of an institution or facility providing 24- hour support, such as a hospital or treatment center, during the course of receiving services as a participant. Health/Medical - The participant exits the program because of medical treatment and that treatment is expected to last longer than 90 days and precludes entry into unsubsidized employment or continued participation in the program. Deceased - a participant is no longer living Reservists or National Guard called to active duty a participant that is a reservist or national guard called to active duty for at least 90 days Foster Care - The participant is in the foster care system and exits the program because the participant has moved from the local workforce area as part of such a program or system.
Negotiated Levels of Performance The negotiated levels of performance are the levels of performance for each primary indicator for each core program, agreed to by the State and the Secretaries, prior to the start of the program year. In addition to the State-negotiated levels of performance, States must work with local areas to establish performance goals for WIOA Title I programs. The local board, the chief elected official, and the Governor must negotiate and reach agreement on the local levels of performance. The performance levels are negotiated for a two-year period.
Negotiated Levels of Performance for Program Years 2020 & 2021 Performance indicator Adult Employment rate 2nd quarter after exit Dislocated Worker Youth 79.0% 89.0% 79.0% Employment rate 4th quarter after exit 79.0% 92.0% 78.0% Median earnings 2nd quarter after exit $7,749 $8,590 $3,550 Credential attainment rate within 4 quarters after exit Measurable skill gains 77.0% 66.0% 55.0% 42.0% 55.0% 50.0%
Determining Success Individual local-area single indicator score. Failure occurs when local area individual indicator score < 50% of the adjusted level of performance for that indicator. Overall local-area single program score. Failure occurs when local area overall single program score < 90% of the adjusted level of performance for that indicator. Overall local-area single indicator score. Failure occurs when local area overall single indicator score < 90% of the adjusted level of performance for that indicator. The adjusted levels of performance will be used to determine performance success or failure. The threshold for performance failure is:
Statistical Adjustment This model is an objective regression model used to estimate levels of performance and adjusted levels of performance. Before the program year, the statistical adjustment model determines estimates that are used as a factor in the negotiations process. After the program year, the estimates derived from the statistical adjustment model are applied to the actual economic conditions and characteristics of participants served to determine the adjustment factor.
Factors In The Statistical Adjustment poor work history, lack of work experience, lack of educational or occupational skills attainment, dislocation from high-wage and high-benefit employment, low levels of literacy or English proficiency, disability status, homelessness, ex-offender status, and welfare dependency. The statistical adjustment model also takes into account other factors that, through empirical support, are determined to have an effect on predicting state outcomes. Economic conditions include differences in unemployment rates and job losses or gains in particular industries. Characteristics of participants include, but are not limited to, indicators of:
Example Variable: Adult Q2 Employment Variable type Variable Coefficient - state PY18 prior - state Participant characteristic Highest Grade Completed: Some College The effect (or weight) of the variable calculated at the state level. the last PY prior to negotiations. Variable estimate0 - state Variable estimate1 - local area PY18 actual - local area Adjustment 0.1033 0.3493 0.2727 3.61% 2.82% -0.79% The actual data for each variable for this state in The actual data for each variable for the local area in PY 2018. The amount the variable contributed to the pre-PY 2018 predicted target for this indicator as used as a factor in negotiations. It is calculated by multiplying the Coefficient by the PY Prior - state. The amount the variable contributed to the post- PY 2018 predicted target for this indicator using actual PY 2018 data. It is calculated by multiplying the Coefficient by the PY Actual - local area. -- Adjustment: The amount the variable contributed to the adjustment factor for the indicator. It is calculated by subtracting Variable estimate0 from Variable estimate1. 0.1033 * 02727 = 2.82% 0.3493 * 0.1033 = 3.61% 3.61% - 2.82% = -0.79%
Example Variables: Adult Q2 Employment Variable estimate0 - state Variable estimate1 - local area Variable type Participant characteristic Female Variable Coefficient - state PY18 prior - state PY18 actual - local area Adjustment 0.0751 0.5067 0.1818 3.81% 1.37% - Participant characteristic Age 25 to 44 0.0231 0.5493 0.5909 1.27% 1.36% 0.10% Participant characteristic Age 45 to 54 -0.0079 0.1493 0.1818 -0.12% -0.14% -0.03% Participant characteristic Age 60 or more -0.4201 0.0293 0.0455 -1.23% -1.91% -0.68% Add 46 additional variables TOTAL -3.01%
Example Variable: Adult Q2 Employment Performance indicator: Adult employment rate 2nd quarter after exit PY 2018 negotiated level of performance: Adjustment factor, accounting for actual participant characteristics and economic conditions during PY 2018: 78.00% -3.01% PY 2018 adjusted level of performance: 74.99%
Example Calculation: Individual Indicator PY 2018 negotiated level of performance 4 quarters PY 2018 % of goal based on PY 2018 adjusted level of performance PY 2018 actual level of performance - PY 2018 adjusted level of performance PY 2018 % of goal Performance indicator Employment rate 2nd quarter after exit Calculation 78.0% 81.8% 104.9% 74.9% 96.0% Based on statistical adjustment and negotiation with Nebraska Department of Labor (Numerator/denominator) = Actual Level of Performance (Actual level of performance/ negotiated level of performance) =% of goal United States Department of Labor statistical model applied (Actual level of performance/ adjusted level of performance) =% of goal Determining Success This number at a minimum must be over 50% This number at a minimum must be over 50%
Example Calculation: Overall Single Indicator (aggregate score) Adjusted Individual Indicator Score level of performance PY 2018 % of goal based on PY 2018 adjusted Credential attainment adult Credential attainment dislocated worker Credential attainment youth = Aggregate Score 92.3% 92.1% 75.1% This number at a minimum must be over 90% 86.5%
Example Calculation: Overall Program (aggregate score) Adult PY 2018 % of goal based on PY 2018 adjusted level of performance Adjusted Individual Indicator Score Employment Rate 2nd Quarter after Exit + Employment Rate 4th Quarter after Exit + Median Earnings 2nd Quarter after Exit + Credential Attainment + 96.0% 103.7% 112.4% 92.3% Measurable Skill Gains not available = Aggregate Score This number at a minimum must be over 90% 101.1%
Technical Assistance NDOL must provide technical assistance when a local area fails to meet adjusted levels of performance. Technical assistance provided by NDOL may include: assistance in the development of a performance improvement plan; development of a modified local or regional plan; or other actions designed to assist the local area in improving performance.
Current Mandatory Technical Assistance Greater Lincoln will receive mandatory technical assistance in relation to our failure on the: PY18 overall local-area single program score of 84.7 percent for its youth program, which falls below the 90 percent threshold; and PY18 overall local-area single indicator score of 86.5 percent for credential attainment, which falls below the 90 percent threshold.
Current Technical Assistance NDOL has identified the following areas of concern, technical assistance will be provided to support continuous improvement: youth credential attainment during PY 2018 and PY 2019; and adult employment at Q2 and Q4 during PY 2019.
Corrective Action If a local area fails to meet adjusted levels of performance for the same individual local-area single indicator for the same Title I program for a third consecutive program year, NDOL must take corrective actions, including the development of a reorganization plan under which the Governor: requires the appointment and certification of a new local board, prohibits the use of eligible service providers and one-stop partners that have been identified as achieving poor levels of performance; or takes such other significant actions as the Governor determines appropriate.
We are waning. Programs Year 2017, 2018, & 2019 did not meet negotiated levels of performance for youth credential attainment, nor did we meet the baseline for the aggregate youth program score.
Let's Look At Performance Meeting level of performance Area of concern Not meeting level of performance
Youth Program Performance PY 2018 % of goal based on PY 2018 adjusted level of performance 94.2% PY 2020 & 2021 negotiated level of performance 79.0% PY 2017 % of goal 93.2% PY 2019 % of goal 94.7% Performance indicator Employment rate 2nd quarter after exit Employment rate 4th quarter after exit Median earnings 2nd quarter after exit Credential attainment rate within 4 quarters after exit Measurable skill gains 83.5% 78.0% 84.8% 93.8% baseline $3,550 baseline baseline 31.5% 55.0% 75.1% 34.0% baseline 50.0% baseline baseline Aggregate score 69.4% 74.2% 84.7%
Adult Program Performance PY 2018 % of goal based on PY 2018 adjusted level of performance 96.0% PY 2020 & 2021 negotiated level of performance 79.0% PY 2017 % of goal 114.7% PY 2019 % of goal 82.9% Performance indicator Employment rate 2nd quarter after exit Employment rate 4th quarter after exit Median earnings 2nd quarter after exit Credential attainment rate within 4 quarters after exit Measurable skill gains 63.3% 103.7% 85.7% 79.0% 163.3% 112.4% 126.4% $7,749 71.4% 92.3% 147.5% 77.0% baseline baseline baseline 42.0% Aggregate score 103.2% 101.1% 110.6%
Dislocated Worker Program Performance PY 2018 % of goal based on PY 2018 adjusted level of performance 96.2% PY 2020 & 2021 negotiated level of performance 79.0% PY 2017 % of goal 93.4% PY 2019 % of goal 98.5% Performance indicator Employment rate 2nd quarter after exit Employment rate 4th quarter after exit Median earnings 2nd quarter after exit Credential attainment rate within 4 quarters after exit Measurable skill gains 99.4% 100.1% 96.1% 79.0% 126.0% 116.3% 111.3% $7,749 71.5% 92.1% 106.0% 77.0% baseline baseline baseline 42.0% Aggregate score 97.6% 101.2% 110.6%
Program Year 2020 will define our future. There is still time. Program year 2017 does not count against us. Program year 2019 adjustments have not been completed.
Federal & State Guidance United States Department of Labor: Employment and Training Guidance Letter No. 11-19 United States Department of Labor: Employment and Training Guidance Letter No. 10-16 United States Department of Labor: Employment and Training Guidance Letter No. 14-18 Nebraska Department of Labor: Performance Accountability, Change 2
