Statewide Evaluation of Service Providers at American Job Centers

The Statewide Evaluation of Service Providers at American Job Centers aims to assess the performance and capacity of service providers in delivering career and training services to WIOA Title 1 participants. The evaluation focuses on identifying strengths, weaknesses, and gaps in service provision, reaching all mandated individuals, and providing recommendations for improvement. Methodologies include literature review, field visits, staff interviews, and a detailed evaluation rubric covering various aspects of service provider management.

• Evaluation
• Service Providers
• American Job Centers
• WIOA
• Training Services

cicilia Follow

Uploaded on Sep 12, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Statewide Evaluation of Service Providers at American Job Centers Beyond Information. Intelligence.

2. Objectives To measure the level of performance in the service providers capacity to provide career and training services for WIOA Title 1 participants in accordance with Federal, State, and Local policies To identify strengths and weaknesses, and gaps in the service providers capacity to provide career and training services in accordance with Federal, State, and Local policies To measure the extent to which the service providers are able to reach to all of the Title 1 individuals mandated in the WIOA law To have a set of practical recommendations that the State Workforce can follow-up on to improve on the Title 1 service providers capacity to provide career and training services To research and identify organizations that meet WIOA Title 1 requirements to qualify as future service providers Statewide Evaluation of Service Providers at AJCs October 2021 2 Beyond Information. Intelligence.

3. Literature review of WIOA law, AJC reports, County Plans Field visits to AJC offices and service providers Hawai i July 21 O ahu August 10 Maui August 23 Kaua i August 25 Interviewed staff of service providers in-person and communicated through video conferencing and email with the respective WDB County Executive Director. Methodology Inspected work environment, file storage areas, public access areas. Reviewed case files (1 randomly selected file from each program of the service providers) Online survey for case managers 12 respondents (responses received from all service providers) Statewide Evaluation of Service Providers at AJCs October 2021 3 Beyond Information. Intelligence.

4. Evaluation Methodology: Rubric Scoring Financial Management Information Management Program Management HR Management Knowledge of WIOA program Individuals with barriers to employment Communication structure and procedure Monitoring and evaluation Continuous process improvement Established system of collaboration Adequate staffing Average staff tenure Seeks out and participates in professional learning opportunities Formal policies and procedures are in place Accounting and reporting conform with federal standards Internal regulatory process Written policy and procedures Timeliness in data entry Data integrity Security of case files Accuracy of data Systems and processes in the use of data Leadership Regulatory Compliance COVID-19 Response Outreach Capacity Outreach Effectiveness Effective Communication Works outstandingly with staff in formulating goals Involves all staff in achieving performance outcomes Responsiveness in planning and delivering the County plan Obligations covered under the MoU Outreach strategies Technology readiness Remote work capacity Methods and strategies Frequency Increase number of successful enrollments Statewide Evaluation of Service Providers at AJCs October 2021 4 Beyond Information. Intelligence.

5. Evaluation Rubric Example 1 Below 2 Approaching 3 Meets Expectations 4 Exceeds 0 1 2 3 4 HR Management 2.3 Financial Management 3.0 Information Management 1.5 Program Management 2.6 Leadership 3.7 Regulatory Compliance 1.9 COVID-19 Response 2.7 Statewide Evaluation of Service Providers at AJCs October 2021 5 Beyond Information. Intelligence.

6. - Youth: Goodwill - Adult and DW: Goodwill Hawai i Statewide Evaluation of Service Providers at AJCs October 2021 6 Beyond Information. Intelligence.

7. Telling them we will follow-up on them for 12 months after they exit the program during Orientation has made the difference. Strengths Ability to partner, leverage resources to stretch funding Goodwill Youth (Hawaii) 0 1 2 3 4 HR Management 2.7 Gaps HireNet has not been utilized for performance tracking because of access issues ; uses an internal tracking system During audit of case files, 3 errors were found Not co-located in the AJC Financial Management 3.7 Information Management 2.5 Program Management 3.0 Leadership 2.7 Regulatory Compliance 3.0 COVID-19 Response 2.5 Statewide Evaluation of Service Providers at AJCs October 2021 7 Beyond Information. Intelligence.

8. Outreach Goodwill Youth (Hawaii) Strengths Extensive community collaboration and outreach 0 1 2 3 4 Gaps Concerns regarding support on Kona side; staff feels overstretched Capacity 3.5 Effectiveness 2.5 Statewide Evaluation of Service Providers at AJCs October 2021 8 Beyond Information. Intelligence.

9. We were already anticipating the State shutting down, so we started transitioning by purchasing the needed equipment. Strengths Staff is tech-savvy (able to digitized application forms and utilized HireNet s e-signature) Active one-stop partner Awareness of local business needs (e.g., created a database of organizations and companies Goodwill Adult & DW (Hawaii) 0 1 2 3 4 HR Management 2.3 Financial Management 3.7 Gaps Staffing vacancies HireNet has not been utilized for performance tracking and closing out cases; developed its own internal tracking system Short on ETPs and the process for certification leads to delays to be more responsive to industry needs Information Management 2.7 Program Management 2.7 Leadership 3.0 Regulatory Compliance 2.3 COVID-19 Response 3.5 Statewide Evaluation of Service Providers at AJCs October 2021 9 Beyond Information. Intelligence.

10. Outreach Strengths Extensive community collaboration and outreach Meet once a month to discuss outreach and business engagement Goodwill (Adult & DW) 0.0 1.0 2.0 3.0 4.0 Capacity 3.0 Gaps Collaboration with other AJC partners (e.g., Rapid Response) is a work in progress. Struggled with DW enrollments in 2020 because people were more concerned with their employment insurance Effectiveness 3.5 Statewide Evaluation of Service Providers at AJCs October 2021 10 Beyond Information. Intelligence.

11. - Youth: WorkHawaii - Adult and DW: WorkHawai i Oahu Statewide Evaluation of Service Providers at AJCs October 2021 11 Beyond Information. Intelligence.

12. My case managers will say its [HireNet] another database because they have to input on three databases. Strengths Co-located with the Adult and DW programs to provide flexibility in serving young adults with multiple options Encourage youth to design program activities and utilize their input to improve services (e.g., Leadership Day) Dedicated room for locked cabinets securing the case files WorkHawaii Youth (Oahu) 0 1 2 3 4 HR Management 2.7 Financial Management 2.7 Information Management 2.8 Program Management 3.3 Gaps Transition to virtual platform was challenging for the case managers Cumbersome for case managers to manage three databases Leadership 3.3 Regulatory Compliance 3.3 COVID-19 Response 2.5 Statewide Evaluation of Service Providers at AJCs October 2021 12 Beyond Information. Intelligence.

13. Outreach Strengths Use of social media to keep the youth engaged Established partnerships with organizations to offer community service projects that are meaningful to the youth Utilizes a joint case management approach to maximize opportunities for co-enrollment and leverage of funding resources WorkHawaii Youth (Oahu) 0 1 2 3 4 Capacity 3.5 Effectiveness 3.0 Gaps More space that accommodate social distancing Statewide Evaluation of Service Providers at AJCs October 2021 13 Beyond Information. Intelligence.

14. The dual-enrollment process is very critical for us because if that particular job seeker has a sustainable income, all partners gets a positive outcome. Strengths Dedicated room for locked cabinets securing the case files Ability to partner and stretch funds by leveraging resources Ability to capitalize on HireNet s reporting features WorkHawaii Adult & DW (Oahu) 0 1 2 3 4 HR Management 3.0 Financial Management 3.3 Gaps Staff are not technologically-savvy, challenged during COVID-19 remote work Staff feels there is no teamwork Needs better coordination and integration of services by co-located partners Information Management 3.0 Program Management 3.3 Leadership 2.7 Regulatory Compliance 3.0 COVID-19 Response 2.0 Statewide Evaluation of Service Providers at AJCs October 2021 14 Beyond Information. Intelligence.

15. Outreach Strengths Strong referral practice Leverages staff relationship with Micronesian and Filipino communities Utilizes a joint case management approach to maximize opportunities for co-enrollment and leverage of funding resources WorkHawaii - Adult & DW (Oahu) 0 1 2 3 4 Capacity 3.5 Gaps Rigorous eligibility process to access WIOA funds, some opt to get trainings through other routes Teamwork Hawaii as the outreach and recruitment arm of WorkHawaii (potential risk since they don t do their own outreach) Effectiveness 3.0 Statewide Evaluation of Service Providers at AJCs October 2021 15 Beyond Information. Intelligence.

16. - Youth: UH Maui College - Adult and DW: Goodwill Maui Statewide Evaluation of Service Providers at AJCs October 2021 16 Beyond Information. Intelligence.

17. I started to look at business engagement because work experience numbers were low. Strengths Strong ties to the community (e.g., non-profit, veterans) Fiscal and admin reports were not prepared properly so these are now the responsibility of another entity within the college Maui Youth 0 1 2 3 4 HR Management 2.3 Gaps Insufficient staffing to meet the needs of the program, one person does everything. No filing system Located separately from other service providers, but co- located after the evaluation Should not use interns to conduct check-ins or case management of participants is a breach of data integrity Financial Management 1.3 Information Management 1.3 Program Management 2.3 Leadership 2.7 Regulatory Compliance 2.0 COVID-19 Response 2.5 Statewide Evaluation of Service Providers at AJCs October 2021 17 Beyond Information. Intelligence.

18. Outreach Strengths Strong ties to the community Has a triple enrolled client (Adult, Youth, and McKinley) Guested on TV (Akaku) to promote the program UH Maui Youth (Maui) 0 1 2 3 4 Capacity 2.5 Gaps No funding for in-school youth, only out-of-school Staff shortage Effectiveness 2.5 Statewide Evaluation of Service Providers at AJCs October 2021 18 Beyond Information. Intelligence.

19. Goodwill being a statewide program, my peers (from other counties) and I take turn in holding virtual job readiness training courses. Strengths Utilizes HireNet for outcome reports and follow-ups Aware of local business needs and strong relationship with employers Institutional ability to conduct statewide virtual job readiness training courses Goodwill Adult & DW (Maui) 0 1 2 3 4 HR Management 3.0 Financial Management 3.3 Information Management 2.7 Program Management 2.8 Gaps Missing supporting documents in the case files Audit file forms are submitted externally to Goodwill s QA team on Oahu Utilizes an internal tracking system separate from HireNet Leadership 3.0 Regulatory Compliance 2.7 COVID-19 Response 3.0 Statewide Evaluation of Service Providers at AJCs October 2021 19 Beyond Information. Intelligence.

20. Outreach Strengths Has a triple enrolled client (Adult, Youth, and McKinley) Participated in the Maui county virtual job fair Has utilized the AJC video, radio plugs (KISS FM, KPOA), and appeared on TV (Akaku) to promote the program Goodwill - Adult & DW (Maui) 0 1 2 3 4 Capacity 3.5 Gaps New location has less foot traffic Outreach to Molokai and Lanai Effectiveness 4.0 Statewide Evaluation of Service Providers at AJCs October 2021 20 Beyond Information. Intelligence.

21. - Youth: Hale Opio - Adult and DW: WDD Kaua i Statewide Evaluation of Service Providers at AJCs October 2021 21 Beyond Information. Intelligence.

22. We dont have much to show on HireNet. We did a lot but dont have evidence. Strengths Youth engagement continued during Covid. Has the ability to deliver services virtually Files secured on a separate drive with password protection just for the program Hale Opio Youth (Kauai) 0 1 2 3 4 Gaps Not familiar with goals and priorities of WIOA, no proper handover from previous service provider Scared of enrolling someone and then being called for doing something wrong so they decided to be cautious and ensure the program processes were in place. WDB stopped meeting and monthly check-ins by Oahu program didn t work out due to scheduling HR Management 2.3 Financial Management 2.3 Information Management 2.0 Program Management 1.7 Leadership 2.0 Regulatory Compliance 1.8 COVID-19 Response 2.5 Statewide Evaluation of Service Providers at AJCs October 2021 22 Beyond Information. Intelligence.

23. Outreach Hale Opio Youth (Kauai) Strengths Utilized enrollment and outreach from existing program to find qualified clients 0 1 2 3 4 Capacity 2.5 Gaps Understanding WIOA and its processes took up time Existing program participants have difficulty qualifying for WIOA enrollment Effectiveness 1.0 Statewide Evaluation of Service Providers at AJCs October 2021 23 Beyond Information. Intelligence.

24. If the partners were more available in the center (in person), then maybe we could have serviced better. Strengths Runs reports using HireNet Offers services both in-person and online WDD Adult & DW (Kauai) 0 1 2 3 4 HR Management 2.7 Financial Management 2.7 Gaps Partners are not located at the AJC No formal process to check work of the case manager; several errors found in the case files Information Management 2.0 Program Management 1.8 Leadership 2.0 Regulatory Compliance 2.0 COVID-19 Response 2.0 Statewide Evaluation of Service Providers at AJCs October 2021 24 Beyond Information. Intelligence.

25. Outreach Strengths Awareness of local business needs Active one-stop partner and support WDD - Adult & DW (Kauai) 0 1 2 3 4 Gaps Limited funding restricts broad outreach Partners are not located at the AJC Under enrollment causes classes to be cancelled Capacity 2.5 Effectiveness 2.0 Statewide Evaluation of Service Providers at AJCs October 2021 25 Beyond Information. Intelligence.

26. Recommendations Statewide Evaluation of Service Providers at AJCs October 2021 26 Beyond Information. Intelligence.

27. Hawaii Work with the State Workforce to identify options within HireNet that could be use as an internal tracking system Goodwill s data security policy to be shared with the State Workforce Set up meetings with Rapid Response to better coordinate Resolve understanding of ETP certification process Oahu Regular staff meetings emphasizing objectives Improve communication and formal coordination process with AJC co-located partners Build partnership with organizations that offer other career pathways instead of competing Maui Work with the State Workforce to identify options within HireNet that could be use as an internal tracking system Goodwill s data security policy to be shared with the State Workforce Find opportunities to advertise WIOA programs in the new location Leverage existing university relationships and resources for outreach Kauai Training for case managers on how to utilize the different features of the database to run reports Youth program to set up office at the AJC Update the outreach strategy and work with schools (e.g., use of social media, etc.) Statewide Evaluation of Service Providers at AJCs October 2021 27 Beyond Information. Intelligence.

28. State Workforce To maximize effective use of Hirenet among service providers, create a video tutorial as part of the onboarding process of new case managers A written manual developed by service provider and Executive Director to facilitate transition procedures and maintain standards and accountability Reassess the funding formula of WIOA to reflect the needs of the county s population and the lack of services in each county Organize training on how to pursue exemptions of federal, state, and county documents for WIOA qualification Statewide Evaluation of Service Providers at AJCs October 2021 28 Beyond Information. Intelligence.

29. Other Organizations Identified Statewide Evaluation of Service Providers at AJCs October 2021 29 Beyond Information. Intelligence.

30. Oahu Maui Hawaii Kauai Family Programs Hawaii Residential Youth Services and Empowerment Goodwill Hawaii CNHA CNHA Maui Hui Malama CNHA Goodwill CNHA Statewide Evaluation of Service Providers at AJCs October 2021 30 Beyond Information. Intelligence.

31. Mahalo! Beyond Information. Intelligence.