Review of DOE Office of Workforce Development for Scientists
Assessment of the 2016 review of the Department of Energy's Office of Workforce Development for Teachers and Scientists (WDTS) including its mission, programs reviewed, charges, and additional considerations to enhance workforce development in STEM fields.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Report to BESAC on the 2016 COV Review of the DOE Office of Workforce Development for Teachers and Scientists (WDTS) Gordon E. Brown, Jr. Stanford University and SLAC National Accelerator Laboratory February 24, 2017 Rockville, MD
Mission of the Office of Workforce Development for Teachers and Scientists (WDTS) to help develop the next generation of scientists and engineers to support Department missions, administer its programs, and conduct research that will realize the nation s science and innovation agenda (from the DOE Strategic Plan, May 2011) WDTS was established in 1990 with the mandate that The Energy Secretary is authorized to establish programs to enhance the quality of [STEM] education. Mission of WDTS continues to be vitally important as major demographic changes in workforce will impact the DOE complex over the next 10-15 years. Key retirements and overturn of the workforce in skilled positions in government Labs will take place. Efforts to recruit and train future workforce must reflect changes in ethnic composition of the workforce, which will vary substantially by region of the country.
2016 WDTS COV Charge The 2016 WDTS COV was charged by BESAC Chair John C. Hemminger with assessing the (1)efficacy and quality of processes used to solicit, review, recommend, monitor, and document application, proposal, and award actions, and (2) quality of resulting portfolio, including its breadth, depth, and national and international standing in the following National Lab-based programs:
WDTS Programs Reviewed (1) Science Program (786 participants in FY 16) Undergraduate Laboratory Internship (SULI) (2) Community College Internship (CCI) Program (88 participants in FY 16) (3) Visiting Faculty Program (VFP) (107 participants in FY 16) (4) Office of Science Graduate Student Research (SCGSR) Program (88 participants in FY 16) Not Reviewed: National Science Bowl (NSB), Einstein Fellows, Lindau Awards
Additional Charges to the WDTS COV 2016 WDTS COV also asked by Dr. Hemminger to comment on (1) Changes since the 2010 COV review (2) new directions of WDTS, including development and implementation of an integrated on-line application, review, management system (WDTS Application and Review System - WARS), and evaluation, and data (3) new operational baselines established for the four Laboratory-based WDTS programs
Background Information The four Lab-based programs account for ~70% of WDTS annual budget (~$20M) and involve ~1,100 participants annually. Systematic data needed to evaluate effectiveness of these four programs currently being organized in digital form using WARS. Thus, 2016 COV review primarily a processevaluation rather than an outcomeevaluation .
2016 WDTS COV Members Dr. Gordon E. Brown, Jr. (Chair), D.W. Kirby Professor of Geological Sciences in the School of Earth, Energy & Environmental Sciences, Stanford University, Stanford, CA; Professor of Photon Science, SLAC National Accelerator Laboratory, Menlo Park, CA Dr. Vicki H. Grassian, Distinguished Professor and Distinguished Chair of Physical Chemistry, Departments of Chemistry & Biochemistry, Nanoengineering, and Scripps Institution of Oceanography and Associate Dean, Division of Physical Sciences, U.C. San Diego, La Jolla, CA Dr. Stephen T. Pratt, Senior Chemist and Leader of the Gas-Phase Chemical Dynamics Group, Chemical Sciences and Engineering Division, Argonne National Laboratory, Argonne, IL Dr. Nancy L. Ross, Professor and Head of the Department of Geosciences, Virginia Tech, Blacksburg, VA Dr. Nan N. Sauer, Associate Director for Chemistry, Life, and Earth Sciences, Los Alamos National Laboratory, Los Alamos, NM
WDTS Staff Present During 2016 COV Review Dr. James Glownia Director of the Office of WDTS Dr. Julie Carruthers Senior Science and Technology Advisor, Office of the Deputy Director for Science Programs Dr. Ping Ge SCGSR Program Manager Dr. Karen Skubal WDTS Program Manager Dr. Jan Tyler NSB and Einstein Fellows Program Manager Ms. Karen Talamini (Program Analyst) Ms. Yolanda White (Program Coordinator)
Summary of Changes in WDTS Programs Since 2010 COV Review I Most recommendations suggested by 2010 COV implemented. The following four WDTS student and educator programs were judged to be weak by the 2010 COV: Academies Creating Teacher Scientists (ACTS) Pre-Service Teachers (PST) Real World Design Challenge (RWDC) Undergraduate Research Journal, and College Guide 2010 WDTS COV recommended eliminating four programs listed above and their funding redirected to the following strongly rated WDTS programs: Science Undergraduate Laboratory Internship (SULI) Community College Internship (CCI) National Science Bowl (NSB) Einstein Fellows Lindau Awards Faculty and Student Teams (FaST) program revamped following 2010 COV review into Visiting Faculty Program (VFP).
Summary of Changes in WDTS Programs Since 2010 COV Review II Office of Science Graduate Fellows (SCGF) Program eliminated in 2009 and replaced by SC Graduate Student Research Program in response to OMB recommendations. After one year of operation, the new SCGSR program judged Very Good by the 2010 COV. Following strong recommendation of 2010 COV, WDTS leadership team expanded by addition of three Ph.D.-level Program Managers. Development of close relationships and monthly teleconferences with LEDs - an important improvement that will unify standards and improve "best practices" for the programs across Laboratory system. Development of WDTS Application and Review System (WARS), an online management system - excellent addition to WDTS and to the DOE. One WDTS goal was to bring administration and review of WDTS programs more in line with the rest of the Office of Science programs. This goal has now been largely met.
Participants in Virtual Town Hall Meeting with National Laboratory Education Directors Meridith Bruozas Argonne National Laboratory Linda Lung National Renewable Energy Laboratory Julie Malicoat Oak Ridge Institute for Science and Engineering Spencer Pasero Fermi National Accelerator Laboratory Scott Robbins Los Alamos National Laboratory Ken White Brookhaven National Laboratory All five 2016 WDTS COV members
Summary of Virtual Town Hall Meeting with National Laboratory Education Directors I All LEDs expressed strong support for WARS software system, particularly it s ability to track applicants and alumni of WDTS programs. Good interaction among LED s and between LED s and WDTS staff. The COV believes LEDs and WDTS staff view themselves as part of the same team. Optimism about direction of WDTS programs and WDTS leadership, particularly that of Director Glownia - a notable improvement since 2010 COV review. Increasing coordination among DOE National Labs - already having impact on determining best practices, unifying procedures, etc.
Summary of Virtual Town Hall Meeting with National Laboratory Education Directors II General consensus that five of six Laboratories interviewed have more capacity for SULI and VFP participants. Number and willingness of high-quality mentors are not limitations for these programs. LEDs suggested development of a SULI-like program for early Ph.D./M.S. students. LEDs suggested that term length of VFP participants (3 summers) be increased. COV feels that an energetic group of LEDs has been assembled, and monthly meetings with WDTS staff members are clearly effective.
Summaries of the Missions of the WDTS Laboratory-Based Programs Science Undergraduate Laboratory Internship (SULI) Program provides undergraduate students from 2- and 4-year undergraduate institutions with paid internships in science and engineering activities within DOE National Labs. Internships are intended to encourage students to pursue careers in STEM- related areas.($8.3M in FY 16) Community College Internship (CCI) Program provides students from community colleges with opportunities for technical internships in science and engineering at DOE National Labs. ($1.0M in FY 16) Visiting Faculty Program (VFP) designed to increase research competitiveness of faculty members/students from educational institutions historically underrepresented in the research community in order to expand the workforce that addresses DOE mission areas ($1.7M in FY 16) Office of Science Graduate Student Research (SCGSR) Program provides outstanding graduate students with opportunities to conduct research in collaboration with DOE Lab scientists and engineers in areas central to Office of Science mission. ($2.5M in FY 16)
Summary of 2016 COV Recommendations for the Science Undergraduate Laboratory Internship (SULI) Program COV recommends that WDTS carry out a detailed evaluation of SULI program to determine if desired outcome of encouraging students to pursue STEM careers is being achieved.Effort facilitated by contacting National Lab mentors of SULI undergrads to find out their career paths. WARS data management system will help. COV recommends increase in number of SULI internships, consistent with capacity of National Lab staff to mentor increased number of undergrad interns. LEDs indicated additional capacity at each of their Labs for SULI students. COV recommends development of SULI-like program for early Ph.D./M.S. students combined with existing SULI program. COV also recommends the name of expanded program be changed to Science Laboratory Internship (SLI). COV recommends that all National Lab staff involved in evaluation process of SULI applicants be required to view an appropriate webinar on implicit bias.
Summary of 2016 COV Recommendations for the Community College Intern (CCI) Program COV strongly recommends that WDTS leadership, in consultation with National Laboratory Education Directors, seek additional funding to grow this program. CCI program viewed by COV as one means of increasing ethnically diverse, technologically savvy workforce in the DOE complex because CCI participants have higher degree of diversity than the national science and engineering undergraduate population. COV recommends that Labs increase outreach to local community colleges as a means of increasing number of under-represented minorities in program.
Summary of 2016 COV Recommendations for the Visiting Faculty Program (VFP) COV recommends that WDTS and LED s develop and implement more active outreach programs to bring VFP to the attention of more potential participants. This objective is particularly important because the VFP is designed to attract faculty and students from institutions historically underrepresented in the research community and thus could result in an increase in ethnic diversity within the science and engineering workforce. COV recommends an increase in breadth and impact of program as well as in number of program participants.Can successful VFP faculty be used as ambassadors (with stipends) to help expand the program? COV recommends that students accompanying VFP faculty to Labs should be offered career advice and mentoring to keep them engaged in Lab activities. COV urges WDTS to consider suggestion by LEDs that outcomes of VFP could be enhanced by extending allowable term of appointments (3 summers).Such an extension could limit total number of faculty and student participants in program. Further discussion between WDTS and LED s needed about possible extension needed.
Summary of 2016 COV Recommendations for the Science Graduate Student Research (SCGSR) I COV recommends extending SCGSR program period to better match typical Ph.D. timescale (3-5 years).Program should consider alternatives that allow grad students in good standing with strong support from Labs to automatically be eligible for renewal for up to 3 years. COV recommends that SCGSR program assess potential impact of multi-year awards and increase the program budget to accommodate some multi-year awards. COV recommends that SCGSR evaluate interest and potential impact of including M.S. degree students from targeted disciplines, particularly engineering and computer science.
Summary of 2016 COV Recommendations for the Science Graduate Student Research (SCGSR) II COV recommends additional structuring of SCGSR program that would result in a more standardized approach to identifying National Lab mentors for graduate student researchers. COV recommends that National Lab scientists and engineers who mentored and collaborated with SCGSR grad students provide information on their professional activities following receipt of M.S. or Ph.D., including place of employment. This information is critical for building a database on SCGSR participants for evaluating effectiveness of SCGSR program.
Comparison of the 2010 and 2016 COV Ratings for the Laboratory-Based WDTS Programs 2010 COV Rating Excellent Good Fair* Very Good** 2016 COV Rating Excellent Very Good Very Good Excellent SULI CCI VFP SCGSR Following 2010 COV review, Faculty and Student Teams (FaST) Program morphed into Visiting Faculty Program (VFP). ** At 2010 COV review, this was the SC Graduate Fellowship Program. 2016 rankings indicate that the processes used to solicit, review, recommend, monitor, and document application, proposal, and award actions in the SULI and SCGSR programs and the quality of the resulting program portfolios are judged to be Excellent. The CCI and VFP programs could be improved in various ways as indicated by the 2016 COV recommendations.
New Directions for WDTS New project begun with ORISE in 2012 to develop a WDTS-wide applications and data-management system to replace a system managed by PNNL. Resulting WDTS Application and Review System (WARS) online management system. Developed in collaboration with ORISE over the past five years at a cost of ~$3M, based on program logic models that utilize program goals and descriptive elements that define program success factors and supports the collection and archiving of applicant information, mentor information, participant information, and outcome data into a long- term database for tracking, assessment, and evaluation of WDTS programs. WARS provides extensive automated data analytics and visualization capabilities Saves significant staff time in analysis of data. 2016 COV considers development of WARS as a major achievement of WDTS over the past five years and recognizes Dr. James Glownia as the driving force and brains behind WARS.
Success in Establishing New Operational Baselines for the Four Laboratory-Based WDTS Programs Additional reviews of WDTS occurred in a 2012 Reverse Site Visit and a 2015-2016 Peer Review involving all participating Labs. Purpose of 2012 review - ensure that common core requirements employed by all National Labs for WDTS Lab programs. Outcome of 2012 Reverse Site Visit - identified areas where common core requirements are needed including application process, applicant review and selection, program execution, tracking to monitor participant progress, mentor selection and preparation and monitoring, WDTS funding and maintenance of financial records. Purpose of 2015-2016 Peer Review - evaluate whether SULI, CCI, and VFP are managed and executed so that program participants receive best-in-class experiences that increase their preparedness for a STEM career. Outcome of 2015-2016 Peer Review generally very positive for all National Labs participating in WDTS programs.
Additional COV Recommendations COV recognizes constraints that limit increases in diversity among WDTS program participants. Can address this goal by increasing diversity in applicant pools. Could accomplish this goal by increasing awareness of WDTS programs at historically minority colleges, or smaller colleges in neighboring urban communities. COV questions if efforts are sufficient to significantly increase ethnic and gender diversity of future science and engineering workforce and if efforts can be made to engage students from underrepresented minorities at an earlier stage of their career (e.g., high school). We challenge WDTS management and the DOE Office of Science to develop a more integrated, holistic national program that would involve the DOE Office of Science, the National Science Foundation, and other STEM-related federal agencies that would be charged with developing a more ethnically diverse science and engineering workforce by addressing the pipeline problem at an earlier stage in STEM education.
Additional COV Recommendations WDTS encouraged to leverage outreach efforts with workforce development and outreach activities at the Labs. Closely coordinate with Laboratory Education Directors, who expressed very strong satisfaction with the WDTS management team and programs as part of the 2016 COV review. COV recommends that the Equipment Donation Program be better advertised at National Labs and universities and colleges, particularly institutions historically underrepresented in research community.
Final Comments and Acknowledgments 2016 COV thanks the WDTS staff and Dr. Jim Glownia in particular for their commitment to these important programs, their hospitality and informative presentations during the COV review, and their patience with the many interruptions by COV members during their presentations for questions and comments. The two days of the COV review left no doubt in our minds about the commitment of the WDTS staff to their programs. They care deeply about their program participants and radiate an impressive level of enthusiasm. While this is true for all of the programs, the National Science Bowl seemed to be the pinnacle, bringing everyone together to enjoy the products of their efforts. Thanks also to outgoing BESAC Chair John Hemminger for his tireless and effective leadership of BESAC for the past 13 years and for his leadership role over the years in many COV reviews, including ours. My Stanford colleague Persis Drell has a tough act to follow.
