Review of Biological Systems Sciences Division - 2011 Report
2011 REPORT TO THE
BIOLOGICAL AND ENVIRONMENTAL ADVISORY COMMITTEE
(BERAC)
BY THE COMMITTEE OF VISITORS
FOR THE REVIEW OF THE
Biological Systems Sciences DIVISION
Daniel R. Bush
Chair of the COV
Chair, Department of Biology
Colorado State University
What was the COV’s charge?
1.
For DOE national laboratories and university grants, assess the efficacy and
quality of the processes used to:
–
Solicit, review, recommend and document proposal actions
–
Monitor active awards, projects and program
2.
Within the defined mission and funding of DOE, comment on:
–
The breadth ad depth of the portfolio elements
–
The national and international standing of the portfolio elements
3.
For the DOE Bioenergy Research Centers assess:
–
The divisions management and oversight of the science and operations,
including progress towards key scientific milestones ad deliverables
4.
For the JGI user facility assess:
–
The divisions management and oversight
–
Facility operations, tracking and review
–
User proposal solicitation, review and recommendation procedures
Who was on the COV?
The COV consisted of 17 scientists from around the country, with
representation from academia (9), the National Laboratories (4), and
other federal agencies (4). Nine of the COV members currently receive
DOE funding. Three of the COV members served on the prior BSSD
COV that met in June of 2008.
The COV met on 13 – 15 June 2011, at the DOE headquarters in
Germantown, Maryland. Assistance and support were provided, as
needed, by the BSSD staff.
The review was organized around 3 subcommittees – each assigned to review different
Programs or Projects of the overall BSSD research portfolio.
1
Genomic Science Program and Bioenergy Research Centers
Karen Cone, NSF;
Pam Green, U Delaware;
Cheryl Kuske, LSNL;
Frank Loeffler, ORNL;
Jocelyn Rose, Cornell;
Josh Schimel, UCSB;
Michael Thelen, LLNL
2
Low Dose Radiation, Radiochemistry & Imaging Instrumentation
Brad Barber, U Az;
David Boothman, UT SMC;
Helen Stone, NIH;
Wynn Volckert, U Missouri
3
Joint Genome Institute, Artificial Retina, Structural Biology
Andreas Andreou, JH;
Frank Collart, ANL;
Jack Okamuro, USDA-ARS;
Ward Smith, NIH;
Mike Sussman, UW
As the Chair, I rotated between the three groups
1.
For DOE national laboratories and university grants, assess the
efficacy and quality of the processes used to:
•
Solicit, review, recommend and document proposal actions
•
Monitor active awards, projects and program
For the review of the preproposals and proposals received in response to FOAs, the COV
was impressed with the overall quality and management of the review process. The PMs
should be commended for their role in implementing what the COV perceives to be a very
fair and equitable review process that uses the highest standards of the competitive
funding community to maintain a vigorous research portfolio. The funded programs
appear to have a good balance of risky, solid and innovative science.
Minor recommendations:
1)
Include greater clarity about the criteria for judging preproposals
2)
Use the genomic sciences program as a model for clear documentation of proposal
actions
2.
Within the defined mission and funding of DOE, comment on:
•
The breadth and depth of the portfolio elements
•
The national and international standing of the portfolio
elements
BSSD maintains a high visibility and vigorous research portfolio in a wide range of
research areas. BSSD programs are generally at the forefront of most areas because of
the
Division’s proactive outreach to the research community
. Likewise, many funded
investigators are international leaders in their fields.
Program priorities within BSSD are adjusted annually to respond to emerging research,
national priorities and technological needs.
Workshops are used as an effective mechanism for identifying emerging research areas
and directions for future research activities.
Many programs effectively use annual PI meetings, that frequently include outside
experts, as a useful management tool to examine progress on various BSSD projects and,
at the same time, get a snapshot of the hot topics and future direction in a given field.
3.
For the DOE Bioenergy Research Centers assess:
•
The divisions management and oversight of the science and
operations, including progress towards key scientific
milestones and deliverables
An extensive and well-documented process is in place for evaluating the BRCs and for
subsequently providing feedback and guidance. This ranges from frequent phone
conversations between the BRC directors or senior personnel and BSSD staff, to annual
reports and site visits. The documentation of the evaluation is impressive, well-structured
and clearly presented by the BSSD staff. Based on available reports, progress toward key
milestones seems on track. A detailed review of the scientific achievements to date was
beyond the scope of the COV (both in time and available documentation).
There appears to be excellent interaction between BSSD PMs and the internal management
of each BRC center, resulting in a generally rapid response by each of the BRCs to outside
feedback from PMs and the Annual Review Committee.
Recommendations for BRCs
The COV recommended that closer ties and openness be fostered between the BRCs to
avoid duplication. This would be greatly helped by including members of each center on
the advisory boards of the other two.
The COV recommended better documentation of research activities on public web sites
(to the extent possible), to inform the larger research community about research topics
to avoid broad duplication.
As the BRCs gear up for project renewals in 2012, it is essential that they continue to
address the mandate of original FOA: to pursue ‘high-risk, high-return’ approaches,
reflecting the value of developing large centers, rather than multiple small research
groups. Given the significance of this juncture for the BRCs, the COV recommends that
BSSD considers holding a workshop with key members from all the BRCs and select
members of the larger research community to discuss new directions in the bioenergy
arena
4.
For the JGI user facility assess:
•
The divisions management and oversight
•
Facility operations, tracking and review
•
User proposal solicitation, review and recommendation
procedures
The Joint Genome Institute (JGI) mission is to serve the diverse scientific community as a
user facility, enabling the application of large-scale genomics and transcriptomics analyses
of plants, microbes and communities of microbes in the context of DOE mission in the areas
of bioenergy and the environment.
The COV concluded the management and oversight are strong, and that operations and
user interactions are effective.
The rapid advances in sequencing hardware over the last few years, as well as the
development of sophisticated bio-informatics software, brings to the spotlight challenges
and opportunities for the JGI. Thus, the COV felt that there is an urgent need for JGI to be
proactive in adapting to a rapidly evolving technological and scientific landscape if it is to
maintain a leadership role as a premier genomics facility and a unique user resource.
Given the rapid changes in sequencing and associated bioinformatics, the COV
recommends that the JGI and the BSSD would benefit through the establishment of a
standing External Advisory Panel to provide continuous technical evaluation of the
strategic plan and advise the BSSD program staff in future planning and prioritization. A
workshop addressing Future Directions in the Genomic Sciences would help illuminate
the future JGI should aim for, and such a gathering would be a good place to identify
future members for this committee.
JGI should also consider expanding BSSD's partnership in feedstock genomics with the
five new USDA-ARS Biomass Research Centers (ARS BRCs). The ARS BRCs are using
“genotyping-by-sequencing” to genetically identify genes that control feedstock yield,
composition and biomass conversion efficiency to biofuels.
Scientific Focus Areas (SFAs)
Funding of research at the National Laboratories over the review period has changed from
funding individual, single investigator projects to integrated research programs focused on
collaborative research among several investigators and larger, interdisciplinary teams. The
COV reviewed this change because it also represents a change in BSSD program
development and management. Much of our review asks a series of questions for the PMs
to consider as they implement and manage this new approach.
•
What is the role of the PMs in overseeing all aspects of the SFAs since they are managed
by the National Laboratories?
•
Are there different criteria in terms of expectations, reporting and review for large versus
small SFA projects?
•
What is the PM’s role in requesting reports and providing feedback?
•
From the documentation provided, there seemed too little standardization across the
multiple program directors who are managing the SFA programs.
Thus,
the COV recommends
a standardized operating procedure be developed for all PMs
that manage SFAs.
•
How will the success of the SFA versus funding individual investigators (previous funding
system) be assessed? What metrics for success have been considered?
•
The COV recommends
that the program managers develop a plan with well-defined
criteria for assessing the impact of the SFA program.
Summary
Overall, the COV was very impressed with the processes used to solicit, review, recommend
and monitor the research portfolios within BSSD programs. Program managers (PMs)
adhered to the best practices of the competitive funding community while maintaining a
vigorous research portfolio.
BSSD PM used proactive strategies to identify the key research questions within their
programs and worked with the research community to determine the best approaches for
solving those questions. Funded projects include: 1) PIs with strong records of research
achievement, 2) a healthy mix of high risk, high yield objectives, 3) cutting edge approaches
applied to key research objectives, and 4) evidence of both national and international stature
with respect to overall research achievements.
Two cornerstone programs of BSSD, JGI and the BRCs, have strong records of achievement
and are very well managed by BSSD PMs. However, both JGI and the BRCs are facing key
junctures in preparing for future success. JGI will be challenged to stay at the leading edge
of sequencing technology and associated bioinformatics, and the BRCs will be challenged to
transition initial successes to more mature processes and, at the same time, continue to
develop novel, high risk, high yield approaches for advancing biofuels technology. The COV
endorses continued interaction with the larger research community to help BSSD manage
both programs.
Finally, the COV believes the PMs do an outstanding job running a large and complex program
with minimal administrative support.
The Committee of Visitors (COV) reviewed the Biological Systems Sciences Division in 2011, focusing on the efficacy of processes for soliciting, reviewing, and monitoring proposals, as well as assessing the DOE Bioenergy Research Centers and JGI user facility. The COV consisted of 17 scientists who evaluated various programs and projects within the research portfolio. Overall, the review highlighted the fair and quality-driven approach in managing proposals and active projects.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
2011 REPORT TO THE BIOLOGICAL AND ENVIRONMENTAL ADVISORY COMMITTEE (BERAC) BY THE COMMITTEE OF VISITORS FOR THE REVIEW OF THE BIOLOGICAL SYSTEMS SCIENCES DIVISION DANIEL R. BUSH CHAIR OF THE COV CHAIR, DEPARTMENT OF BIOLOGY COLORADO STATE UNIVERSITY
What was the COVs charge? 1. For DOE national laboratories and university grants, assess the efficacy and quality of the processes used to: Solicit, review, recommend and document proposal actions Monitor active awards, projects and program 2. Within the defined mission and funding of DOE, comment on: The breadth ad depth of the portfolio elements The national and international standing of the portfolio elements 3. For the DOE Bioenergy Research Centers assess: The divisions management and oversight of the science and operations, including progress towards key scientific milestones ad deliverables 4. For the JGI user facility assess: The divisions management and oversight Facility operations, tracking and review User proposal solicitation, review and recommendation procedures
Who was on the COV? The COV consisted of 17 scientists from around the country, with representation from academia (9), the National Laboratories (4), and other federal agencies (4). Nine of the COV members currently receive DOE funding. Three of the COV members served on the prior BSSD COV that met in June of 2008. The COV met on 13 15 June 2011, at the DOE headquarters in Germantown, Maryland. Assistance and support were provided, as needed, by the BSSD staff.
The review was organized around 3 subcommittees each assigned to review different Programs or Projects of the overall BSSD research portfolio. 1 Genomic Science Program and Bioenergy Research Centers Karen Cone, NSF; Pam Green, U Delaware; Cheryl Kuske, LSNL; Frank Loeffler, ORNL; Jocelyn Rose, Cornell; Josh Schimel, UCSB; Michael Thelen, LLNL 2 Low Dose Radiation, Radiochemistry & Imaging Instrumentation Brad Barber, U Az; David Boothman, UT SMC; Helen Stone, NIH; Wynn Volckert, U Missouri 3 Joint Genome Institute, Artificial Retina, Structural Biology Andreas Andreou, JH; Frank Collart, ANL; Jack Okamuro, USDA-ARS; Ward Smith, NIH; Mike Sussman, UW As the Chair, I rotated between the three groups
1. For DOE national laboratories and university grants, assess the efficacy and quality of the processes used to: Solicit, review, recommend and document proposal actions Monitor active awards, projects and program For the review of the preproposals and proposals received in response to FOAs, the COV was impressed with the overall quality and management of the review process. The PMs should be commended for their role in implementing what the COV perceives to be a very fair and equitable review process that uses the highest standards of the competitive funding community to maintain a vigorous research portfolio. The funded programs appear to have a good balance of risky, solid and innovative science. Minor recommendations: 1) Include greater clarity about the criteria for judging preproposals 2) Use the genomic sciences program as a model for clear documentation of proposal actions
2. Within the defined mission and funding of DOE, comment on: The breadth and depth of the portfolio elements The national and international standing of the portfolio elements BSSD maintains a high visibility and vigorous research portfolio in a wide range of research areas. BSSD programs are generally at the forefront of most areas because of the Division s proactive outreach to the research community. Likewise, many funded investigators are international leaders in their fields. Program priorities within BSSD are adjusted annually to respond to emerging research, national priorities and technological needs. Workshops are used as an effective mechanism for identifying emerging research areas and directions for future research activities. Many programs effectively use annual PI meetings, that frequently include outside experts, as a useful management tool to examine progress on various BSSD projects and, at the same time, get a snapshot of the hot topics and future direction in a given field.
3. For the DOE Bioenergy Research Centers assess: The divisions management and oversight of the science and operations, including progress towards key scientific milestones and deliverables An extensive and well-documented process is in place for evaluating the BRCs and for subsequently providing feedback and guidance. This ranges from frequent phone conversations between the BRC directors or senior personnel and BSSD staff, to annual reports and site visits. The documentation of the evaluation is impressive, well-structured and clearly presented by the BSSD staff. Based on available reports, progress toward key milestones seems on track. A detailed review of the scientific achievements to date was beyond the scope of the COV (both in time and available documentation). There appears to be excellent interaction between BSSD PMs and the internal management of each BRC center, resulting in a generally rapid response by each of the BRCs to outside feedback from PMs and the Annual Review Committee.
Recommendations for BRCs The COV recommended that closer ties and openness be fostered between the BRCs to avoid duplication. This would be greatly helped by including members of each center on the advisory boards of the other two. The COV recommended better documentation of research activities on public web sites (to the extent possible), to inform the larger research community about research topics to avoid broad duplication. As the BRCs gear up for project renewals in 2012, it is essential that they continue to address the mandate of original FOA: to pursue high-risk, high-return approaches, reflecting the value of developing large centers, rather than multiple small research groups. Given the significance of this juncture for the BRCs, the COV recommends that BSSD considers holding a workshop with key members from all the BRCs and select members of the larger research community to discuss new directions in the bioenergy arena
4. For the JGI user facility assess: The divisions management and oversight Facility operations, tracking and review User proposal solicitation, review and recommendation procedures The Joint Genome Institute (JGI) mission is to serve the diverse scientific community as a user facility, enabling the application of large-scale genomics and transcriptomics analyses of plants, microbes and communities of microbes in the context of DOE mission in the areas of bioenergy and the environment. The COV concluded the management and oversight are strong, and that operations and user interactions are effective. The rapid advances in sequencing hardware over the last few years, as well as the development of sophisticated bio-informatics software, brings to the spotlight challenges and opportunities for the JGI. Thus, the COV felt that there is an urgent need for JGI to be proactive in adapting to a rapidly evolving technological and scientific landscape if it is to maintain a leadership role as a premier genomics facility and a unique user resource.
Given the rapid changes in sequencing and associated bioinformatics, the COV recommends that the JGI and the BSSD would benefit through the establishment of a standing External Advisory Panel to provide continuous technical evaluation of the strategic plan and advise the BSSD program staff in future planning and prioritization. A workshop addressing Future Directions in the Genomic Sciences would help illuminate the future JGI should aim for, and such a gathering would be a good place to identify future members for this committee. JGI should also consider expanding BSSD's partnership in feedstock genomics with the five new USDA-ARS Biomass Research Centers (ARS BRCs). The ARS BRCs are using genotyping-by-sequencing to genetically identify genes that control feedstock yield, composition and biomass conversion efficiency to biofuels.
Scientific Focus Areas (SFAs) Funding of research at the National Laboratories over the review period has changed from funding individual, single investigator projects to integrated research programs focused on collaborative research among several investigators and larger, interdisciplinary teams. The COV reviewed this change because it also represents a change in BSSD program development and management. Much of our review asks a series of questions for the PMs to consider as they implement and manage this new approach. What is the role of the PMs in overseeing all aspects of the SFAs since they are managed by the National Laboratories? Are there different criteria in terms of expectations, reporting and review for large versus small SFA projects? What is the PM s role in requesting reports and providing feedback? From the documentation provided, there seemed too little standardization across the multiple program directors who are managing the SFA programs. Thus, the COV recommends a standardized operating procedure be developed for all PMs that manage SFAs. How will the success of the SFA versus funding individual investigators (previous funding system) be assessed? What metrics for success have been considered? The COV recommends that the program managers develop a plan with well-defined criteria for assessing the impact of the SFA program.
Summary Overall, the COV was very impressed with the processes used to solicit, review, recommend and monitor the research portfolios within BSSD programs. Program managers (PMs) adhered to the best practices of the competitive funding community while maintaining a vigorous research portfolio. BSSD PM used proactive strategies to identify the key research questions within their programs and worked with the research community to determine the best approaches for solving those questions. Funded projects include: 1) PIs with strong records of research achievement, 2) a healthy mix of high risk, high yield objectives, 3) cutting edge approaches applied to key research objectives, and 4) evidence of both national and international stature with respect to overall research achievements. Two cornerstone programs of BSSD, JGI and the BRCs, have strong records of achievement and are very well managed by BSSD PMs. However, both JGI and the BRCs are facing key junctures in preparing for future success. JGI will be challenged to stay at the leading edge of sequencing technology and associated bioinformatics, and the BRCs will be challenged to transition initial successes to more mature processes and, at the same time, continue to develop novel, high risk, high yield approaches for advancing biofuels technology. The COV endorses continued interaction with the larger research community to help BSSD manage both programs. Finally, the COV believes the PMs do an outstanding job running a large and complex program with minimal administrative support.
