Roles and Responsibilities of Study Section Chair and Meeting Participants
Understanding the distinct roles of study section chairs and meeting participants is crucial for effective research proposal evaluation. The study section chair leads discussions and implements policies, while meeting participants, including the Scientific Review Officer (SRO) and reviewers, play complementary roles in evaluating proposals. Discussions are essential for panel members to make informed judgments about proposals, focusing on research significance, feasibility of approaches, and team qualifications. Key questions are answered through discussions, aiding in decision-making processes.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
- Study Section Chair
- Meeting Participants
- Proposal Evaluation
- Research Significance
- Team Qualifications
Presentation Transcript
Preparing to Chair Bruce Reed, PhD Deputy Director, CSR Summer 2023
Additional Preparation Shift roles o Being study section chair is quite different from serving as a reviewer o Switch from reviewing applications to leading discussions about applications o Contemplate your role; Get clear on the main things you will be trying to accomplish Consider this opportunity to change things o How well do your study section practices align with today s talk? Work in partnership with the SRO o Review concerns with your SRO. 22
The complementary roles of meeting participants SRO Chair Every Reviewer NIH review policy & practice expert Teams with Chair to implement policies and promote good practices Independently selects & assigns reviewers Trains reviewers Takes notes and writes the resumes Leads discussion Summarizes discussion Teams with SRO to implement policies and promote good practices On assigned applications o Writes critiques o Orally informs panel about the application and the basis for their score o Engages in discussion On every application o Listens o Questions/comments as needed o Judges the merits of the application o Votes 33
Why discuss at all? Review outcomes are committee decisions. Panels add valuable additional perspectives on the significance of the research and strength of the approach. o Everyone votes and all votes are weighted equally o Every vote should reflect an informed, independent judgment Oral critiques: should inform the panel of the basis for that reviewer s score Discussions allow o Assigned reviewers to explore or explain their differences o Clarifications o Challenges o Adding new (score driving) considerations o Score calibration A good discussion gives panel members the information they need to make an informed judgment about the proposal. Clarity, not consensus, is the goal. 44
Get key questions answered When a discussion concludes you (and the panel) should understand: 1. How important is the proposed research? a) Evaluating the importance of the science is hard, but vital. b) Reviewers confuse public health need with scientific importance. c) A clear message that significance is lacking can be a favor to everyone. 2. How rigorous and feasible is the approach? a) Don t nit pick. b) Don t require certainty can they overcome challenges. c) Do require rigorous and feasible methods. 3. Is the team well suited to the science and are the needed resources available? a) An area where bias tarnishes NIH review. b) Don t confuse reputation with qualifications. 55
Set the tone Lively, respectful engagement of the entire panel enables good discussions (and keeps things interesting) Maintain a collegial atmosphere where every voice matters Encourage disagreement, Don t tolerate disrespect. Beware of differences in power--Expect good manners Clarity, rather than consensus. Nobody has to win, nobody has to concede. Be neutral, be fair and cutoff discussions that are repetitive or nonproductive. Intervene if you hear bias. Model good critiques. Limit your reviewer comments while chairing 66
Listen for bias Bias is most apparent in comments on investigators and environment. Big names may get a pass; unknowns may get extra scrutiny. Evaluations should be specific and pertinent to the proposed science. Discourage general remarks on reputation. E.g. This application comes from a world- renowned expert .leading authority .thought leader . Discourage empty comments on environment: It s from ------- so obviously the environment is excellent 77
Cues to step in There is silence after assigned reviewers present Major differences in assigned reviewers views are unexplored e.g. one says significance is high, one says its low Comments do not seem to justify the scores Differences in scores are not explained Score drivers are clear but the discussion carries on Reviewers are having an interesting scientific discussion that is not informing the scores Differences are clear but the reviewers are trying to win an argument or achieve consensus Reviewers are in the weeds 88
How to summarize Summaries should focus the panel s attention on the score driving points. A good summary should Be concise (a minute or two) Summarize score driving strengths and weaknesses Address Significance Note areas of consensus and points of difference Be balanced and fair A good summary does not Spend more than a sentence describing the grant List every little point made Summaries are not a record of the discussion Summarize the written critiques SROs are taking detailed notes and are solely responsible for writing the resume of discussion. 99
Read applications and critiques in advance? CSR does not expect or recommend that you read every application, or every set of abstracts and aims. Ditto for critiques Read enough to be comfortable with what will be discussed. Assigned reviewers should be explaining the basis for their score to the panel. If you don t understand it, others likely don t, either. 1010
Keeping on Time Good time management is a matter of fairness Problems with time often reflect more fundamental problems, e.g. o Critiques that are overly descriptive, excessively detailed, o A focus on methodological minutiae o Repetitive or circular discussions o A failure to focus on what matters Well presented critiques and good discussions cover a lot of ground in 15 minutes. Be flexible, but keep on track Set expectations and enforce them. 1111
Communicate with your SRO! Before, during and after the meeting At the premeeting meeting o Process: Anything need attention? e.g. score compression, time management o Policy/Practice: Anything new? o People: ESA, tech support, new reviewers, ECRs o Technology: how will you communicate in the meeting. What s your backup plan? After the meeting o Debrief: What went well, what didn t? o Feedback: on ad hoc reviewers to SRO o Plan ahead: Are there items that need attention going forward? 1212
To be effective 1. 2. 3. 4. Lead discussions. Keep discussions focused and productive. Aim to answer key questions Set the tone, protect a culture of respect Keep it on time (be flexible, but monitor and adjust early) Communicate with the SRO And don t make it harder than it needs to be 1. Clear the deck, no multi-tasking 2. Use a cheat sheet 3. Have your in-meeting technology plan (and backup plan) set 1313
Thanks for your service! bruce.reed@nih.gov 14
