ARC College of Experts Assessment Process Overview
Detailed explanation of the assessment process at ARC College of Experts for proposals, including the role of general and detailed assessors, assignment procedures, assessment timeline through Selection Advisory Committee, and key considerations in funding recommendations.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
ARC College of Experts: assessment process ARC College of Experts: assessment process Amanda Nettelbeck Amanda Nettelbeck ARC College general assessors Carriage 1 and Carriage 2 Detailed (expert) assessors Minimum 2, average 3-4 50% combined weighting Raw scores A-E will be normalised 50% combined weighting Scores A-E + detailed reports Speak to both general and detailed (expert) assessors: they carry equal weighting. General assessors have broad expertise and read many proposals; detailed assessors have depth of expertise and read a small number of proposals. Highlight compelling reasons to fund. Follow Instructions to Applicants to the letter. Aims, scope/framework, significance and innovation, benefits, etc need to be immediately clear and readily understood. Ensure benefits are robust and are tied back to Communication of results . ROPE: research interruptions matter and receive due consideration.
Proposals go to different Selection Advisory Committees across the College of Experts:
Process of assigning assessors ARC Exec Director assigns each proposal to 2 general assessors with broad expertise (Carriage 1 & 2 = 50% combined). Carriage 1 has principal responsibly for shepherding proposal through assessment process. Carriage 1 (or Exec Director) assigns proposal to detailed assessors, including reserves (acceptance minimum 2, average 3-4 = 50% combined) Institutional conflicts of interest are automatically excluded. All assessors declare any other professional or personal conflict of interest. Applicants can request up to 3 people not to assess . You can exclude potential assessors by name, but not by anonymous assessor code from a previous submission. Resubmitted proposals and first-round proposals are treated the same: the assessment process begins from scratch. Word cloud helps to identify detailed assessors generated from RMS database by application title & summary, impact statement, FoR/SEO codes. Scholars you ve requested to exclude will not appear as possible assessors for your proposal. If you re an assessor, ensure your RMS profile is up to date and relevant to research interests & capacity so that the cloud can read your expertise.
Assessment timeline through Selection Advisory Committee Assessment timeline through Selection Advisory Committee All assessors (general and detailed) assess independently. Carriages 1 & 2 submit preliminary scores on RMS. Carriages 1 & 2 receive detailed assessor reports/rejoinder as single package. Co- Carriages discuss before finalising own scores. ARC creates preliminary ranking of all proposals based on total assessor submissions. Carriage 1 & 2 scores are normalised. Proposal rankings are not fixed but are subject to discussion at SAC panel meeting. Everyone on SAC panel has access to all proposals (excluding conflicts of interest). Discussion includes attention to reports, rejoinders, disparate scores/comments. Rankings can move accordingly. Rejoinders are read closely and are an important part of the assessment process. On basis of discussion, SAC makes funding recommendations, including on budget. For more detail, see https://www.arc.gov.au/peer-review/roles
Some common misconceptions One critical assessor can scuttle a proposal . The co-Carriages discuss and weigh critical assessments against other detailed reports, their own assessments, and the rejoinder. The ARC also flags disparate scores for the attention of the SAC panel. The rejoinder is a waste of time because decisions are already cast . The co-Carriages receive the rejoinder at the same time as the reports, so hold no preconceptions from reports. Rejoinders are discussed by the SAC panel and they matter. Don t use space to repeat good comments or deride bad ones: address queries/criticisms as fully as possible in a neutral tone. Only applicants who ve already got prior funding will be successful . Project quality and investigator/capacity are weighted separately, and ROPE concerns are considered in detail. All project budgets will be cut xx% / lower-ranked proposals will be cut more than higher- ranked ones . No proposal is subject to random cuts based on fixed percentage or ranking. Each budget is individually considered for feasibility, and expenses that aren t well justified are likely to be cut on grounds of value for money. Budget padding will be recognised and cut.
