Effective DMP Review: Rubrics, Quality, and Support
Discover how to enhance your Data Management Plan (DMP) review process by using rubrics, ensuring quality, and leveraging institutional support. Learn from expert insights on what constitutes a good DMP and the significance of funder requirements. Improve your Research Data Management competencies through thoughtful DMP assessments and find out how institutions are offering assistance in this crucial area.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
How to review DMPs (using review rubrics) Myriam Mertens Open Access Week Belgium, 20 October 2020
Research funders initiated DMP requirements OECD Declaration on Access to Research Data from Public Funding BBSRC H2020 ERC BELSPO BOF Wellcome Trust FWO NWO NERC MRC NIH NSF ZonMW DMPs as part of grant applications, and/or post-award
How about DMP quality? Often broad, open-ended questions Perception that poor DMP quality has few consequences can undermine policies Some funders assess DMPs, others outsource DMP evaluation "BUREAUCRACY"byLina Kusaite, licensed underCC BY-NC-ND 4.0
https://www.zonmw.nl/en/research-and-results/fair-data-and-data-management/data-management-in-your-project/https://www.zonmw.nl/en/research-and-results/fair-data-and-data-management/data-management-in-your-project/
Institutions stepping up to offer support Recruiting RDM support staff RDM advocacy & training Developing DMP feedback services
(P)reviewing DMPs Requires significant effort Requires right knowledge & skills specialist input sometimes required Also an opportunity to improve your own RDM competences!
What does a good DMP look like? No absolute right answers Look for evidence that researcher has given proper thought to issues Some things to pay attention to: Plan is appropriate? (e.g. relevant standards, alignment with disciplinary norms, use of (local) infrastructure/support services) Feasible to implement? Sufficiently detailed information? Choices made are properly justified? Advice has been sought where needed? From Sarah Jones (2016), Reviewing Data Management Plans. CC-BY
Tip: use a DMP review rubric Concept initially developed in DART project (US) Analytic rubric to standardise the review of DMPs Matrix with performance criteria + performance levels
Identifies a number of performance levels Criteria for evaluating DMPs: - each criterion evaluates only 1 aspect - matched to funder guidelines Describes all performance levels for each criterion Concrete examples can also be added to descriptions https://osf.io/gwvqb
For example We will use suitable formats to ensure that our data can be preserved and sustained over the long term. PERFORMANCE LEVELS Complete/detailed Addressed issue, but incomplete/too vague Did not address issue PERFORMANCE CRITERIA Clearly describes file formats in which data will be stored, and explains why. Only partially or vaguely describes file formats used and/or the rationale. Does not describe file formats used. Describes what file formats will be used for the data
For example We will use suitable formats to ensure that our data can be preserved and sustained over the long term. PERFORMANCE LEVELS Complete/detailed Addressed issue, but incomplete/too vague Did not address issue PERFORMANCE CRITERIA Clearly describes file formats in which data will be stored, and explains why. Only partially or vaguely describes file formats used and/or the rationale. Does not describe file formats used. Describes what file formats will be used for the data Format(s) not named Better: We will store data in .csv and .txt files, which are open formats and therefore ensure sustainable access.
Getting started Best 1 rubric per DMP template Guide on how to develop a rubric: https://osf.io/gwvqb Publicly available rubrics from UGent (CC0): FWO DMP rubric BELSPO DMP rubric Horizon 2020 FAIR DMP rubric ERC DMP rubric
Some final remarks Review rubrics may also be used for self-assessment Collective DMP review practice is a good training exercise Review rubrics may also be used to analyse DMPs and inform development of (institutional) RDM services for which topics are performance levels are high/low? what services could be offered to target areas of lower performance?
Thanks! Any questions?
