Transformation Through Reproducible Analytical Pipeline (RAP) at DfE
Discovery and scaling of RAP initiative at the Department for Education (DfE) to enhance statistical reporting, improve data dissemination efficiency, and meet user needs. The journey began with a focus on code effectiveness and reproducibility, leading to notable findings and recommendations for expanding RAP's use department-wide. Lessons learned emphasize the importance of cultural change, prioritizing resource allocation, and showcasing benefits for buy-in.
- Data Analysis
- Reproducible Research
- Department for Education
- Statistical Reporting
- Efficiency Improvement
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
RAP in DfE - discovery and how we scale Laura Selby (DfE)
RAP in DfE Started RAP journey almost 2 years ago Began with official statistics reporting, focussing mainly on using code more effectively to streamline the production of our statistics outputs and making them more reproducible Since then we ve been trialling and developing what RAP in DfE might look like, and moving production teams towards these methods Reproducibility has been adopted amongst secondary analysts too In early 2018 we carried out a Discovery which recommended we continue our use of RAP
Official Statistics Discovery Aim: To understand how and why our users want to consume data, how we can better meet those needs and how we can improve efficiency in delivering data. Noteworthy findings: Producers: Committed to increasing skills and finding new ways to disseminate data but frustrated with time pressures and amount of time spent on routine tasks. Additional requests for information are time consuming. Users: Frustrated by inconsistency between and within releases and the length of time it takes for statistics to be released.
Official Statistics Discovery Recommendation: That the existing Reproducible Analytical Pipeline (RAP) approach is built upon and it s use expanded throughout the Department. This will free up analysts time so they can focus on the quality of reports, provide more information in a consistent format, increase timeliness and improve the story we tell. The research has given us another lever to push the roll-out of RAP within DfE and to adopt this production process for all official statistics products.
Thoughts and Lessons Culture change takes time, even the smallest improvements in the right direction are worth celebrating Business as usual takes priority, protect allocated resource and/or iterate small improvements It s easier to get buy in when you can show the benefit The hard work comes first, tidy data doesn t always exist How do we still add value? Statisticians are there to provide insight, automation shouldn t get in the way Mixed profession teams aren t the norm, how do we make sure this is easy for statisticians to pick up?
Whats next? We ve made a lot of progress but we have some things that we re still working on - Discussions with Microsoft regarding reproducibility over time and dependency management Cementing RAP into DfE analytical infrastructure Post Discovery alpha work Building skills and setting the expectation
For more information Contacts Laura Selby: laura.selby@education.gov.uk Adam Robinson: adam.robinson@education.gov.uk Tom Franklin: thomas.franklin@education.gov.uk Sean Gibson: sean.gibson@education.gov.uk Further reading Our discovery findings: https://assets.publishing.service.gov.uk/government/uploads/sy stem/uploads/attachment_data/file/721729/HiveIT_- _DfE_dissemination_discovery.pdf
