Insights on MRCPsych Exams During COVID-19
Survey results on the impact of COVID-19 on MRCPsych exams show that most respondents found online delivery suitable, citing benefits like cost savings and a relaxed home environment. However, concerns about internet issues and invigilation challenges were raised, suggesting a need for improvements in the exam process.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Taking Exams during the COVID-19 Pandemic Survey opened: 05/06/2022 Survey closed: 27/06/2022 Audience: Trainees that have taken an MRCPsych since COVID Response rate: 281 (16.5%)
Background The Academy wrote to College CEOs last summer to seek agreement for them to send out this survey. After many exams moved to online/remote delivery during the pandemic, Colleges are reviewing whether these exams will remain online or revert back to a face-to-face format. The Equality, Diversity & Impact assessment, looks at how changes affected trainees. The short-life working group has agreed it would be best to run the survey during the first three weeks of June, once the GMC National Training Survey has closed. The survey is intended for all UK-based candidates who have taken a College/Faculty exam delivered across a different format to that used prior to the pandemic. The plan is for each College and Faculty to analyse its own results for any submission it wants to make to the GMC re any planned changes to exams and to consider implications for how it runs its exams and supports its trainee in future. Our short-life working group also plans to reconvene in the autumn to facilitate shared analysis of the survey findings to identify any cross-specialty commonalities and overarching themes/concerns.
Key findings Survey response rate was 16.5%. 86% of respondents felt the location was suitable. The main reason for the location not being suitable was internet issues Out of the 15 factors respondents were asked to comment on, online was considered better by the majority of respondents for 10 factors Some respondents highlighted concerns with this question in terms of their ability to compare if they had online sat online. The majority of respondents responded that the delivery method had no difference on their performance in relation to 15 protected characteristics. Of those that considered the delivery method to affect performance 12 characteristics scored more positively for online out of 15 characteristics. Positive themes included cost, home environment more relaxed, ability to take the exam despite illness, pregnancy, childcare, less stressful Negative themes include invigilator or examiner issues, IT and internet issues, home environment not suitable, screen time too long, risk of bias, lack of empathy via video 3% of respondents commented that the College was unable to meet their request for reasonable adjustments The majority of respondents commented don t know with regard to how online versus face-to-face invigilation compared Suggestions to improve the CASC included, use of online exam centres, return to face-to-face, continue online, increase frequency, more piloting, adapting cognitive and physical exam stations
"I don't understand the question - I did my exam online but surely you know this, by the fact I did it after September 2020! What are you asking here?" There was some confusion with this question (3) which we did indicate to AoMRC that there would be before sending the survey.
Themes why location was not suitable 17 responses No responses 86% of candidates felt the location was suitable. Time difference 1 Childcare 2 Internet issues 11 Noise from other participants 1 Hearing impairment 1 Uncomfortable environment 1
Based on your overall experience during the pandemic of exams in this location, how do the following factors compare to your previous experiences of exam delivery? Factor Better Same Worse Don't Know Convenience (travel time) 93.3% 1.7% 0.8% 4.2% Health and safety (social distancing, sanitation) 89.9% 5% 0.8% 4.2% Transport links 89.1% 4.2% 1.7% 5% Cost (parking, public, transport/ hotel) 83.2% 10.1% 2.5% 4.2% Parking availability 82.4% 3.4% 0.8% 13.4% Accessibility (location) 81.5% 6.7% 6.7% 5% Exam conditions (quiet, comfortable, free from distractions) 56.3% 19.3% 19.3% 5%
Based on your overall experience during the pandemic of exams in this location, how do the following factors compare to your previous experiences of exam delivery? Factor Better Same Worse Don't Know Invigilation (ability to obtain help if experiencing issues) 24.4% 26.9% 30.3% 18.5% Technical support (during exam) 18.5% 31.1% 28.6% 14.3% Information/support during exam (guidance/ instructions/time remaining) 26.9% 32.8% 26.1% 14.3% Quality of access to computer hardware (speed, monitor size/readability/font size/color) 34.5% 18.5% 23.5% 23.% Internet capability (speed/reliability) 31.9% 21% 21.8% 25.2% Ability to take a break (toilet, breastfeeding, religio us practices) 36.1% 32.8% 18.5% 12.6% Information/support pre-exam registration... 31.1% 45.4% 12.6% 10.9% Provision of reasonable adjustments 26.9% 26.9% 7.6% 38.7%
If you would like to expand on any of the points above and highlight any particular areas of concern of improvement, please include further comments here. Themes regarding online experience Number of comments Internet connectivity or hardware concerns online 11 Online more convenient 10 Face-to-face less stressful 4 Online less stressful 4 More difficult to assess clinical skills online 4 Problems with question in terms of ability to compare if you have only sat online 4 Noise problems online 1 F2F noise problem 1
I believe the change in delivery method of the examination(s) may have affected my performance (experience and/or outcome) due to my having the following protected or other characteristic(s). Positively Negatively No difference Age 16.5 9.6 73.9 Disability 9.7 6.2 84.1 Gender Reassignment 3.6 1.8 94.6 Marriage and civil partnership 10.6 2.7 86.4 Pregnancy/ Maternity 13.6 1.8 84.5 Race 6.3 9.9 83.8 Religion or belief 6.3 5.4 88.3 Sex 7.1 6.2 86.7 Sexual Orientation 2.7 0 97.3 Socio-economic status 17.3 8.2 74.5 Long-term physical and/or mental health condition 11 5.5 83.5 Ethnicity 6.3 9.9 83.8 Migrant Status 9 7.2 83.8 Prefer not to say 3.2 2.2 94.6 Other 4.3 5.4 90.2
Please can you explain why you believe you have been positive of negatively affected? For example, you may reflect on your experience and /or the outcome of your exam. NEGATIVE IMPACT THEMES. No of comments POSITIVE IMPACT THEMES. No of comments Online is more convenient and there is a cost benefit 9 Invigilator or examiner issues 3 IT or Internet issues 4 Home environment more relaxed 5 Home environment not suitable 2 Still able to take the exam despite illness 3 Screen time is too long 2 Better for pregnancy 1 Risk of bias 2 Online helped with childcare 1 Lack of empathy with video format 1 I did not feel discriminated online 1 Total 14 Online less stressful 1 Positively impacted 1 OTHER -THEMES. No of comments Total 21 Not positively or negatively impacted 4 Can't compare because have only sat online 5 Convenience versus technical challenges 1 Online is better for 1st language English speakers 1 Total 11
If negatively impacted what might have helped the situation Comments Generic background for all participants Recording and independent scoring to standardise the result Extending the validity period Online exam stations should be longer Role player dialects should be easier to understand for international candidates Passing should be adjusted by gender Sort out examiner connection problems Using bigger computer screen Offer facility with good internet connection to take the exam
What different or reasonable adjustments did you require? Comments No of comments Extra reading time 6 Quiet room and good internet connection 4 Good internet connection 5 Headphones and internet 1 Pregnant needed breaks 1 Camera and room set up 1 Access to glucometer and sweets 1 Larger font to read text 2 Technical support from the College 1 Upgrade to computer, webcam and earphones 1 Study format 1
What different or reasonable adjustments did you require? Comments No of comments Extra reading time 6 Quiet room and good internet connection 4 Good internet connection 5 Headphones and internet 1 Pregnant needed breaks 1 Camera and room set up 1 Access to glucometer and sweets 1 Larger font to read text 2 Technical support from the College 1 Upgrade to computer, webcam and earphones 1 Study format 1
Better Same/ Similar Worse NA or Don't Know In person/face-to-face with invigilator present 28.6 21 10.9 39.5 Remote exam delivery with invigilator present online 16.8 30.3 22.7 30.3 Remote exam delivery with remote proctoring using video or AI 12.6 21.8 22.7 42.9
Do you have any other suggestions to improve the use of remove and/or online exams? Comments No of comments Use online exam centres 7 Return to face-to-face exams 6 Continue online 4 Increase frequency 2 More piloting /mock online exams 2 Improve online cognitive and examination assessment 2 Use single document to display station resources 1 Restart station if technical issues 1 Examiners should be more aware of technical issues with role players 2 Provide distraction to reduce stress during the quarantine period 1
Did you take an RCPsych exam in or after September 2020? Paper A Paper B CASC 58% 48% 43% 57% 52% 42%
Which location did you use for your examination(s)? Paper A Paper B CASC 91% 58% 39% 47% 49%
Was your exam location suitable for the purposes of your exam delivery? Paper A Paper B CASC 14% 8% 9% 86% 92% 91%
