Prescribing Safety Assessment - Item Development Process Overview

 
Prescribing Safety Assessment
Training Workshop
 
Introduction to PSA item authoring
PWS, REV & CAL items
Dr Lynne Bollington
 
 
 
Agenda
 
PSA item development process
Creating new items
 
Item development is a multi-stage
process
 
5 stages of version control (V0–V4)
Only items at V0 & VR can be edited by authors
Items may be returned to VR for revision and resubmission at any stage
 
Copy editors ensure the
items are in house style
 
information is in the correct order
spelling is corrected
superfluous information is deleted
all abbreviations are from the approved list
reference ranges are consistent with house style
metadata are complete and correct
 
Clinical reviewer checks item content:
 
items on correct template
PMH matches DH
indications and doses match BNF
keyed answer matches BNF (if appropriate)
Identify potential problems in item construct
issues are fixed, or flagged for further consideration
 
Peer review for clinical
appropriateness:
 
reviewers come from various backgrounds and have differing expertise
check face validity
match contemporary practice
unambiguous and clear
appropriate difficulty
relevant to FY1
 
Assessment Board approves paper(s);
Standard Setting Group set pass mark:
 
Assessment Board:
approves paper build, including item anchoring, re-use etc
checks against assessment blueprint
confirms coverage of therapeutic domains and topics
checks item content
elaborates on mark schemes
reviews item usage post-hoc
SSG:
checks item content
considers difficulty of items
sets pass mark/item
 
Item progress can be
viewed by the
author in the
Dashboard
Feedback is visible at
all stages
 
V1
 
V2
 
V4
 
V3
 
V0
 
VR
 
DD
 
Agenda
 
PSA item development process
Creating new items
 
Adding new item
 
Side menu
expands to
show text
 
Prescribing items (PWS)
 
PWS items 
present a scenario where a prescription is
required
Reasoning and judgement
Deciding on the most appropriate prescription to write, based
on the clinical circumstances
Measurable action
Writing a safe, effective prescription for a single medicine to
tackle a specific indication or syndrome.
 
Starting a new item
 
Select a prescription chart
 
Write a case presentation
 
Edit the prescribing request as
appropriate
 
Suggest
 a marking scheme
 
Or select a drug set (if available)
Use the 
British National
Formulary
, to which the
candidates will have access
throughout, as the ultimate
arbiter of dose ranges, routes,
indications, contra-indications, etc.
 
Include justifications for optimal and
suboptimal answers
 
Remember to save the
mark scheme after
adding each drug
.
 
For IV fluids, indicate which volumes
and
 which rates are acceptable
 
System navigation help tools
 
In viewing mode:
 
In edit mode:
 
Item usage and performance data
 
This item was insufficiently focused
 
Fac: 98.6%
Dis: 0.084
 
 . . there were too many correct answers
 
Fac 98.6%
Dis: 0.084
 
Prescription Review items (REV)
 
REV items 
present a prescription for review, that was 
written
by someone else
 
Reasoning and judgement
Deciding which components of the prescription are
inappropriate, unsafe or ineffective
 
Identifying ADRs, interactions,
contraindications and dose errors
 
Only ONE dose error permitted - more than one is very unlikely in practice
 
Two questions, with 1-3 marks each
(max 4 marks in total)
 
If one of the questions is an interaction, include ONE of the
interacting drugs in the case presentation and lead-in so candidates
don’t have to pick both drugs from the list of medicines.
 
House style is important here
 
lower case,
generic name,
no strength
 
Select from drop-down lists that
appear when you begin to type
 
List between 6–10 medicines
 
Calculation skill items (CAL)
 
Competencies
Reasoning and judgement
Making an accurate drug calculation based on
numerical information
Measurable action
Recording the answer accurately with appropriate
units of measurement
 
All facts required to do the calculation
must
 be included
 
This item was too easy and did not
discriminate
 
Fac: 98.0%
Dis: 0.055
 
Complex, multi-step calculations tend to work best
Slide Note
Embed
Share

In the Prescribing Safety Assessment Workshop, item authoring is a crucial process involving multiple stages such as new item creation, editing, peer review, and final assessment board approval. Copy editors and clinical reviewers ensure the accuracy and quality of item content, while peer reviewers assess clinical appropriateness. The Standard Setting Group establishes pass marks, ensuring thorough coverage of therapeutic domains. This comprehensive process ensures the development of high-quality assessment items for prescribing safety.


Uploaded on Aug 06, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. Prescribing Safety Assessment Training Workshop Introduction to PSA item authoring PWS, REV & CAL items Dr Lynne Bollington

  2. Agenda PSA item development process Creating new items

  3. Item development is a multi-stage process New, V0 (returned VR) Submit, V1 Edit, V2 PRM, V3 AB & SSG, V4 Use 5 stages of version control (V0 V4) Only items at V0 & VR can be edited by authors Items may be returned to VR for revision and resubmission at any stage

  4. Copy editors ensure the items are in house style New, V0 Submit, V1 PRM, V3 AB & SSG, V4 Edit, V2 Use information is in the correct order spelling is corrected superfluous information is deleted all abbreviations are from the approved list reference ranges are consistent with house style metadata are complete and correct

  5. Clinical reviewer checks item content: New, V0 Submit, V1 PRM, V3 AB & SSG, V4 Edit, V2 Use items on correct template PMH matches DH indications and doses match BNF keyed answer matches BNF (if appropriate) Identify potential problems in item construct issues are fixed, or flagged for further consideration

  6. Peer review for clinical appropriateness: New, V0 Submit, V1 Edit, V2 PRM, V3 AB & SSG, V4 Use reviewers come from various backgrounds and have differing expertise check face validity match contemporary practice unambiguous and clear appropriate difficulty relevant to FY1

  7. Assessment Board approves paper(s); Standard Setting Group set pass mark: New, V0 V1 V2 Submit, Edit, PRM, V3 AB & SSG, V4 Use Assessment Board: approves paper build, including item anchoring, re-use etc checks against assessment blueprint confirms coverage of therapeutic domains and topics checks item content elaborates on mark schemes reviews item usage post-hoc SSG: checks item content considers difficulty of items sets pass mark/item

  8. Item progress can be viewed by the author in the Dashboard VR V1 V2 V3 Feedback is visible at all stages V4 V0 DD

  9. Agenda PSA item development process Creating new items

  10. Adding new item Side menu expands to show text

  11. Prescribing items (PWS) PWS items present a scenario where a prescription is required Reasoning and judgement Deciding on the most appropriate prescription to write, based on the clinical circumstances Measurable action Writing a safe, effective prescription for a single medicine to tackle a specific indication or syndrome.

  12. Starting a new item

  13. Select a prescription chart

  14. Write a case presentation

  15. Edit the prescribing request as appropriate

  16. Suggest a marking scheme

  17. Or select a drug set (if available)

  18. Use the British National Formulary, to which the candidates will have access throughout, as the ultimate arbiter of dose ranges, routes, indications, contra-indications, etc.

  19. Include justifications for optimal and suboptimal answers Remember to save the mark scheme after adding each drug.

  20. For IV fluids, indicate which volumes and which rates are acceptable

  21. System navigation help tools In viewing mode: In edit mode: delete copy edit

  22. Item usage and performance data

  23. Fac: 98.6% Dis: 0.084 New, V0 Submit, V1 PRM, V3 AB & SSG, V4 Edit, V2 Use This item was insufficiently focused

  24. Fac 98.6% Dis: 0.084 New, V0 Submit, V1 PRM, V3 AB & SSG, V4 Edit, V2 Use . . there were too many correct answers

  25. Prescription Review items (REV) REV items present a prescription for review, that was written by someone else Reasoning and judgement Deciding which components of the prescription are inappropriate, unsafe or ineffective

  26. Identifying ADRs, interactions, contraindications and dose errors Only ONE dose error permitted - more than one is very unlikely in practice

  27. Two questions, with 1-3 marks each (max 4 marks in total) If one of the questions is an interaction, include ONE of the interacting drugs in the case presentation and lead-in so candidates don t have to pick both drugs from the list of medicines.

  28. House style is important here List between 6 10 medicines lower case, generic name, no strength Select from drop-down lists that appear when you begin to type

  29. Calculation skill items (CAL) Competencies Reasoning and judgement Making an accurate drug calculation based on numerical information Measurable action Recording the answer accurately with appropriate units of measurement

  30. All facts required to do the calculation must be included

  31. Fac: 98.0% Dis: 0.055 New, V0 Submit, V1 PRM, V3 AB & SSG, V4 Edit, V2 Use This item was too easy and did not discriminate Complex, multi-step calculations tend to work best

Related


More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#