Writing Effective MCQs - Marc J. Kahn, MD, MBA - Guidelines for MCQ Creation
Writing Effective MCQs
Marc J Kahn, MD, MBA
Peterman-Prosser Professor
Sr. Associate Dean
Tulane University School of Medicine
Disclosures
Nothing to disclose
There are no perfect
questions, only perfect
answers
MCQ Types
A through R and X
USMLE Step 1 only uses A-type, Step 2 also uses R-
type
A-type = single best answer MCQ, 4 or more
options
R-type = extended matching in sets of 2-20
K-type
Hyperleuckocytosis is typically associated with
(1)
infection
(2)
acute leukemia
(3)
chronic leukemia
(4)
baseball
A=1,2,3
B=1,3
C=2,4
D=4
E=all
Problem with multiple T/F
Totally Wrong
Totally Correct
1
2
3
4
Problem with multiple T/F
Totally Wrong
Totally Correct
1
3
4
2
Psychometrics
How precise is my test?
Kuder Richardson—tests reliability for measures with
dichotomous choices. Ranges 0.00 to 1.00
KR=[N/(N-1)][1-{M(N-M)}]/(N*V)N= number of items
M=arithmetic means of test scores
V= variance of raw scores
> 0.7 is good
Point Biserial
Measures correlation of individual question with
overall examinee performance
How good is my question at distinguishing high
from low performers?
r
pb
=M
1
-M
0
/s
n-1
* [n
1
n
0
/n(n-1)]^
1/2
Want POSITIVE value (-1.00 to 1.00)
>0.2 very good
TEST Time
MCQ rules
Focus on potentially catastrophic or common
problems
Assess application of knowledge not a factoid
Stem should be clear
All distractors should be homogenous
Avoid technical flaws
Anatomy of a MCQ
STEM
A.
Distractor
B.
Distractor
C.
Correct Answer
D.
Distractor
E.
Distractor
Lead in
Bad Stem
Beta blockers are associated with what common side
effect?
Better stem
A 67 year old woman complains of one month
duration of severe shortness of breath associated
with a dry cough. Her symptoms are worse with
exercise. She has been diagnosed with severe
glaucoma that is being treated with timolol,
dipivefrine, travaprost, and brimonidine eye drops.
She is also taking oral acetazolamide.
Which of her medications is most likely responsible
for her symptoms?
Other STEM problems
Too much information
Unnecessary data
Red herrings
Should all follow same format:
Age and gender
Complaint and duration
PMH
Physical
Labs
Lead-in problems
Multiple negatives
All of the following are not contraindications to beta
blockers except for:
Disguised TRUE/FALSE
Which of the following are true about his condition?
Avoid Factoids
GLIB = GLAB
A glib is purveyed
Which of the following are also purveyed:
A.
Binks
B.
Astons
C.
Glabs
D.
Wims
E.
Creys
What has been learned??
Answer Choices
The perils of “all” and
“none of the above”
A. mitochondria
B. endoplasmic reticulum
C. Nucleus
D. None of the above
E. All of the above
Schrodinger’s Cat
Dead
Alive
Or in between?
Bad Distractor
A.
red blood cell
B.
lymphocyte
C.
neutrophil
D.
eosinophil
E.
basophil
Universality
A.
Prevalence is highest in childhood
B.
Prevalence is highest in adulthood
C.
Prevalence is highest in the elderly
D.
Is autosomal dominant
E.
Prevalence is highest in women
Non-homogeneity
A.
review peripheral smear
B.
give IV glucose
C.
discuss end of life care
D.
Consult a cardiologist
E.
order a blood gas
Other key points
Perform psychometrics
Keep a secure item bank
Add and retire questions (~30% new questions per
year)
Share with peers for comment
Exam item Review Committee
Insure clinical relevance
It is better to give than to
receive
This presentation covers the essential elements of creating effective Multiple Choice Questions (MCQs) for educational assessments. It includes tips on question types, pitfalls to avoid, psychometrics, and MCQ rules, ensuring the development of high-quality assessment tools to evaluate knowledge and comprehension. The content delves into MCQ anatomy, types, and optimizing question construction for accurate assessment outcomes.
Uploaded on Feb 21, 2025 | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Writing Effective MCQs Marc J Kahn, MD, MBA Peterman-Prosser Professor Sr. Associate Dean Tulane University School of Medicine
Disclosures Nothing to disclose
There are no perfect questions, only perfect answers
MCQ Types A through R and X USMLE Step 1 only uses A-type, Step 2 also uses R- type A-type = single best answer MCQ, 4 or more options R-type = extended matching in sets of 2-20
K-type Hyperleuckocytosis is typically associated with (1) infection (2) acute leukemia (3) chronic leukemia (4) baseball A=1,2,3 B=1,3 C=2,4 D=4 E=all
Problem with multiple T/F 1 2 3 4 Totally Wrong Totally Correct
Problem with multiple T/F 1 3 4 2 Totally Wrong Totally Correct
Psychometrics How precise is my test? Kuder Richardson tests reliability for measures with dichotomous choices. Ranges 0.00 to 1.00 KR=[N/(N-1)][1-{M(N-M)}]/(N*V) N= number of items M=arithmetic means of test scores V= variance of raw scores > 0.7 is good
Point Biserial Measures correlation of individual question with overall examinee performance How good is my question at distinguishing high from low performers? rpb=M1-M0/sn-1 * [n1n0/n(n-1)]^1/2 Want POSITIVE value (-1.00 to 1.00) >0.2 very good
MCQ rules Focus on potentially catastrophic or common problems Assess application of knowledge not a factoid Stem should be clear All distractors should be homogenous Avoid technical flaws
Anatomy of a MCQ STEM Lead in A. Distractor B. Distractor C. Correct Answer D. Distractor E. Distractor
Bad Stem Beta blockers are associated with what common side effect?
Better stem A 67 year old woman complains of one month duration of severe shortness of breath associated with a dry cough. Her symptoms are worse with exercise. She has been diagnosed with severe glaucoma that is being treated with timolol, dipivefrine, travaprost, and brimonidine eye drops. She is also taking oral acetazolamide. Which of her medications is most likely responsible for her symptoms?
Other STEM problems Too much information Unnecessary data Red herrings Should all follow same format: Age and gender Complaint and duration PMH Physical Labs
Lead-in problems Multiple negatives All of the following are not contraindications to beta blockers except for: Disguised TRUE/FALSE Which of the following are true about his condition?
A glib is purveyed Which of the following are also purveyed: A. Binks B. Astons C. Glabs D. Wims E. Creys
The perils of all and none of the above A. mitochondria B. endoplasmic reticulum C. Nucleus D. None of the above E. All of the above
Schrodingers Cat Dead Alive Or in between?
Bad Distractor A. red blood cell B. lymphocyte C. neutrophil D. eosinophil E. basophil
Universality A. Prevalence is highest in childhood B. Prevalence is highest in adulthood C. Prevalence is highest in the elderly D. Is autosomal dominant E. Prevalence is highest in women
Non-homogeneity A. review peripheral smear B. give IV glucose C. discuss end of life care D. Consult a cardiologist E. order a blood gas
Other key points Perform psychometrics Keep a secure item bank Add and retire questions (~30% new questions per year) Share with peers for comment Exam item Review Committee Insure clinical relevance
It is better to give than to receive ???
