Effective Feedback Strategies for Prime Evaluators in Internal Medicine

How to Be a PRIME Evaluator
Written comments and grading in IM
Susan Merel, MD
Carmelita Mason-Richardson
Describe the PRIME rubric and how IM uses it
in grading
Distinguish between Reporter-, Interpreter-,
and Manager-level students in your written
comments
Learning Objectives
 
Used by student to help improve
Record of our evaluation of the student
Used in MSPE, department letter, your own
personal LOR
In IM, can be used in determining final grade
How are written comments
helpful?
Helpful?
I was delighted to supervise Margo Chen for a week on the
General Medicine service at UWMC. She demonstrated an
exceptional rapport with her patients, spending countless hours
with them before and after rounds, getting to know them as
individual people rather than merely as patients. Her patients
adored her, and knew that they could depend on her for virtually
anything they might need. We expect our medical students to
pay attention to details as well as the "big picture," but Margo
was truly special in this regard. I believe that her clinical work
before medical school has provided her with an exceptional
sense of compassion, and a need to "connect" with her patients.
This is a rare gift, and she should be commended . . .
Helpful? (part 1, grade HP)
. . . Margo's medical knowledge was right on target, and she was passionate
about learning more every day. For example, she cared for a patient who
struggled valiantly against a severe case of macrophage activation syndrome;
she provided us with a lovely chalk talk about the condition, based on reading
she had done and discussions with our consultants. Our medical resident and
intern simply adored her and came to rely upon her for her meticulous care.
Her "rounding skills" were on target for the beginning of her third year. I do
anticipate that her confidence will rise as she gains experience; she is a
skillful student and she should let that show, so to speak. I am sure she will
become a wonderful physician, and I wish we had more time together.
Helpful? (part 2)
Helpful? (grade HP)
Joe is professional in his behavior and is responsive to feedback
regarding the advancement of his clinical skills. Over the course
of our two weeks together his reporting skills improved steadily,
he was able write great ambulatory notes and give a concise and
well organized oral case presentation by the end of time our
together. Joe is able to integrate information well and formulate
reasonable assessments. He continues to work on development
of his management skills. Joe has great communication skills and
interpersonal skills, he made my patients feel well-listened too.
PRIME
Professionalism
Timeliness, honesty, attitude
Reporter
Gathering info, writeups and oral presentations
Interpreter
Weighted differential, tests
Manager
Offers reasonable dx and tx plan
Enhanced communication
First described in the literature by Lou Pangaro in
1999
Intention was to improve the credibility of educators’
descriptive evaluations of students
Meant to provide a synthetic approach to the
evaluation of skills, knowledge and attitudes with
particular attention paid to professionalism.
Growing independence is the underlying premise
PRIME (RIME)
Pangaro L Acad Med 74(11) 1999; Pangaro L Clin Anat 19:419-28 2006
Reporter
: “
basic skills to obtain a a hx and do a PE
and the basic knowledge to know what to look for
;”
day-to-day reliability
. . .”
Interpreter
: ability to prioritize problems identified,
and offer a ddx. Has to be an active participant in
patient care.
Manager
: Selecting options and proposing plan;
Requires “
more knowledge, more confidence and
more judgement
.” Requires higher-level
interpersonal skills to tailor plan to patient.
According to Dr. Pangaro . . .
Pangaro L Acad Med 74(11) 1999
Is PRIME a pyramid?
PROFESSIONALISM
REPORTER
INTERPRETER
MANAGER
ENHANCED
COMMUNICATION
Sort of!
Professionalism and
reporter necessary for
other skills –
BUT students can have
manager and interpreter
skills in some contexts
and not others
 
Oral case presentations and notes are concise,
complete
Information presented is accurate
Can modify presentation to fit setting
Notes change appropriately;
minimal/appropriate use of copy-paste and
EMR importing
Important elements of strong
reporter-level student
 
Strong knowledge base
Differential that is reasonable 
and 
weighted
and
 supported
Able to change differential as clinical course
evolves 
and 
support changes
Ability to interpret testing – is a lab change
clinically significant?
Important elements of strong
interpreter-level student
Ability to prioritize problems in the inpatient
and outpatient settings
Able to assess status of chronic disease
“visit script” for outpt problems
Able to “zoom in or out” if there are issues
more important than the presenting problem
“Why” or differential within the differential
E.g. yes the patient is dyspneic from a PE, but why does
this healthy young person have a PE?
Important elements of strong
interpreter-level student
 
Can defend the plan to the team and explain
it to a patient
Takes initiative – follow up on tests, call for
records
Triage new problems, take appropriate steps
Develop a patient-centered management plan
– incorporate goals and values into plan,
especially on-the-fly as a patient questions a
plan
Important elements of strong
manager-level student
 
Strong knowledge base
Ability to choose/order appropriate testing
(not just order everything!)
Accurate, nuanced management plan that
changes over time – e.g get patient ready for
discharge, length of antibiotics, monitoring,
counseling of side effects
Important elements of strong
manager-level student
If students need lots of reminders, coaching, help
then they probably are not showing meaningful
independence
This is the biggest determinant of what level they
attain
For all levels of PRIME:
Meaningful Independence
RARELY there will be a student who is truly
managing at a high level but has reporting
deficits (this is not common and usually due
to anxiety with oral presentations and/or
trouble expressing themselves in writing)
The grade for this student should be high
pass, to reflect that they are truly managing
but need to improve some reporting skills
Student who truly manages but
has some reporting deficits
1.
Do
’er
“Checks off boxes”
2.
T
riager
Developing clinical intuition/”sick not sick”
“Recognizes when boxes need to change”
3.
E
xecutive
Suggests and starts to execute new plan of care
Defends plan to team
Engages with pt in shared decision-making about plan
Three aspects of management:
Adapted from Goren E and Kogan J, CDIM national meeting poster 2014
“Snow Day” – if you can’t make it in, can you
trust the student to get a safe plan going?
This can be a tough area to give feedback in
Not just any management plan . . .
Written feedback using PRIME
Joseph was an absolute joy to work with closely for one
week as a subintern on a busy hospitalist service. He
was working very solidly at the Manager level on the
PRIME system with excellent Professionalism and
Enhanced Communication skills. He was extremely
capable and thoughtful, able to see patients
independently, come up with a fairly nuanced
treatment plan and carry it out with appropriate
supervision.
Written feedback using PRIME
He had excellent clinical judgment given his level of
training. He truly excelled in communication with
patients and families, even able to run a family
conference alone when I was called away to an
emergency - when I came back, he had developed
excellent rapport with the family and had obtained a
very relevant extended social history and started a
helpful conversation about the patients' goals of care. 
Jane quickly formulates organized oral case
presentations and presented these in the room well.
Jane’s interpretation skills are quite good, she offers a
differential diagnosis for new patient issues and makes
accurate assessments of patients' chronic medical
conditions. She is beginning to develop her
management skills - thinking about what labs and
studies are indicated and follow-up on these.
(areas of strength)
Written comments using PRIME
Jane and I discussed continuing to work on efficiency in
history collection and managing her time with the
patient in the room, mastering her reporting skills. She
is working on further developing her management
skills, she is beginning to take more ownership of he
patient during and following the visit, I would
encourage her to continue to do this.
(areas needing improvement)
Written comments using PRIME
PRIME in grading in IM
Reporter = Pass
Interpreter = High Pass
Manager with Professionalism and Enhanced
Communication = Honors
Should be at a level consistently (>75%)
WRITE preceptors asked to fill out eval twice
We do not use the benchmarks in grading – they
are for feedback.
Please choose the final grade based on the PRIME
rubric
Average WRITE grade counts as 50% of clinical
grade
Grading for IM portion of WRITE
Clinical GPA (50% WRITE, 50% inpatient)
NBME IM exam score converted into a
number that is added to the clinical grade
Grading for IM clerkship
If clinical GPA is near a final grade cut-off, test
score above or below 77 can raise  or lower
grade one level
Test scores
 
below 75 are ineligible for a final
grade of "Honors".
Test scores 30 or below are a "Fail"
score. Students who fail the NBME exam will
fail the clerkship.
Effect of NBME on final grade
Meets quarterly
All IM site directors, all associate directors
involved with core clerkship
Reviews all students completing clerkship;
spends more time on students near a cutoff
Decisions made by consensus (sometimes
vote)
Rely heavily on comments
IM Grading Committee
Honors 38%
High Pass 41%
Pass 21%
IM grade distribution 2018-19
 
Questions?
Slide Note
Embed
Share

Learn how to provide constructive feedback using the Prime rubric in Internal Medicine evaluations. Understand the differences between Reporter, Interpreter, and Manager-level students in written comments. Discover the importance of detailed feedback in helping students improve and its role in assessing final grades.

  • Feedback
  • Prime Rubric
  • Internal Medicine
  • Evaluation
  • Student Assessment

Uploaded on Sep 29, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. How to Be a PRIME Evaluator Written comments and grading in IM Susan Merel, MD Carmelita Mason-Richardson

  2. Learning Objectives Describe the PRIME rubric and how IM uses it in grading Distinguish between Reporter-, Interpreter-, and Manager-level students in your written comments

  3. How are written comments helpful? Used by student to help improve Record of our evaluation of the student Used in MSPE, department letter, your own personal LOR In IM, can be used in determining final grade

  4. Helpful?

  5. Helpful? (part 1, grade HP) I was delighted to supervise Margo Chen for a week on the General Medicine service at UWMC. She demonstrated an exceptional rapport with her patients, spending countless hours with them before and after rounds, getting to know them as individual people rather than merely as patients. Her patients adored her, and knew that they could depend on her for virtually anything they might need. We expect our medical students to pay attention to details as well as the "big picture," but Margo was truly special in this regard. I believe that her clinical work before medical school has provided her with an exceptional sense of compassion, and a need to "connect" with her patients. This is a rare gift, and she should be commended . . .

  6. Helpful? (part 2) . . . Margo's medical knowledge was right on target, and she was passionate about learning more every day. For example, she cared for a patient who struggled valiantly against a severe case of macrophage activation syndrome; she provided us with a lovely chalk talk about the condition, based on reading she had done and discussions with our consultants. Our medical resident and intern simply adored her and came to rely upon her for her meticulous care. Her "rounding skills" were on target for the beginning of her third year. I do anticipate that her confidence will rise as she gains experience; she is a skillful student and she should let that show, so to speak. I am sure she will become a wonderful physician, and I wish we had more time together.

  7. Helpful? (grade HP) Joe is professional in his behavior and is responsive to feedback regarding the advancement of his clinical skills. Over the course of our two weeks together his reporting skills improved steadily, he was able write great ambulatory notes and give a concise and well organized oral case presentation by the end of time our together. Joe is able to integrate information well and formulate reasonable assessments. He continues to work on development of his management skills. Joe has great communication skills and interpersonal skills, he made my patients feel well-listened too.

  8. PRIME Professionalism Timeliness, honesty, attitude Reporter Gathering info, writeups and oral presentations Interpreter Weighted differential, tests Manager Offers reasonable dx and tx plan Enhanced communication

  9. PRIME (RIME) First described in the literature by Lou Pangaro in 1999 Intention was to improve the credibility of educators descriptive evaluations of students Meant to provide a synthetic approach to the evaluation of skills, knowledge and attitudes with particular attention paid to professionalism. Growing independence is the underlying premise Pangaro L Acad Med 74(11) 1999; Pangaro L Clin Anat 19:419-28 2006

  10. According to Dr. Pangaro . . . Reporter: basic skills to obtain a a hx and do a PE and the basic knowledge to know what to look for; day-to-day reliability. . . Interpreter: ability to prioritize problems identified, and offer a ddx. Has to be an active participant in patient care. Manager: Selecting options and proposing plan; Requires more knowledge, more confidence and more judgement. Requires higher-level interpersonal skills to tailor plan to patient. Pangaro L Acad Med 74(11) 1999

  11. Is PRIME a pyramid? Sort of! Professionalism and reporter necessary for other skills BUT students can have manager and interpreter skills in some contexts and not others

  12. Important elements of strong reporter-level student Oral case presentations and notes are concise, complete Information presented is accurate Can modify presentation to fit setting Notes change appropriately; minimal/appropriate use of copy-paste and EMR importing

  13. Important elements of strong interpreter-level student Strong knowledge base Differential that is reasonable and weighted and supported Able to change differential as clinical course evolves and support changes Ability to interpret testing is a lab change clinically significant?

  14. Important elements of strong interpreter-level student Ability to prioritize problems in the inpatient and outpatient settings Able to assess status of chronic disease visit script for outpt problems Able to zoom in or out if there are issues more important than the presenting problem Why or differential within the differential E.g. yes the patient is dyspneic from a PE, but why does this healthy young person have a PE?

  15. Important elements of strong manager-level student Can defend the plan to the team and explain it to a patient Takes initiative follow up on tests, call for records Triage new problems, take appropriate steps Develop a patient-centered management plan incorporate goals and values into plan, especially on-the-fly as a patient questions a plan

  16. Important elements of strong manager-level student Strong knowledge base Ability to choose/order appropriate testing (not just order everything!) Accurate, nuanced management plan that changes over time e.g get patient ready for discharge, length of antibiotics, monitoring, counseling of side effects

  17. Meaningful Independence For all levels of PRIME: If students need lots of reminders, coaching, help then they probably are not showing meaningful independence This is the biggest determinant of what level they attain

  18. Student who truly manages but has some reporting deficits RARELY there will be a student who is truly managing at a high level but has reporting deficits (this is not common and usually due to anxiety with oral presentations and/or trouble expressing themselves in writing) The grade for this student should be high pass, to reflect that they are truly managing but need to improve some reporting skills

  19. Three aspects of management: 1. Do er Checks off boxes 2. Triager Developing clinical intuition/ sick not sick Recognizes when boxes need to change 3. Executive Suggests and starts to execute new plan of care Defends plan to team Engages with pt in shared decision-making about plan Adapted from Goren E and Kogan J, CDIM national meeting poster 2014

  20. Not just any management plan . . . Snow Day if you can t make it in, can you trust the student to get a safe plan going? This can be a tough area to give feedback in

  21. Written feedback using PRIME Joseph was an absolute joy to work with closely for one week as a subintern on a busy hospitalist service. He was working very solidly at the Manager level on the PRIME system with excellent Professionalism and Enhanced Communication skills. He was extremely capable and thoughtful, able to see patients independently, come up with a fairly nuanced treatment plan and carry it out with appropriate supervision.

  22. Written feedback using PRIME He had excellent clinical judgment given his level of training. He truly excelled in communication with patients and families, even able to run a family conference alone when I was called away to an emergency - when I came back, he had developed excellent rapport with the family and had obtained a very relevant extended social history and started a helpful conversation about the patients' goals of care.

  23. Written comments using PRIME (areas of strength) Jane quickly formulates organized oral case presentations and presented these in the room well. Jane s interpretation skills are quite good, she offers a differential diagnosis for new patient issues and makes accurate assessments of patients' chronic medical conditions. She is beginning to develop her management skills - thinking about what labs and studies are indicated and follow-up on these.

  24. Written comments using PRIME (areas needing improvement) Jane and I discussed continuing to work on efficiency in history collection and managing her time with the patient in the room, mastering her reporting skills. She is working on further developing her management skills, she is beginning to take more ownership of he patient during and following the visit, I would encourage her to continue to do this.

  25. PRIME in grading in IM Reporter = Pass Interpreter = High Pass Manager with Professionalism and Enhanced Communication = Honors Should be at a level consistently (>75%)

  26. Grading for IM portion of WRITE WRITE preceptors asked to fill out eval twice We do not use the benchmarks in grading they are for feedback. Please choose the final grade based on the PRIME rubric Average WRITE grade counts as 50% of clinical grade

  27. Grading for IM clerkship Clinical GPA (50% WRITE, 50% inpatient) NBME IM exam score converted into a number that is added to the clinical grade

  28. Effect of NBME on final grade If clinical GPA is near a final grade cut-off, test score above or below 77 can raise or lower grade one level Test scores below 75 are ineligible for a final grade of "Honors". Test scores 30 or below are a "Fail" score. Students who fail the NBME exam will fail the clerkship.

  29. IM Grading Committee Meets quarterly All IM site directors, all associate directors involved with core clerkship Reviews all students completing clerkship; spends more time on students near a cutoff Decisions made by consensus (sometimes vote) Rely heavily on comments

  30. IM grade distribution 2018-19 Honors 38% High Pass 41% Pass 21%

  31. Questions?

More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#