Step-by-Step Guide to Using the GRADE Approach for Guideline Development

 
Using the GRADE approach for
Guideline Development
 
A step-by-step guide
 
Thomy Tonia MSc, David Rigau MD. ERS Methodologists
 
1
 
Before starting….
 
Configure Guideline panel
Declare conflicts of interest and ways they will
be dealt with
Define the scope of the Guideline
Target population
What to cover (diagnostic and/or intervention
and/or prevention strategies)
 
2
 
Documents producing
recommendations have to use GRADE
 
If you aim to make recommendations for
clinical practice, you have to develop your
document by conducting a systematic review,
grading of the evidence and the
recommendations’ strength (i.e. use the
GRADE approach)
 
3
 
Step 1:
 Formulate the questions
 
4
 
Step 2: 
Select outcomes of interest
and rate their importance
 
Step 3:
 Systematic Review
 
Step 4:
 GRADE the evidence
 
Step 5:
 Going from evidence to
recommendations
 
Step 1: Formulate the questions
 
Use the PICO format
 
5
 
Step 1: Formulate the questions
 
Restrict the number of questions (about 7)
Each PICO question will result in one
recommendation
No PICO question, no recommendation!
 
6
 
Step 2: Select outcomes of interest
 
Select outcomes for each PICO question
before
 conducting the literature review
Outcomes should be importance driven and
not evidence driven
Select outcomes that are important to
patients and important for clinicians decision-
making
Avoid surrogate outcomes (for example
biomarkers)
 
7
 
Step 2: Rate outcome importance
 
Rate the importance of the selected outcomes 
before
conducting the literature review
How important is each outcome for decision making?
Not important, important, of critical importance
Suggested rating scale:
 
8
 
Step 3: Systematic Review
 
Pragmatic literature search
Search for recent, well-conducted systematic
reviews and build up on them
Search main database(s) only
MEDLINE
Consider including CENTRAL (
Cochrane Central Register
of Controlled Trials
) or EMBASE
One assessor (no need of double reading of all
references)
 
9
 
Step 3: Systematic Review
 
Select studies according to predefined criteria
 
Extract outcomes of interest
 
Meta-analyse, when applicable
 
10
 
Step 4: GRADE the evidence
 
Create evidence tables for each PICO question
Evidence tables contain 2 parts: results and
quality of evidence
There are 4 degrees of quality: high,
moderate, low and very low
When assessing INTERVENTIONS: RCTs start
from high quality; observational studies start
from low quality
 
11
 
Step 4: GRADE the evidence
 
Quality is always evaluated per outcome and
not per study
Outcome #1 – Quality of evidence
Outcome #2 – Quality of evidence
Outcome #3 – Quality of evidence
 
OVERALL quality of evidence comes from
those CRITICAL outcomes for decision making
 
12
 
Step 4: GRADE the evidence
 
Quality can be decreased (one or two levels
per factor) based on the following factors:
Risk of bias
Indirectness
Inconsistency
Imprecision
Publication bias
 
QUALITY means CONFIDENCE IN RESULTS
 
13
 
Step 4: GRADE the evidence
Risk of bias
 
Consider potential methodological limitations
of the studies that assess each outcome.
Consider: concealment of allocation, intention to
treat analyses, large loss to follow-up, etc.)
 
A study (or group of studies) might have
limitations for certain outcomes but not for
others.
 
14
 
Step 4: GRADE the evidence
Indirectness
 
Indirect comparison (e.g. we want to compare drug A
to drug B but have only found studies comparing drug
A to placebo and drug B to placebo)
 
Differences in population, intervention, comparison,
outcomes of interest between the identified studies
and the clinical question we want to answer.
 
15
 
Step 4: GRADE the evidence
Inconsistency
 
It refers to important and unexplained variability
(heterogeneity) of the results among studies. This
decreases our confidence in the common (pooled)
estimate of results.
 
Look at the similarity of each study estimates
 
Look at the degree of overlap of 95%CIs
 
Statistical criteria (e.g. I
2
)
 
Look for possible reasons of variability
 
16
 
Step 4: GRADE the evidence
Imprecision
 
Is the 95%CI of the effect precise enough to take
a decision?
If 95%CI ranges from appreciable benefit to
meaningless effect (or even harm), our confidence is
decreased because both upper or lower values of
95%CI may represent the true effect.
 
Set a threshold for clinical decision making:
Would your decision to recommend an intervention
be different considering the lower or the upper values
of the 95%CI?
 
17
 
Step 4: GRADE the evidence
Publication bias
 
18
 
Failure of reporting studies that were
undertaken (often: showing negative effect)
Difficult to estimate!
Examine funnel plots
Higher risk of publication bias when there are
only very few small studies available that show
positive effect
 
Step 4: GRADE the evidence
 
19
 
Quality of evidence (QoE) per outcome
Outcome #1 – High QoE – CRITICAL
Outcome #2 – Moderate QoE – CRITICAL
Outcome #3 – High QoE – IMPORTANT
Outcome #4 – Low QoE – IMPORTANT
 
OVERALL quality per question: MODERATE
Rule: look at the critical outcomes; what is the
lowest
 quality for a critical outcome?; this would
be the overall quality for this particular question
 
Step 4: GRADE the evidence
 
Quality is always evaluated per outcome and
not per study
Outcome #1 – Quality of evidence
Outcome #2 – Quality of evidence
Outcome #3 – Quality of evidence
 
OVERALL quality of evidence comes from
those CRITICAL outcomes for decision making
 
20
 
Step 5: Going from evidence to
recommendations
 
21
 
Main factors that need to be considered:
 
- Quality of evidence
 
- Balance between benefits and harms
 
- Values and preferences of the patients
 
- Costs
 
Step 5: Going from evidence to
recommendations
 
22
 
Two degrees of recommendations (for or
against an intervention):
 
- Strong (“We recommend....”)
 
- Conditional (“We suggest…)
It is possible to derive strong
recommendations from low quality evidence,
as quality is only one factor to take into
account (see previous slide)
 
Need more help?
 
23
 
Please see the separate document in the ERS
website with further resources for producing
evidence-based guidelines
Read our FAQs found in our website
Contact the ERS Methodologists, Thomy Tonia
(
thomy.tonia@ersnet.org
) and David Rigau
(
david.rigau@ersnet.org
)
Slide Note
Embed
Share

This step-by-step guide outlines the process of utilizing the GRADE approach for developing guidelines in healthcare. It covers key stages such as formulating questions using the PICO format, selecting and rating outcomes of interest, conducting a systematic review, grading evidence, and translating evidence into recommendations. By following these steps, guideline developers can ensure a structured and evidence-based approach to creating high-quality recommendations for clinical practice.

  • Guideline Development
  • GRADE Approach
  • Healthcare
  • PICO Format
  • Evidence-Based

Uploaded on Oct 03, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. Using the GRADE approach for Guideline Development A step-by-step guide Thomy Tonia MSc, David Rigau MD. ERS Methodologists 1

  2. Before starting. Configure Guideline panel Declare conflicts of interest and ways they will be dealt with Define the scope of the Guideline Target population What to cover (diagnostic and/or intervention and/or prevention strategies) 2

  3. Documents producing recommendations have to use GRADE If you aim to make recommendations for clinical practice, you have to develop your document by conducting a systematic review, grading of the evidence and the recommendations strength (i.e. use the GRADE approach) 3

  4. Step 1: Formulate the questions Step 2: Select outcomes of interest and rate their importance Step 3: Systematic Review Step 4: GRADE the evidence Step 5: Going from evidence to recommendations 4

  5. Step 1: Formulate the questions Use the PICO format Population Intervention Comparison Outcome In patients hospitalized for COPD exacerbations is initial treatment with IV corticosteroids compared to oral corticosteroids better (e.g. reduction in length of hospital stay)? 5

  6. Step 1: Formulate the questions Restrict the number of questions (about 7) Each PICO question will result in one recommendation No PICO question, no recommendation! 6

  7. Step 2: Select outcomes of interest Select outcomes for each PICO question before conducting the literature review Outcomes should be importance driven and not evidence driven Select outcomes that are important to patients and important for clinicians decision- making Avoid surrogate outcomes (for example biomarkers) 7

  8. Step 2: Rate outcome importance Rate the importance of the selected outcomes before conducting the literature review How important is each outcome for decision making? Not important, important, of critical importance Suggested rating scale: 8

  9. Step 3: Systematic Review Pragmatic literature search Search for recent, well-conducted systematic reviews and build up on them Search main database(s) only MEDLINE Consider including CENTRAL (Cochrane Central Register of Controlled Trials) or EMBASE One assessor (no need of double reading of all references) 9

  10. Step 3: Systematic Review Select studies according to predefined criteria Extract outcomes of interest Meta-analyse, when applicable 10

  11. Step 4: GRADE the evidence Create evidence tables for each PICO question Evidence tables contain 2 parts: results and quality of evidence There are 4 degrees of quality: high, moderate, low and very low When assessing INTERVENTIONS: RCTs start from high quality; observational studies start from low quality 11

  12. Step 4: GRADE the evidence Quality is always evaluated per outcome and not per study Outcome #1 Quality of evidence Outcome #2 Quality of evidence Outcome #3 Quality of evidence OVERALL quality of evidence comes from those CRITICAL outcomes for decision making 12

  13. Step 4: GRADE the evidence Quality can be decreased (one or two levels per factor) based on the following factors: Risk of bias Indirectness Inconsistency Imprecision Publication bias QUALITY means CONFIDENCE IN RESULTS 13

  14. Step 4: GRADE the evidence Risk of bias Consider potential methodological limitations of the studies that assess each outcome. Consider: concealment of allocation, intention to treat analyses, large loss to follow-up, etc.) A study (or group of studies) might have limitations for certain outcomes but not for others. 14

  15. Step 4: GRADE the evidence Indirectness Indirect comparison (e.g. we want to compare drug A to drug B but have only found studies comparing drug A to placebo and drug B to placebo) Differences in population, intervention, comparison, outcomes of interest between the identified studies and the clinical question we want to answer. 15

  16. Step 4: GRADE the evidence Inconsistency It refers to important and unexplained variability (heterogeneity) of the results among studies. This decreases our confidence in the common (pooled) estimate of results. Look at the similarity of each study estimates Look at the degree of overlap of 95%CIs Statistical criteria (e.g. I2) Look for possible reasons of variability 16

  17. Step 4: GRADE the evidence Imprecision Is the 95%CI of the effect precise enough to take a decision? If 95%CI ranges from appreciable benefit to meaningless effect (or even harm), our confidence is decreased because both upper or lower values of 95%CI may represent the true effect. Set a threshold for clinical decision making: Would your decision to recommend an intervention be different considering the lower or the upper values of the 95%CI? 17

  18. Step 4: GRADE the evidence Publication bias Failure of reporting studies that were undertaken (often: showing negative effect) Difficult to estimate! Examine funnel plots Higher risk of publication bias when there are only very few small studies available that show positive effect 18

  19. Step 4: GRADE the evidence Quality of evidence (QoE) per outcome Outcome #1 High QoE CRITICAL Outcome #2 Moderate QoE CRITICAL Outcome #3 High QoE IMPORTANT Outcome #4 Low QoE IMPORTANT OVERALL quality per question: MODERATE Rule: look at the critical outcomes; what is the lowest quality for a critical outcome?; this would be the overall quality for this particular question 19

  20. Step 4: GRADE the evidence Quality is always evaluated per outcome and not per study Outcome #1 Quality of evidence Outcome #2 Quality of evidence Outcome #3 Quality of evidence OVERALL quality of evidence comes from those CRITICAL outcomes for decision making 20

  21. Step 5: Going from evidence to recommendations Main factors that need to be considered: - Quality of evidence - Balance between benefits and harms - Values and preferences of the patients - Costs 21

  22. Step 5: Going from evidence to recommendations Two degrees of recommendations (for or against an intervention): - Strong ( We recommend.... ) - Conditional ( We suggest ) It is possible to derive strong recommendations from low quality evidence, as quality is only one factor to take into account (see previous slide) 22

  23. Need more help? Please see the separate document in the ERS website with further resources for producing evidence-based guidelines Read our FAQs found in our website Contact the ERS Methodologists, Thomy Tonia (thomy.tonia@ersnet.org) and David Rigau (david.rigau@ersnet.org) 23

Related


More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#