Study Design Considerations for Observational Comparative Effectiveness Research
This presentation outlines key considerations for study design in observational comparative effectiveness research, including rationale for design choice, defining start of follow-up, inclusion/exclusion criteria, exposures of interest, outcomes, and potential confounders. It discusses various study designs like cohort, case-control, and self-controlled designs, highlighting advantages and limitations of each. Additionally, it covers issues of bias in observational research and the importance of internal validity for ensuring the credibility of study findings.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Study Design Considerations for Observational Comparative Effectiveness Research Prepared for: Agency for Healthcare Research and Quality (AHRQ) www.ahrq.gov
Outline of Material This presentation will: Provide a rationale for study design choice and describe key design features Define start of followup Define inclusion and exclusion criteria at start of followup Define exposures of interest at start of followup. Define outcome(s) of interest Define potential confounders
Study Design Overview Conventional designs Cohort Case-control Case-cohort Self-controlled designs Case-crossover Case-time-controlled Self-controlled case series
Issues of Bias in Observational Comparative Effectiveness Research Exposures or treatments are not assigned, a situation which leads to challenges ensuring internal validity, that is, the absence of bias. To ensure internal validity, treatment groups compared must have the same underlying risk for outcome within subgroups definable by measured covariates (e.g., no unmeasured confounding). Confounding by indication leads to higher propensity for/more intensive treatment in those with the most severe disease. With confounding by frailty, frail patients (close to death) are less likely to be treated with preventive treatments. Ensuring a study s internal validity is a prerequisite for its external validity or generalizability.
Study Design: Cohort Study Cohorts are defined by their exposure at a certain point in time (i.e., baseline date) and are followed over time for the occurrence of the outcome. Advantages: Has a clear timeline separating potential confounders from the exposure and the exposure from the outcome Allows estimation of actual incidence (risk or rate) Can assess multiple outcomes Is easy to conceptualize Limitations: Is inefficient for ad hoc studies when the incidence of the outcome is low
Study Design: Case-Control Study Identifies all incident cases that develop an outcome and compares exposure history to controls Samples controls at random from cohort members at risk for developing an outcome Advantages: Oversampling cases increases computational efficiency of ad hoc studies when compared with a cohort study Can assess multiple exposures Limitations: Is difficult to conceptualize Has potential for recall bias in ad hoc studies
Study Design: Case-Cohort Study Cohorts defined as in a cohort study Cohort members followed for incidence of outcomes Additional information required for analysis collected for a random sample of the cohort and all cases Increased efficiency, when compared with a full-cohort design, if additional information needs to be collected Decreased efficiency, when compared with a nested case-control design, unless studying multiple outcomes or estimating risk
Study Design: Case-Crossover Study Prior exposure history of cases used as the control Removes confounding effect of measured and unmeasured characteristics that are stable over time (e.g., genetics) Appropriate for studying acute effects of transient exposures Advantages: Self-controlled Ability to assess short-term reversible effects Ability to inform about the time window for these effects Limitations: Assumes constant prevalence of treatments over time Does not allow estimation of treatment effect in a population
Study Design: Case-Time-Controlled Study Adjusts for calendar time trends in the prevalence of treatments, which can bias the case-crossover design Divides the case-crossover odds ratio by the equivalent odds ratio estimated in controls Advantages: Not dependent on assumption of no temporal changes in the prevalence of treatment Limitations: Need for controls adds complexity Control for time trend can introduce confounding
Study Design: Self-Controlled Case-Series Estimates the immediate effect of treatment in those treated at least once Dependent on cases that have changes in treatment during a defined observation period Advantages: Controls for factors that are stable over time Cohort design has the potential to increase efficiency Well suited for rare adverse events in vaccine safety studies Limitations: Limited applicability in many comparative effectiveness research studies
Study Design Features Study setting Consideration of the study population and data source(s) Inclusion and exclusion criteria Should be clearly defined Include details about the study time period Choice of comparators Reduces potential for confounding by comparing treatment of interest with a different treatment for the same indication or an indication with the same potential for confounding
Other Study Design Considerations New-User Design The conventional prevalent user design is prone to confounding and selection bias as a result of changes in treatment effects over time. Including only new users reduces bias and confounding associated with inclusion of prevalent users. There must be a clear starting point for followup under similar conditions of medicalization. Immortal Time Bias Occurs as a result of defining the exposure during the followup time rather than before followup New-user design and use of comparator treatments reduce potential for this bias
Conclusions Knowledge of study design options is essential to increase internal and external validity of observational comparative effectiveness research. Biases introduced by suboptimal study design cannot usually be removed by statistical analysis. Cohort design is preferred when data have already been collected; the validity of a nested case-control study is equivalent, given proper control selection and timing of exposures and covariates. It is important to define the start of followup, inclusion and exclusion criteria, outcome of interest, and potential confounders at the outset.
Summary Checklist (1 of 3) Guidance Provide rationale for study design choice and describe key design features Key Considerations Cohort study proposals should clearly define the cohort entry date (baseline date), employ a new-user design (or provide a rationale for including prevalent users), and plans for reporting losses to followup. Case-control study proposals should clearly describe the control sampling method, employ a new-user design (or provide a rationale for assessing confounders at the index date), and assess potential for recall bias (if applicable). Case-cohort study proposals should include how the sampling scheme will be accounted for during analysis. Case-crossover study proposals should discuss the potential for confounding by time-varying factors and clearly state how the resulting effect estimate can be interpreted. Case-time-controlled study proposals should clearly weigh the pros and cons of accounting for calendar time trends in the prevalence of exposure.
Summary Checklist (2 of 3) Guidance Key Considerations Define start of followup (baseline) The time point for start of followup should be clearly defined, meaningful, and ideally anchored to the time of a medical intervention (e.g., initiation of drug use). If alternative approaches are proposed, the rationale should be provided and implications discussed. Define inclusion and exclusion criteria at start of followup Exclusion and inclusion criteria should be defined at the start of followup (baseline) and solely based on information available at this point in time (i.e., ignoring potentially known events after baseline). The definition should include the time window for assessment (usually the same for all cohort members).
Summary Checklist (3 of 3) Guidance Key Considerations Define exposure (treatments) of interest at start of followup Use an active comparator (indicated as alternative treatment at same stage of disease progression) when possible. Define outcome(s) of interest Provide information on measures of accuracy if possible. Define potential confounders Potential confounders known to be associated with treatment and outcome should be prespecified when possible. Confounders should be assessed before exposure or treatment initiation to ensure they are not affected by the exposure. Approaches to empirical identification of confounders should be described if planned.
