Introduction to Interrupted Time Series Analysis: Methods and Applications
Interrupted Time Series Analysis (ITSA) is a method used to evaluate the effects of interventions on outcome variables over time. This analysis involves observing data before and after an intervention to detect changes in levels and trends. ITSA can be applied to single-group or multiple-group designs, with statistical modeling approaches such as ARIMA models and OLS regression commonly used. By providing definitions, examples, and methods for causal inference, this approach helps researchers understand the impact of interventions in various contexts.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Conducting interrupted time-series analysis for single- and multiple-group comparisons Ariel Linden, DrPH Department of Medicine Medical School University of California, San Francisco
Agenda Agenda Provide definitions and illustrate possible patterns Briefly describe modelling approaches (advantages/disadvantages) Detail OLS model parameters Present some examples Describe methods to improve causal inference in ITSA
Definition of interrupted time series analysis Definition of interrupted time series analysis An outcome variable that is observed: over multiple, equally-spaced time periods before and after the introduction of an intervention which is expected to interrupt its level and/or trend
Definition of interrupted time series analysis Definition of interrupted time series analysis Single group time-series design (Campbell and Stanley 1966): 0 0 0 0 X0 0 0 0 Multiple-group time-series design (Campbell and Stanley 1966): 0 0 0 0 X0 0 0 0 0 0 0 0 0 0 0 0
When is ITSA useful When is ITSA useful Single-group ITSA: For N-of-1 studies When the only data available are summarized at the population-level (e.g. mortality or morbidity rates) When observations are available for multiple evenly-spaced time points Multiple-group ITSA: Same as above but when a control group is available for comparison
Some possible single Some possible single- -group ITSA patterns group ITSA patterns* E A F B C G H D * Reproduced (more or less) from Campbell and Stanley (1966)
Statistical modeling approaches for ITSA Statistical modeling approaches for ITSA Statistical models used for ITSA account for autocorrelation (the relationship between a variable's current value and its past values) The two general approaches historically used in ITSA are: Autoregressive Integrated Moving-Average (ARIMA) models w/transfer function (Box and Tiao 1975) Y = Y-1,-2,-3...X, (#p,#d,#q) , p is autoregressive, d is differencing, and q is moving-average Y = Y-1,-2,-3...X, (#p,#d,#q) (#P,#D,#Q,#s), where second part relates to seasonality (can be implemented in Stata using tstf-) OLS regression models designed to adjust for autocorrelation (see Simonton [1977])
Advantages of ARIMA Advantages of ARIMA Designed to handle autocorrelation, seasonality and cyclical trends Non-linear models fit the data better than linear models
Disadvantages of ARIMA Disadvantages of ARIMA At least 50, and preferably more than 100 observations are needed to stabilize the estimates (Box and Jenkins 1976) ARIMA models are inherently designed for univariate time-series with a single intervention, not sequential interventions. Comparison to a control group is possible, but complicated The procedural complexity of the methodology requires a great deal of expertise. Velicer and Harrop (1983) demonstrated that even highly trained researchers classified only 28 percent of computer-generated time series data correctly
Advantages of OLS Advantages of OLS- -ITSA models ITSA models OLS may require as few as four observations (Simonton 1977) OLS models are sufficiently flexible to accommodate multiple treatment periods and comparisons with one or more control groups Post-estimation tests are available to assess whether the model correctly adjusted for autocorrelation -lincom- can be used post-estimation to produce other estimates OLS models can be used in conjunction with weighting or matching to improve causal inference OLS models avoid the problems of model identification that encumbers the ARIMA methodology
Disadvantages of OLS Disadvantages of OLS- -ITSA models ITSA models They produce linear estimates that may not fit the data well Adjusting for seasonality and cyclical trends adds complexity to the modeling process
The single The single- -group OLS group OLS- -ITSA model ITSA model Yt= 0 + 1Tt+ 2Xt+ 3XtTt+ t Ytis the summarized outcome variable measured at each equally spaced time point t, Ttis the time since the start of the study, Xtis a dummy (indicator) variable representing the intervention (pre-intervention periods 0, otherwise 1), and XtTtis an interaction term
The single The single- -group OLS group OLS- -ITSA model ITSA model 3(XT) Outcome (Y) 2(X) 0 1(T) Pre-intervention Post-intervention
The multiple The multiple- -group OLS group OLS- -ITSA model ITSA model Yt= 0 + 1Tt+ 2Xt+ 3XtTt+ 4Z + 5ZTt+ 6ZXt+ 7ZXtTt+ t 0 to 3 represent the control group, and 4 to 7 represent the treatment group Z is a dummy variable to denote the cohort assignment (treatment or control) ZTtZXt, and ZXtTtare all interaction terms among previously described variables
The multiple The multiple- -group OLS group OLS- -ITSA model ITSA model Outcome (Y) 6(ZX) 3(XT) 5(ZT) 4(Z) Pre-intervention Post-intervention
Threats to internal validity of ITSA models Single-group ITSA: History -- some event other than the intervention produces the shift in the time-series (Campbell and Stanley 1966) Seasonality Multiple-group ITSA: Unmeasured confounding that affects the treatment group s series differently than the control group s series
Examples of where SG-ITSA results can be misinterpreted (math free)
Floridas 2000 repeal of the helmet law on motorcycle deaths Motorcycle deaths in Florida before and after repeal of the helmet law in July 2000
Floridas 2000 repeal of the helmet law on motorcycle deaths Motorcycle registrations in Florida before and after repeal of the helmet law in July 2000
Floridas 2000 repeal of the helmet law on motorcycle deaths Florida s motorcycle deaths vs those of all other States, and matched control States
Californias Proposition 99 (1988) on per capita cigarette sales (a) Cigarette sales in California before and after Proposition 99. (b) Structural break in 1983
Californias Proposition 99 (1988) on per capita cigarette sales Comparing California s cigarette sales to all other States, and to matched control States
Louisianas repeals and reinstatements of the helmet law Louisiana s multiple repeals/amendments/reinstatements of helmet law on motorcycle deaths
Louisianas repeals and reinstatements of the helmet law Louisiana s motorcycle deaths and registrations compared to all other States
Take-aways about SG-ITSA? A single-group ITSA is unlikely to be more valid than a simple pre-post test Perform structural break analysis to further support/refute a treatment effect Adding one or more cross-overs may not improve validity Identifying confounders, history, secular trends, may support/refute a treatment effect A multiple-group ITSA is ALWAYS preferred to a single-group ITSA A matched (or weighted) multiple-group ITSA is EVEN better!
Improving causal inference in MG-ITSA Use weights derived from the synthetic controls method (Abadie and Diamond 2010) in conjunction with ITSA Match on covariates to identify controls using ITSAMATCH then estimate treatment effects using ITSA Run permutation tests (which includes matching and estimation) using ITSAPERM
Synthetic controls and ITSA SYNTH assigned weights to Colorado (.159), Connecticut (.068), Montana (.203), Nevada (.235) and Utah (.335)
ITSAMATCH and ITSA ITSAMATCH found Colorado, Idaho, and Montana as best matches to California
ITSAPERM After iteratively matching the treatment unit (CA) and pseudo-treatment units (all other States) to controls, only CA appears to have a statistical (and directionally correct) effect
Conclusions Interrupted time series analysis is an observational (natural) study design that capitalizes on having many data-points for determining treatment effects (both visually and statistically) A single-group ITSA may be no more valid than the simple pre-post design if some (non-observed) event other than the intervention produced the shift in the time-series A multigroup ITSA that compares the treated unit to one or more comparable controls (via weighting or matching) is the most valid approach with observational data Adding permutation tests provides an additional robustness check
ITSA related packages for Stata 1. ITSA Performs interrupted time series analysis for single and multiple group comparisons 2. ITSAMATCH Performs matching in multiple group interrupted time series analysis 3. ITSAPERM Performs permutation tests for matched multiple group interrupted time series analysis 4. ITSARAND Performs randomization tests for single-case and multiple-baseline AB phase designs 5. XTITSA - Performs interrupted time series analysis with panel data
References 1. Abadie A, Diamond A. 2010. Synthetic control methods for comparative case studies: Estimating the effect of California's tobacco control program. J Am Stat Assoc. 105: 493 505. 2. Box, G. E. P., and G. M. Jenkins. 1976. Time Series Analysis: Forecasting and Control. San Francisco, CA: Holden Day. 3. Box, G. E. P., and G. C. Tiao. 1975. Intervention analysis with applications to economic and environmental problems. Journal of the American Statistical Association 70: 70 79. 4. Campbell, D. T., and J. C. Stanley. 1966. Experimental and Quasi-Experimental Designs for Research. Chicago, IL: Rand McNally. 5. Linden A. 2015. Conducting interrupted time series analysis for single and multiple group comparisons. Stata Journal 15: 480-500. 6. Linden A. 2016. Challenges to validity in single-group interrupted time series analysis. Journal of Evaluation in Clinical Practice 23: 413 418. 7. Linden A. 2017. A comprehensive set of post-estimation measures to enrich interrupted time series analysis. Stata Journal 17: 73-88. 8. Linden A. 2017. Persistent threats to validity in single-group interrupted time series analysis with a crossover design. Journalof Evaluation in Clinical Practice 23: 419 425. 9. Linden A. 2018. A matching framework to improve causal inference in interrupted time series analysis. Journalof Evaluation in Clinical Practice 24: 408-415. 10. Linden A. 2018. Combining synthetic controls and interrupted time series analysis to improve causal inference in program evaluation. Journalof Evaluation in Clinical Practice 24: 447-453. 11. Linden A. 2018. Using permutation tests to enhance causal inference in interrupted time series analysis. Journalof Evaluation in Clinical Practice 24:496-501. 12. Linden A. 2022. Erratum: A comprehensive set of post-estimation measures to enrich interrupted time series analysis. Stata Journal 22:231-233. 13. Simonton, D. K. 1977. Cross-sectional time-series experiments: Some suggested statistical analyses. Psychological Bulletin 84: 489 502. 14. Velicer, W. F., and J. Harrop. 1983. The reliability and accuracy of time series model identification. Evaluation Review 7: 551 560.
