The Need for New Statistical Methodology in Health Studies

The discussion centers around the necessity for advancements in statistical methodology, particularly in trials, observational studies, and meta-analysis. The role of statisticians in interpreting data, designing studies, and addressing missing data is highlighted. The importance of understanding existing methodologies and embracing innovative approaches is emphasized to enhance the effectiveness and efficiency of health research studies.

• Statistical methodology
• Health studies
• Methodological advances
• Clinical trials
• Observational studies

keon736 Follow

Uploaded on Sep 16, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Do we need new statistical methodology? Ian White <ian.white@ucl.ac.uk> MRC Clinical Trials Unit at UCL UCL Network for Applied Statisticians in Health (NASH) 21st May 2018 MRC Clinical Trials Unit at UCL

2. What is NASH? John Nash (English architect, 1752 1835) Ogden Nash (American poet, 1902 1971) There was an old man of Calcutta, Who coated his tonsils with butter; Thus converting his snore From a thunderous roar To a soft oleagenous mutter. Hypothesis generating needs randomised evaluation Russian: = ours 2 MRC Clinical Trials Unit at UCL

3. Do we need new statistical methodology? It depends 3 MRC Clinical Trials Unit at UCL

4. Statement of interest I m a methodologist interested in trials, observational studies & meta- analysis esp. missing data, treatment changes in trials, network meta-analysis MRCCTU s mission: carrying out challenging and innovative studies developing and implementing methodological advances in study design, conduct and analysis 4 MRC Clinical Trials Unit at UCL

5. What do we want? Do our jobs more effectively efficiently pleasantly fulfillingly profitably Do we need new statistical methodology for this? it depends on what role we are playing 5 MRC Clinical Trials Unit at UCL

6. Roles: the cant-do statistician You can t interpret P>0.05 as no association ! You can t publish that association without describing all the other associations you also explored! (in an observational study) You can t replace non-complying participants with new ones! (in a trial) You can t say the treatment only works in a subgroup without doing an interaction test! (in a trial) OK from time to time, but not as a permanent mindset. What do we need for this role? sound grasp of existing methodology thick skin 6 MRC Clinical Trials Unit at UCL

7. Roles: the can-do statistician How can I explore this aspect of my data? how can I model children s growth through puberty? my data are missing not at random, what should I do? some trial participants received rescue medication, what should I do? How can I design my study? should I run this trial as a factorial design? What do we need for this role? Often involves clever use of existing methodology May need tweaking / extending existing methodology May need really new methodology usually not used in an applied paper 7 MRC Clinical Trials Unit at UCL

8. Roles: defending our work Against the can t-do reviewer that analysis wasn t pre-specified so you mustn t report it Against the can-do (and therefore should do) reviewer you have missing data, you must use multiple imputation What do we need for this role? Good understanding of the methods Good understanding of theory 8 MRC Clinical Trials Unit at UCL

9. Is some new methodology useful? Certainly! e.g. intention-to-treat analysis multiple imputation network meta-analysis 9 MRC Clinical Trials Unit at UCL

10. Is all new methodology useful? Certainly not! Much new methodology has zero impact why not? 10 MRC Clinical Trials Unit at UCL

11. What do applied statisticians need before we start using a new methodology? Valid theory? less important than we might think (e.g. MICE) Published use? in more than one data set Generality? e.g. allows both categorical and continuous covariates; handles missing data Simulations? showing good performance comparing new method with standard alternatives but how do we know if they apply to our setting? Useable software? almost essential How do we decide? 11 MRC Clinical Trials Unit at UCL

12. How do we judge when a new methodology is ready for our use? Probably not very scientifically It s useful to understand how well developed a methodology is I believe we can identify phases of methodological development like phases of drug development but not formalised 12 MRC Clinical Trials Unit at UCL

13. Phases of methodological development 1. Technique is used by methodological statistician with software purpose-built for the application First-in-data publication: e.g. Biostatistics 2. Technique is developed as a methodological project for use in new data sets publication: e.g. Stat Med 3. Technique is used by applied statisticians in clinical publications publication: e.g. AJE, BMJ software available 4. Review articles promote use of technique; referees start to demand its use publications in clinical or statistical journals 13 MRC Clinical Trials Unit at UCL

14. Phases of methodological development: multiple imputation Phase 1: Technique is used by methodological statistician with software purpose-built for the application 2: Technique is developed as a methodological project for use in new data sets 3: Technique is used by applied statisticians in clinical publications Publication Rubin DB. Multiple Imputation for Nonresponse in Surveys. New York: John Wiley and Sons; 1987. Rubin DB. Nested multiple imputation of NMES via partially incompatible MCMC. Stat Neerl. 2003;57:3 18. Horton NJ, Lipsitz SR. Multiple imputation in practice: Comparison of software packages for regression models with missing variables. Am Stat. 2001;55:244 54. Sullivan TR, White IR, Salter AB, Ryan P, Lee KJ. Should multiple imputation be the method of choice for handling missing data in randomized trials? Stat Methods Med Res. 2016 4: Review articles promote use of technique; referees start to demand its use 14 MRC Clinical Trials Unit at UCL

15. Can new methodology be harmful? Statistics is an unusual science in that we don t just have to use a valid technique, we have to defend our choice to avoid accusations of cherry-picking / data- dredging So new methodology can create confusion especially if claims made for it are exaggerated 15 MRC Clinical Trials Unit at UCL

16. Arm-based network meta-analysis New proposal (~2014): change from modelling contrasts (asymmetrically) to modelling arms (symmetrically) e.g. Lin L, Zhang J, Hodges JS, Chu H. Performing Arm-Based Network Meta-Analysis in R with the pcnetmeta Package. J Stat Softw. 2017;80(5). claimed to have better missing data properties & to lead to more appropriate summaries can give quite different results from standard contrast-based methods critics argue that it breaks randomisation Consequences for applied network meta-analysis are unclear Lots of debate in methodological fora has debate been useful? 16 MRC Clinical Trials Unit at UCL

17. Hierarchy for evaluating the effects of a diagnostic technique [if time] Here described for magnetic resonance imaging (MRI): Technical performance: Does MRI result in good quality images which are anatomically representative? Diagnostic performance: Do the images allow accurate diagnoses to be made? Diagnostic impact: Does MRI change diagnostic confidence and displace other investigations? Therapeutic impact: Do the results of MRI contribute to planning and delivery of treatment? Impact on health: Does the use of MRI contribute to the improved health of the patient? Mackenzie R, Dixon AK (1995). Measuring the effects of imaging: An evaluative framework. Clinical Radiology 50:513-518. 17 MRC Clinical Trials Unit at UCL

18. Hierarchy for evaluating the effects of a diagnostic / statistical technique? Imaging technique Statistical technique Technical performance Diagnostic performance Diagnostic impact Good quality images Results are internally consistent Results are convincing to statisticians Results are convincing to clinicians Images allow accurate diagnoses Images change diagnostic confidence and displace other investigations Contributes to planning and delivery of treatment Therapeutic impact Results lead to change of treatment (compared to different analyses?) Results lead to improved public health Impact on health Contributes to the improved health of the patient 18 MRC Clinical Trials Unit at UCL

19. Conclusions and questions Sometimes we need new methodology Sometimes new methodology is forced on us We need new methodology to be carefully evaluated need to be clear when methods are under development and when they are ready for practice How can we judge when new methodology is ready for use? should methodological journals rate their papers according to their readiness for practice? do we need a methodological journal restricted to methods actually used in applied journals? other suggestions? 19 MRC Clinical Trials Unit at UCL