Pilot Studies in Advancing Dementia Care: Core Considerations

Slide Note

Pilot studies play a crucial role in exploring interventions, determining necessary modifications, assessing feasibility, and confirming stakeholder acceptance. They help in testing study protocols, assessing data collection procedures, and evaluating outcome measures. Feasibility studies are essential to determine if an intervention can be implemented successfully and meet the requirements for a larger trial. Sample size in pilot studies should align with defined feasibility objectives rather than intervention efficacy.

tane601 Follow

Uploaded on Sep 11, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript

The Geriatric Emergency care Applied Research Network 2.0 Advancing Dementia Care (GEAR 2.0 ADC) Research Core Considerations on Pilot Studies Heather Allore, PhD Prof Geriatrics and Biostatistics Yale School of Medicine Yale School of Public Health September 20, 2022

https://www.nccih.nih.gov/research/blog/uses-and-misuses-of-pilot-studieshttps://www.nccih.nih.gov/research/blog/uses-and-misuses-of-pilot-studies

Why conduct a pilot study? Why conduct a pilot study? A pilot study is a good opportunity to: explore your intervention learn whether you need to make modifications decide whether you can proceed to a larger trial confirm whether the proposed intervention is needed confirm whether the proposed intervention is acceptable to all stakeholders investigate whether the relevant population will agree to take part in a potential randomized study check whether participants can be recruited check whether participants can be retained measure the integrity of study protocol test randomization procedure test data collection/retrieval explore potential outcome measures can they be recorded?

Pilot and Feasibility Studies Pilot and Feasibility Studies A feasibility study is undertaken to ascertain whether or which parts of an intervention can be performed, and whether the intervention meets a threshold to proceed to a larger scale. All pilot studies are feasibility studies intended to guide the planning of a large-scale trial, but not all feasibility studies are pilot studies. Reasons for conducting pilot trials: assess the feasibility of proposed intervention or components assess the feasibility of the logistics (fits into workflow), recruitment, retention and resources requirement assess the feasibility of outcome measurement, data collection and treatment safety Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, et al. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PLoS One. 2016;11(3):e0150205. 4

Sample Size for Pilot Studies? Sample Size for Pilot Studies? The goal of a pilot study is to investigate areas of uncertainty for a future full-scale trial. Unlike a full-scale trial, pilot studies do not have intervention efficacy as their focus. Therefore, sample size rationale for a pilot should not be based on demonstrating efficacy of the intervention. Pilot studies require clearly formulated feasibility objectives with defined outcome measures for each objective. Selecting the most important feasibility outcome(s) should motivate the required sample size for the pilot study. If there are multiple equally important feasibility outcomes, it may be necessary to justify the sample size (maximum or minimum) for each. An essential ingredient of a pilot study is clearly defined progression criteria or feasibility targets that will guide the decision whether to proceed to a full ePCT. Thus, it may be possible to quantitatively estimate the sample size based on ensuring a high degree of confidence that the feasibility targets are met, i.e., ensuring that the lower limit of a 95% confidence interval around the feasibility outcome does not drop below the feasibility target. 5

Sample Size Guidance Sample Size Guidance Possible quantitative approaches proposed to help guide sample size requirements for pilot studies for individual-level randomization: Estimate pilot sample size based on percentage of the number of participants required in the future ePCT; the literature cites of a range 3%-10%. [1-3] Estimate pilot sample size based on percentage of the number required for the future definitive ePCT and its anticipated standardized effect sizes. For a main trial designed with 90% power and two-sided 5% significance, the recommended pilot trial sample sizes per treatment arm of 75, 25, 15 and 10 for standardized effect sizes that are extra small (0.1), small (0.2), medium (0.5) or large (0.8), respectively. [3,4] 1. Cocks K, Torgerson DJ. Sample size calculations for pilot randomized trials: a confidence interval approach. J Clin Epidemiol. 2013 Feb;66(2):197-201. doi: 10.1016/j.jclinepi.2012.09.002. 2. Stallard N. Optimal sample sizes for phase II clinical trials and pilot studies. Stat Med. 2012;31(11-12):1031-42. doi: 10.1002/sim.4357. 3. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, Lancaster GA; PAFS consensus group. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot Feasibility Stud. 2016;2:64. doi: 10.1186/s40814-016-0105-8. PMC5154046. 4. Whitehead AL, Julious SA, Cooper CL, Campbell MJ. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat Methods Med Res. 2016;25(3):1057-73. doi: 10.1177/0962280215588241. PMC4876429

Sample size for feasibility/progression Sample size for feasibility/progression An essential ingredient of a pilot study is clearly defined progression criteria or feasibility targets that will guide the decision whether to proceed to a full trial. It may be possible to quantitatively estimate the sample size based on ensuring a high degree of confidence that the feasibility targets are met, i.e., ensuring that the lower limit of a 95% confidence interval around the feasibility outcome does not drop below the feasibility target. Examples of feasibility targets: Participant enrollment rate Proportion of participants who adhere to the intervention protocol Proportion of participants who find the intervention acceptable Consider a traffic light system for specifying progression criteria, green (or go) indicates that the criteria were met, and the trial should proceed; amber (or amend) indicates that some changes should be made prior to the larger trial; red (or stop) indicates that the intervention should not move forward with the larger trial. Avery KNL, Williamson PR, Gamble C, Connell Francischetto E, Metcalfe C, Davidson P, Williams H, Blazeby JM. Informing efficient randomised controlled trials: exploration of challenges in developing progression criteria for internal pilot studies. BMJ Open. 2017;7(2)

Pilot versus underpowered trial Pilot versus underpowered trial Pilot study results should not be used for sample size determination due to the inherent imprecision in between treatment group effect size estimates from studies with small sample (Leon et al. 2010) Lack of precision, or large uncertainty can misguide future studies It is not a small underpowered study. It does not test efficacy. Always report according to the CONSORT guidance http://www.consort-statement.org/extensions/overview/pilotandfeasibility Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al.A tutorial on pilot studies: the what, why and how. BMC Medical Research Methodology. 2010;10(1).