Generalizing Research on Older Adults in Seattle Integrated Health System

Slide Note

This research project led by Laura Gibbons focuses on generalizing findings from the Adult Changes in Thought (ACT) study in a Seattle integrated health delivery system to all older adults in the region. By comparing ACT participants with the current Seattle area population and using survey weights from the Behavioral Risk Factor Surveillance System, the study aims to determine the generalizability of the research findings across the region. Strategies include constructing weights to align ACT with the broader region and addressing missing data through multiple imputation analyses.

arran Follow

Uploaded on Mar 08, 2024 | 1 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript

Generalizing research based in a Seattle integrated health delivery system to all older adults in the region Laura Gibbons, PhD University of Washington gibbonsL@uw.edu

Motivation Motivation Invitation to participate in the Adult Changes in Thought (ACT) study is sent to a randomly-selected group of Kaiser Permanente Washington members aged 65+. Assessed for dementia and join study if not demented. More of a community-based sample than most research on older adults. How well do the ACT participants enrolled from 1994-2020 compare to the current Seattle area? Will our findings generalize to all older adults currently in our region? Transport study population -> target population

Strategy Strategy Use the 1158 people age 65+ in the 2019 Behavioral Risk Factor Surveillance System (BRFSS), which has survey weights that allow the 1158 respondents in the Seattle Metropolitan Region to represent over 400,000 people age 65+ in our entire region. To compare the samples, find measures available in both the BRFSS and ACT. Demographics, self-reported health, self- reported ADLs.

Strategy Strategy Construct weights which make ACT look more like the Seattle Metropolitan Region (BRFSS). Demonstrate the use of these weights in transporting the prevalence of common eye diseases and also their effect on Alzheimer s disease incidence.

Stata files https://github.com/gibbonsL/ACT_transport (overview in Do_file_directory.do in GitHub) Code for data preparation and the alignment of the 2 datasets is in that folder.

Harmonized measures Harmonized measures ACT baseline (1994 to 2020) and 2019 BRFSS Demographics: age (grouped), sex, education, race, Hispanic ethnicity, marital status, employment status. Health (self-report): smoking, current hypertension medication, current cholesterol medication, history of heart disease, stroke, asthma, COPD or emphysema, diabetes, non-melanoma cancer, or osteoarthritis, self-perceived general health. ADLs (self-report): any difficulty dressing or bathing, any difficulty walking or climbing stairs.

Missing data Missing data Cumulatively 12% of BRFSS and 15% of ACT participants were missing at least one variable. Multiple imputation analyses (40 imputations) so that we could make weights for all participants.

impute_missing_data_by_group.do mi register regular id act ageg5yr sex xmmsawt // no missingness mi register imputed ed3 race3 hispanic marry work smoke bpcurr cholcurr mi heart_dis stroke asthma copd diabetes cancer osteoa ratehealth diffdres diffwalk // variables with missing data mi impute chained (ologit) ed3 smoke ratehealth (mlogit) race3 marry work /// (logit) hispanic bpcurr cholcurr mi heart_dis stroke asthma copd diabetes cancer osteoa diffdres diffwalk = ageg5yr sex /// , add(40) rseed(2466755) force by(act) augment // augment finesses perfect prediction of something in BRFSS /// /// ///

Participation weights Participation weights Weights: Winsorized, stabilized inverse odds of participation for all ACT participants.

weights_with_imputed_data.do mi svyset [pweight=xmmsawt] mi estimate, saving(temp3, replace): svy: logit act i.ageg5yr sex i.ed3 i.race3 hispanic i.marry i.work i.smoke bpcurr cholcurr mi heart_dis stroke asthma copd diabetes osteoa ratehealth diffdres diffwalk * default is linear prediction, not probability of a positive outcome mi predict xbbase using temp3 gen prbase=exp(xbbase)/(1+exp(xbbase)

weights_with_imputed_data.do mi svyset [pweight=xmmsawt] mi estimate, saving(temp3, replace): svy: logit act i.ageg5yr sex i.ed3 i.race3 hispanic i.marry i.work i.smoke bpcurr cholcurr mi heart_dis stroke asthma copd diabetes osteoa ratehealth diffdres diffwalk * default is linear prediction, not probability of a positive outcome mi predict xbbase using temp3 gen prbase=exp(xbbase)/(1+exp(xbbase))

weights_with_imputed_data.do, continued mi unset // no longer using mi - only wanted prbase on everyone gen iowbase=(1-prbase)/prbase // odds of being in ACT gen pract=5762/431701.1 // numerical probability of being in ACT. *stabilized inverse odds weight is IOW * odds of being in ACT: gen siowbase=iowbase*pract/(1-pract) *Winsorize . wsiowbase (Not standardized)

ACT baseline fairly similar to the 2019 BRFSS ACT baseline fairly similar to the 2019 BRFSS Without ACT participation weights More people with < HS education (cohort effect) More Whites More retired More past smokers Less likely to be on meds for hypertension and cholesterol Better self-reported health More likely to have difficulty walking or climbing stairs

Less than HS ed HS ed HS ed Standardized mean differences: Mean in ACT minus the mean in BRFSS, divided by the SD in BRFSS. A positive value means greater frequency in ACT. Cholest. meds Heart Dis. Heart Dis. Cholest. Meds Ideally all values between +/- 0.10. (For variables with more than 2 levels, each level is shown compared to all others.) ADL Walking/stairs

Participation weights Participation weights Unweighted ACT 8% Weighted ACT 1.3% Education level Less than high school BRFSS 1.9% High school College degree or higher 42% 49% 51% 48% 55% 43% Secular trend in education. Early ACT enrollees account for most of the < HS. Recent ACT enrollees have a higher level of education than in 2019 BRFSS. Note that we can get these further aligned with interaction terms in the model for the weights, but multiple imputation with interaction terms in Stata is biased, even/especially with mi passive . (Blimp)

Prevalence of common eye diseases eye_disease_prev.do Four common eye diseases from electronic medical records. drtodate = history of diabetic retinopathy. proportion drtodate // to get CI; unweighted svyset [pweight=wsiowbase] svy: tab drtodate, col // weighted

Selected eye disease findings Selected eye disease findings Transport results that can only be obtained in ACT to all older adults in the Seattle Metropolitan Region. Eye disease (EMR at baseline) Total Diabetic retinopathy ACT study (unweighted) Transported to the SeattleMR (weighted) 4.1% 5.6% Cataract history ACT study (unweighted) Transported to the SeattleMR (weighted) 48.6% 51.8%

longitudinal_eye_example.do // AD risk To get weighted point estimates: stset surv_age [pweight=wsiowbase], failure(anyad) origin(time 0) enter(age_bl) id(subj) Then run the regular Cox models.

Baseline eye disease history -> AD Hazard ratios (HR) and 95% confidence intervals (CI) for Alzheimer s disease in separate Cox models for time-dependent eye disease status within the ACT study (unweighted) and transported to the SeattleMR (weighted, but these are NOT the bootstrapped CI). All models adjusted for baseline age, years of education, and smoking history. To meet proportional hazards assumptions, we allowed the baseline hazards to differ by gender and by the presence of any APOE 4 alleles. Age is the time axis. Years with diagnosis Diabetic retinopathy HR (95% CI) p-value ACT study (unweighted) >0 5 >5 1.32 (0.87, 2.00) 1.39 (1.02, 1.88) 0.187 0.037 Transported to the SeattleMR (weighted) (but these are not the bootstrapped CI) >0 5 >5 1.84 (1.08, 3.14) 1.29 (0.87, 1.91) 0.025 0.201

bootstrap_CI.do For the weighted analyses, CIs were bootstrapped Error in estimating the weights Rubin s rules for combining variance across imputations. 1) Make 40 datasets, in the first: a) Keep complete data and _1_* b) Renaming _1_ed3 to ed3 , etc, for all variables 2) Merge in eye data

bootstrap_CI.do, continued 3) Bootstrap 500 times in each of the 40 datasets 1. Estimate the weights 2. Do the analyses 3. Output the desired parameters 4. Manipulate those parameters to the form we want 5. Append the 40 x 500 results 6. Take the 2.5thand 97.5thpercentile. 4) The resulting CI are wider than un-bootstrapped, the price we pay.

Sensitivity analyses Sensitivity analyses I calculated new weights, excluding ACT participants born before 1920 and thus not eligible to be in BRFSS 2019. The overall rate of cataract history was even higher. Diabetic retinopathy predicting AD: weighted and unweighted more similar than in full sample. I also computed new weights omitting the ADL variables because ADLs could possibly be on the causal pathway from eye disease to AD, but the AD results were almost identical.

Discussion Discussion On variables measured in both groups, differences between those in ACT and the general population of older adults in the region were mostly small in magnitude. (Only education and cholesterol meds > +/-0.25.) Still might affect the generalizability of some research findings.

Limitations Limitations Can t account for differences not measured in both studies. Secular trends in education, medical care. ACT had health insurance now and (mostly) in the past. BRFSS eligible for Medicare, but may have lesser coverage, or earlier health care may not have been as good.

When do we have to do this? When do we have to do this? Prevalence estimates for sure. Causal questions: If we suspect that the relationship may be confounded or moderated by measured differences between the study sample and the target population. Future studies should gather data in a way that aligns with variables collected in population-representative surveys so that better participation weights can be constructed. As more research is transported, we will have a better sense of how often and for what kind of questions this matters.

Conclusions Conclusions Generalizing research on older adults to the target population may require the construction of analytic weights. (As well as careful consideration of secular trends)

Thank you Coauthors: UCLA transport methods experts Taylor Mobley, Elizabeth Rose Mayeda, and Eleanor Hayes-Larson UW Eye expert Cecilia S. Lee ACT MPIs Andrea LaCroix, Linda McEvoy, and Paul K. Crane Funding: U19AG066567, U01AG006781, R56AG069126, P50AG05136, R01AG060942, 5R13AG030995.

Selected questions from the talk Q: Did you use the BRFSS survey weights in the multiple imputation? I did not, based on the advice of my transport experts. I later took a missing data short course and the instructor thought I should. However, I don t think this is an option in Stata and I have not been able to find existing software that can handle my particular data. Let me know if you have ideas! Q: Why did you impute first and then bootstrap in that data, rather than the other way around? Only because it s quicker to run. See Schomaker & Heumann (https://pubmed.ncbi.nlm.nih.gov/29682776/ ) for more on this. They find both orders to be valid.