Strategies to Boost Teen Participation in Web-Push Surveys

Exploring effective methods to engage teenagers in web-push surveys and address declining adolescent interview rates due to parental permission challenges. Insights from the California Health Interview Survey (CHIS) reveal the importance of obtaining parent permission and fostering teen cooperation to improve data collection among adolescents.

• Teen participation
• Web-push surveys
• Parental permission
• CHIS
• Adolescent interviews

hevi_9 Follow

Uploaded on Oct 06, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Finding effective ways to encourage teens to participate in a web-push survey (and getting their parents to let them) Brian M. Wells1, Todd Hughes1, Royce Park1, Kathy Langdale2, Suzanne Ryan-Ibarra3 1UCLA Center for Health Policy Research 2SSRS 3Public Health Institute ESRA 2019 17 July 2019

2. Special thanks Funding for this research provided by: Kaiser Permanente California Department of Health Care Services Guidance and consultation from the CHIS Redesign Working Group Kyli Gallington and Rebecca Garrow from the Public Health Institute for their assistance with our focus groups 2

3. California Health Interview Survey (CHIS) Main Objectives California s assessment tool to meet state and local needs for population-based health data Provide Health and Health-related Estimates 1) at local-level for counties, cities with health departments, and statewide 2) for adults, teens, and children 3) for California s major race/ethnic groups and (if possible) some smaller ethnic groups 4) to wide audiences 3

4. How is CHIS currently conducted? Through 2018 used RDD sampling (landline and cell) and Computer-Assisted Telephone Interview (CATI) CHIS collects detailed information for: One adult (age 18+) in the household One adolescent (age 12-17) if present, and One child (age 0-11) if present (by parent proxy) Sample Teen Interview (with Parent s Permission) Sample Child Interview (by Parent Proxy) Screener Interview Sample Adult Interview 4

5. Teen permission vs. cooperation While CHIS collects 20,000 adult interviews annually, the number of adolescent interviews is dwindling Primarily due to declining permission rates However, CATI benefits from the parent handing the phone to selected teen after adult interview once permission obtained Key to high teen cooperation rate 5

6. Declines in teen interviews Adolescent permission and cooperation rates 100 94.9 90 84.8 83.9 83.7 81.0 80 73.6 70 61.2 60.7 58.6 57.8 60 Percent 49.6 50 45.1 40 43.6 43.6 42.1 26.1 30 33.6 20 18.8 10 0 2007 2009 2011-2012 2013-2014 2015-2016 2017-2018* Permission rate Cooperation rate Completion rate 6

7. Research questions How do we get from a parent providing permission for a teen on the web to the teen completing a survey on the web? What contact strategy works best for a push-to-web survey of teens from an ABS household survey? Can the telephone hand-off effect be replicated within a push-to-web survey? What design elements can be included in an ABS push- to-web survey to better incentivize parents to provide permission for their teens? 7

8. Two experiments: Overall design Spring 2018 Feasibility Test ABS mail push-to-web w/ CATI NRFU April June 2018 3 purposively selected counties (~250 adult interviews per county) English web instrument Fall 2018 Statewide Pilot ABS mail push-to-web w/ CATI NRFU October December 2018 Statewide, ~10% of annual CHIS sample (~2,500 adult interviews) English and Spanish web instrument 8

9. New CHIS Data Collection Approach for Adults Phase 2: Telephone Nonresponse Follow-up Phase 1: Push-to-Web 3 mailings to sample addresses Call center staff call addresses with available telephone numbers up to 6 times Ask 1 sample adult to respond... Call in to complete by phone Online or 9

10. Adult mail materials 1) Initial mail invitation No greeting on the front of the envelope English invitation letter and FAQs $2 bill pre-incentive Multilingual letter (Spanish, Chinese, Korean, Vietnamese, Tagalog) 2) Pressure-sealed postcard 3) Final invitation letter 10

11. Two experiments: Teen design Spring 2018 Feasibility Test Permission request in middle of adult survey No incentive Single page consent script Multi-mode contact strategy Text/SMS, email, mail, CATI Ask for teen s phone number Emphasize text invitation Results were disappointing, therefore we redesigned the approach for the Fall pilot. Fall 2018 Statewide Pilot Permission request in middle of adult survey $10 conditional incentive Partitioned consent script Reduced multi-mode contact strategy Mail, text/SMS, CATI Ask for best phone number Text reminder only if best number = teen s mobile number Nested mailing (Matthews et al., 2017) 11

12. Teen multi-mode contact strategy Invite Dear teen Invite Spring test (if available) (if available) Text/SMS Email Mail Phone Dear parent Invite Dear teen Fall test Dear teen (if available) Mail Text/SMS Phone Mail (nested) 12

13. Parent focus groups Conducted in September 2018 by PHI Key findings included: Parents preferred a partitioned consent script Doesn t overwhelm them with information Broken down by topic, pop-ups for specific questions Incentives (specifically a gift card) are an important motivator for teens to participate Parents want more information on how their teens data will be used, and support available for their teen 13

14. Parental permission experiments For random half-sample, we offered the parent a $10 incentive for permission to survey their teen Permission refusal conversion letter mailed if permission was not obtained on the web Letter mailed to parent along with sealed envelope for teen (just like when permission was received) Offered $10 increase in parental incentive (see table below) Original parent incentive Refusal conversion incentive No incentive $10 gift card $10 check $20 gift card 14

15. Teen results (pre-refusal conversion) Fall test approach led to significant increase in permission rates and overall completes compared to spring test The rest of this presentation focuses on the fall test Final fall response rate equivalent to CHIS 2017 Traded permission rate for cooperation rate Eligible Teen Teen Permission Rate CHIS 2017 Permission Teen Completion Rate Weighted RR CHIS 2017 RR Teen Permission Completes Spring (3 counties) 125 38 30.4% 12 31.6% 14.0% 26.3% 23.4% Fall (Statewide)1 297 151 50.8% 72 47.7% 23.9% 15 1 These counts and rates do not include permission refusal conversion efforts.

16. Parental permission incentive results Parental incentive lead to virtually no difference in teen permission before refusal conversion Looking at subgroups, parental incentive reduced permission for single parent and foreign-born households Incentive Experiment1 Eligible Teen Teen Permission Rate Completes/ Permission Completes/ Eligible Completes Permission No Incentive 130 69 33 53.1% 47.8% 25.4% $10 Incentive 137 73 38 53.3% 53.3% 27.7% 1 These counts and rates do not include permission refusal conversion efforts. 16

17. Permission refusal conversion results When we include permission refusal conversion, we see $20 incentive resulted in more refusal conversions than $10 incentive (p 0.05) Nominal increases in permission, completion, and response rate, but not statistically significant No NRFU Completes Completes/ NRFU Teens Incentive Experiment Permission $10 NRFU Incentive 61 3 4.9% $20 NRFU Incentive 64 10 15.6% Teen Permission Rate Completion Rate Weighted RR1 CHIS 2017 RR2 Teen Web Phone Completes Permission Pre-NRFU 151 71 1 72 50.8% 47.7% 23.9% 23.4% 17 Post-NRFU 164 84 1 85 55.2% 51.8% 27.8%

18. Other findings Examined a logistic regression predicting initial permission in the fall test Significant findings: Fathers significantly less likely to give permission for younger teens (age 12-14) than older teens (controlling for gender) Mothers significantly more likely to give permission for younger teens (age 12-14) than older teens (controlling for gender) Mothers also more likely to give permission for daughters than sons (controlling for age) 18

19. Discussion and conclusions Text/SMS and email contacts were not directly effective methods for recruiting teens Teen incentive and nested mailing (i.e., parental hand-off) showed great improvements Parental incentive discouraged key groups from providing initial permission $20 parental incentive for refusal conversion more effective than $10 (conditional on original incentive) Consider how to target permission materials to specific parent/child characteristic interactions 19

20. References Matthews, P., Fahliogullari, S., & Bell, E. (2017, July). Successfully surveying young people online. Paper presented at the European Survey Research Association Conference, Lisbon, Portugal. Wells, B. (2018, May). When design changes don t pan out: The unexpected decline in child and adolescent interviews in the California Health Interview Survey. Poster presented at the 73rd Annual Conference of the American Association for Public Opinion Research, Denver, CO. 20

21. Thank you! Todd Hughes toddhughes@ucla.edu The full report is available at: healthpolicy.ucla.edu/chis/design/Pages/data- quality9.aspx 21