Using Behavioral Science to Improve Survey Participation: Experiment with National Beneficiary Survey
This study presented at the Disability Statistics Compendium Event explores the use of behavioral science to enhance survey participation in the National Beneficiary Survey conducted by the Social Security Administration. It discusses strategies such as incentives, advance letters, and leveraging behavioral economics concepts to address common bottlenecks like limited attention and forgetfulness, aiming to increase response rates from SSDI and SSI beneficiaries.
- Behavioral Science
- Survey Participation
- National Beneficiary Survey
- Disability Statistics
- Social Security Administration
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Using Behavioral Science to Improve Survey Participation: An Experiment with the National Beneficiary Survey Jason Markesich, Ryan Callahan, Jesse Chandler and Amy Johnson Presented at the 2017 Annual Disability Statistics Compendium Event Washington, DC February 13, 2018
About the National Beneficiary Survey (NBS) Sponsored by the Social Security Administration s (SSA s) Office of Retirement and Disability Policy Mathematica has conducted six rounds of the survey since the NBS began in 2004 Collects data on the employment-related activities of working-age Social Security Disability Insurance (SSDI) and Supplemental Security Income (SSI) beneficiaries
About the NBS (contd) Dual-mode administration: telephone interviewing with in-person locating and interviewing Methods to promote response include: Incentives ($30) Advance letters and reminder notifications Spanish language interviews Assistive technologies Assisted interviews and proxies
Growing Challenge: Reaching Sample Members and Gaining Cooperation Trend Metric (2004 to 2015) Call-in rate Completion rate Refusal rate Attempts/complete Attempts/non-complete Interviewer hours/complete
Strategy: Leverage Behavioral Science The study of how people make decisions in a complex world where making the right decision is often difficult Involves diagnosing and designing solutions to common behavioral bottlenecks: Limited attention Forgetfulness Procrastination
Could the Advance Letter be Contributing to Bottlenecks?
Comparing the Four Advance Letters Tested in the Experiment Letter Type Original Feature States that an interviewer will call soon Invites sample members to call in or call/email with questions Concrete Provides a clear action step. Call to: complete the interview, schedule a time to be interviewed, or decline participation Personally Relevant Incorporates language that explains how SSA would like you to tell us about yourself and your experiences with programs like SSDI and SSI Concrete and Personally Relevant Includes both the personable language and the action step mentioned above
Methods Randomly assigned 5,020 cases to one of four experiment groups Mailed the advance letters first class Did not conduct any outbound dialing for 8 days after letters mailed; accepted call-ins and tracked call-in outcomes during this timeframe
Concrete Letter Boosted Call-In Rates by More Than 50% Percent of Initial Contacts That Were Call-Ins 8.0% 7.2% 6.7% 7.0% 6.0% 5.3% 4.7% 5.0% 4.0% 3.0% 2.0% 1.0% 0.0% Original Concrete Personally Relevant Concrete and Personally Relevant
Concrete Letter Generated Highest Percentage of Completed Interviews for Call-Ins Call-In Outcomes 60% 56% 51% 48% 48% 50% 43% 39% 40% 36% 32% 30% 20% 14% 12% 10% 10% 5% 3% 2% 0% 0% 0% Original Concrete Personally Relevant Concrete and Personally Relevant Complete Appointment Refusal Other Note: Others include gatekeepers/family members who provided new contact information, and sample members who were ineligible for the interview.
Behavioral Science Tips to Improve Your Communications Apply a skim test Put your bottom-line up front Use simple, non-technical language Provide a clear, single next step or action for the reader to take
Contact Information Jason Markesich Center for Studying Disability Policy Mathematica Policy Research 600 Alexander Park Princeton, NJ 08540 (609) 275-2326 JMarkesich@mathematica-mpr.com http://www.DisabilityPolicyResearch.org
