Enhancing Data Quality through Administrative Records in American Community Survey
This presentation discusses the potential of using administrative records to improve data quality and reduce respondent burden in the American Community Survey. It explores key priorities, research findings, data source evaluation, and the process of simulating housing estimates using administrative records. The presentation highlights the benefits of leveraging administrative records in the ACS data collection process and adaptive design strategies for integrating admin record values.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Operational Agility in the American Community Survey: The Promise of Administrative Records Victoria Velkoff and Jennifer Ortman American Community Survey Office U.S. Census Bureau April 10, 2018 Presentation for the 2018 State Data Centers Annual Training Conference 1
ACS Program Priorities Respondent Experience QUALITY Data Survey RESPONSIVE REPUTABLE RESEARCHED 2
Research to Ensure Success We are all in agreement that we can use administrative records to reduce respondent burden and improve data quality. What is the best way to do so? We are considering: Reliability Accuracy Consistency Quality Timeliness Reputable Responsive Researched 3
Evaluating Data Sources Telephone Service Condo Fees Mortgages NOT PROMISING PROMISING 4
Using Administrative Records to Simulate Housing Estimates Preliminary Research Findings 5
Background Previous research shows: Good match rate Promise for some housing items ACS perfect platform for simulating the use of Administrative Records (ARs) Huge sample Spans multiple geographic areas Covers many topics High profile Satisfy ACS program goals to be reputable, researched, and responsive 6
ACS Data Collection Process in 2015 Online Mail Telephone* Personal Visit Self-Response Modes Non-Response Modes *Terminated October 1, 2017 7
Adaptive Design Skip Question, Use Admin Record Value Present 1. Internet, CATI, or CAPI Mode Admin Record Value Present? For each question: Not Use Present Ask Standard Question Respondent Value Match Sample to Admin Records Draw Sample Use Admin Record Value if Available Blank Retrieve Completed Survey Question Left Blank? Edit each question: 2. Mail Survey Mode Not Blank Leave as Respondent Value 8
Simulating this Process Used 2015 ACS Responses Direct substitution of the four most promising housing items: Year Built Acreage Property Value Real Estate Taxes For housing units that have: ACS Internet, CATI, or CAPI return ACS mail return and ACS response is missing 9
Test vs. Control Estimates Calculated summary metrics and key measures Approximately 79% of 575 national-level estimates statistically different Direction of differences varied, but for many items Test (with AR) was lower than Control Test item allocation rates significantly lower than Control by: Acreage - 2.4% pts. Property Value - 6.8% pts. Year Built - 5.2% pts. Real Estate Tax - 12.5% pts. Impacts other survey items besides 4 test items 10
Burden Reduction Burden defined as receiving the survey question Using ARs greatly reduces respondent burden for all 4 items Able to skip question for the following percentage of ACS responders: Acreage - 52.0% Property Value - 47.2%. Year Built - 37.5% Real Estate Tax - 53.5% 11
Percent Difference in Median Property Value: Test minus Control States 12
Percent Difference in Median Property Value: Test minus Control Counties 13
Percent Difference in Median Property Value: Test minus Control Places 14
Burden Reduction Visualized Overall Match Rate 15
Burden Reduction Visualized Match Rate by County and Variable Geographic View 16
Challenges: Source of ARs Rely on outside vendor, CoreLogic (CL) Best source of property value data comes from CL s Automated Valuation Model (AVM) and the methodology is unknown Collection of AR data differs among jurisdictions Data not available for some areas and types of housing units Often a time lag between survey year and AR 17
Challenges: Incorporating AR in the ACS Sometime unable to match AR to ACS Adapting all modes difficult for mail mode Involves changes to entire survey life cycle Significant resources Risky without thorough testing Using AR may complicate data disclosure avoidance 18
Preliminary Conclusions Significantly reduces respondent burden in terms of asking survey items Significant improvement in item allocation rates Must overcome several challenges Source of AR Incorporating in ACS Admin data are different from ACS response data Break-in series Using two estimate sources - that are different and potentially unfairly represent some geographies Impacts other survey items (e.g., income) 19
