Web Survey Design Standards at U.S. Census Bureau

Slide Note

Explore the comprehensive efforts and research initiatives focused on developing web survey design standards for the Enterprise Data Collection System at the U.S. Census Bureau. From collaborative programs like DICE to discussions on linear navigation, edit validations, and web branding, this session delves into enhancing data quality and reducing respondent burden through empirically-based approaches and innovative design strategies.

bel_mic Follow

Uploaded on Sep 18, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript

FCSM Session: Web Survey Design Standards for the Enterprise Data Collection System at the U.S. Census Bureau Jean E. Fox Bureau of Labor Statistics Discussant FCSM October 27, 2022 1 U.S. BUREAU OF LABOR STATISTICS bls.gov

Wow! Huge effort Exciting research Great to have all this together Often work that is put off Too difficult Too minor HIPPO decisions Ambitious schedule 80 surveys in 8 years? 2 U.S. BUREAU OF LABOR STATISTICS bls.gov

Overview of DICE - Miller Data Ingest and Collection for the Enterprise (DICE) program Vision: To provide consistent, cost-effective, efficient, standard, and scalable data collection operations and technical solutions that support all directorate needs Impressive collaboration and agreement Focus of this session is on the Questionnaire Design and Metadata System (QDM) and Electronic Data Collection Good framework for the session 3 U.S. BUREAU OF LABOR STATISTICS bls.gov

DICE Web Standards Team Mission Develop empirically-based web survey design standards that maximize data quality and minimize respondent burden Research: We programmed 9 separate experiments using Qualtrics to test 3 to 5 different design issues in each one. Outcome Design standards that can be implemented for all Census Bureau web surveys 4 U.S. BUREAU OF LABOR STATISTICS bls.gov

Linear Navigation - Nichols Often overlooked design components Nice experimental design to get people to answer a question and select a button Recommendation seems logical and consistent with modern web design 5 U.S. BUREAU OF LABOR STATISTICS bls.gov

Edit Validations - Feuer Generally hard to study edits (they re not always triggered) Nice idea to have participants say what they would do if they saw the message. Good recommendation to go with the USWDS guidelines Looking forward to their next study. 6 U.S. BUREAU OF LABOR STATISTICS bls.gov

Web Branding - Katz Great use of the heat maps Next step might be to see if people can find the features Seems likely, but with banner blindness people may miss them Would like more information about the reasons for preferring the people version vs the logo only design. It might be about placement of the logo. Curious about the evolution from the preferred people wrapper to the final example It s quite a bit different 7 U.S. BUREAU OF LABOR STATISTICS bls.gov

Progress Indicators - Horwitz Studies on progress indicators Researchers: Don t use progress indicators. Survey owners: Let s use progress indicators. What does it mean that progress indicators don t work? Are respondent annoyed? Or do they drop off? Recommendation is a good compromise 8 U.S. BUREAU OF LABOR STATISTICS bls.gov

Personalizing Questions Olmsted-Hawala Allowing people to select the timeframe to report Great value in this approach I d hoped the final design would have done better Can edits warn of some possible frequency mismatch? Confirmation of annual rate seems helpful Especially when the frequency is missing 9 U.S. BUREAU OF LABOR STATISTICS bls.gov

Inputting Monetary Values - Holland Good approach to testing so you know the correct response Challenging results to make a recommendation No clear story Recommendation to use the formatting outside the field seems reasonable May be worth doing some moderated studies next to explore: Why the overall accuracies were all fairly low (<50%) Get more insight into what made the outside the field formatting easier. 10 U.S. BUREAU OF LABOR STATISTICS bls.gov

Looking to the Future Explore differences for cell phone users Publish and Present Your guidelines Journal article bringing all this together With more information about methodologies and sample sizes And about the quality of the participants recruited through Qualtrics How implementation goes Do you have to make exceptions to the guidelines? Share more research and progress on the whole DICE program 11 U.S. BUREAU OF LABOR STATISTICS bls.gov