Understanding and Addressing Survey Fatigue in Program Evaluation
Survey fatigue can lead to insufficient effort responding in program evaluations, affecting the quality of data collected. This presentation explores the causes and impacts of survey fatigue, suggests solutions such as shorter surveys and open-ended questions, and discusses alternatives to traditional survey methods for more effective evaluation practices.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Overcoming Death by Survey KIM STOKES, MHS, PA-C AMI STEELE, MS, PA-C
Learning objectives Learning Objectives: Recognize the role of survey fatigue in insufficient effort responding (IER) and the subsequent impact of IER on program evaluation data Explore ways of transforming program evaluation practices to obtain meaningful learner feedback Evaluate a variety of formal and/or informal survey methods/platforms for potential use in their programs Discover resources for formal and informal feedback for program evaluation
Why do we survey? Consider this: Consider this: How many surveys does your program do per cohort?
Insufficient Effort Responding Socially desirable responses ( faking ) Content responsive-respond in a way that over endorses negatives and positives (complaints, praises) Content non-responsitivity ignores the contents of the items, unwilling to comply with survey procedures, linguistic incompetence Acquiescence Careless or inattentive/random responding Furnham A, Hyde G, Trickey G. Personality and value correlates of careless and erratic questionnaire responses. Personality and Individual Differences. 2015;80:64-67. Nguyen HLT. Tired of Survey Fatigue? Insufficient Effort Responding Due to Survey Fatigue. ProQuest Dissertations Publishing; 2017.
Survey fatigue AUGMENTED BY POSSIBLE SOLUTIONS Long surveys Shorter surveys Difficult topics Meaningful/Purposeful timing Question complexity Technology Open-ended questions Alternatives to surveys O'Reilly-Shah VN. Factors influencing healthcare provider respondent fatigue answering a globally administered in-app survey. PeerJ PrePrints. 2017.
Alternatives to Traditional Surveys EXPLORATION AND DISCOVERY
Activity Activity Choose a table topic of interest (see labels on tables) As a group, brainstorm alternatives to the traditional survey for this evaluation OR if you already have a non-traditional method, share it with the group Names a spokesperson for summary Spokesperson will share with large group the topic of the table and alternatives discussed
Examples of Alternatives Examples of Alternatives QUANTITATIVE QUALITATIVE Point/Click technology Poll anywhere Minute papers (1-2 impressions) Focus groups Kazoo Debriefings Turning point Change open-ended stickers Minute papers (1-2 take aways) 3 good/3 bad Keep/Change/Eliminate
PROS No screens, but VERY visual Keep/Change/Eliminate Keep/Change/Eliminate Full participation (100%) Fun/interactive WITHOUT a SCREEN At a glance Suggested setting Avoid use with Themes/concepts pre-determined Workshop/event with up to 12 components Course evaluation (clin med, anatomy, etc.) Pre-populated items with quick responses. Quick and easy Allows for quick follow up process Interactive Program/course with sell-defined sections Full program evaluation (too many pieces) CONS Loss of long qualitative responses Carrying around large papers Quantitative Analysis Prep time to print labels Time to count
Focus groups/debriefs Focus groups/debriefs PROS No screens Personal Verbiage can be clarified with script for consistency Allows for immediate probing At a glance Suggested setting Avoid use with Full program evaluation Course evaluation (clin med, anatomy, etc.) Divided cohort answering common scripted questions that can be reviewed. End of curriculum (end pre-clinical, end clinical) CONS Possible long responses Time: transcript, analysis, theming Whole cohorts at one time Qualitative Analysis
Survey Topic Suggested Alternate Sharing YOUR Sharing YOUR examples! examples! (This slide will be completed based on discussions we hear around the room)
References Furnham A, Hyde G, Trickey G. Personality and value correlates of careless and erratic questionnaire responses. Personality and Individual Differences. 2015;80:64-67. Thank you for Thank you for coming to our coming to our presentation presentation and sharing and sharing your ideas! your ideas! Nguyen HLT.Tired of Survey Fatigue? Insufficient Effort Responding Due to Survey Fatigue.ProQuest Dissertations Publishing; 2017. O'Reilly-Shah VN. Factors influencing healthcare provider respondent fatigue answering a globally administered in-app survey. PeerJ PrePrints. 2017.
