Biases in Measurement of Thoughts, Feelings, and Behaviors: Current Research Insights
Current research explores biases in measuring thoughts, feelings, and behaviors, focusing on the attenuation effect observed in subsequent reports. This effect, illustrated through various studies and examples, highlights a decrease in reported symptoms over time, potentially influenced by measurement reactivity, initial awareness, therapeutic expression, and conversational norms.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Current Research on Biases in the Measurement of Thoughts, Feelings, and Behaviors SEAN P. LANE, PH.D. DEPARTMENT OF PSYCHOLOGICAL SCIENCES ADVANCED METHODS AT PURDUE (AMAP) BROWN BAG MARCH 19, 2018
Overview 1) The phenomenon of interest 2) First set of studies Experimentally establishing the effect Exploring two possible mechanisms 3) New analyses Additional measures Possible physiological mechanism 4) Methodological Implications Correlational vs. experimental studies Simulations 5) Conclusions (i.e., what next?)
The Phenomenon of Interest The attenuation effect in second interviews respondents frequently fail to report [lifetime] symptoms that they reported in the first interview. The strange result is that the proportion of life-time cases at re-interview seems to shrink if the second interview rather than the first is used to calculate prevalences (p. 919, Robins, 1985) Interpretation there is an inexplicable tendency for cumulative reports to decrease over time
The Phenomenon of Interest But, also Such a decline between first and follow-up interviews in the number of current symptoms, even in the absence of treatment, has long been noted Also, localized, momentary reports decrease over time (c.f., regression to the mean)
Sharpe & Gilbert (1998) 1-week follow-up scores much lower than initial reports
Knowles et al. (1995) Within-test improvement in a matter of minutes Scores larger as a function of: Randomized serial position Initial vs. 2-week retest
Other Examples Windle (1954) Meta-analysis of personality measures Perkins & Goldberg (1964) MMPI-Depression Jensen et al. (1999) Depression/ADHD Piacentini et al. (1999) Internalizing & Externalizing symptoms Parent & Child
The Attenuation Effect Implied decrease in subsequent reports Largely found in reports of negatively valenced thoughts and feelings Less consistent with respect to behaviors Mechanism(s)? Measurement reactivity (French et al., 2010) Initial awareness (Silvia, 2002) Therapeutic expression (Pennebaker, 1997) Conversational norms (Schwarz, 1995)
The Attenuation Effect Experimentally document the attenuation effect Clarify if it is actually an initial elevation or later decline Include subjective reports & objective behaviors Include positive outcomes Control timing Mechanism(s) Phenomenological awareness/reactivity Conversational only reporting new information
Study 1 Bar Exam 393 law school graduates and romantic partners Graduate preparing for bar exam 35 days before exam, 2 day of, 7 days after Proportionately randomly assigned to four conditions Diary (44 days), Day-22 only, Day-35 only, Day-44 only Complete morning (AM) and evening (PM) measures Positive (vigorous) and negative (anxious) mood Physical symptoms (headache, couch, cold, etc.) Study time (examinee only)
Bar Exam Results Examinee Anxiety Group 1 higher on Day 1 than comparable Day 8 Day 22 Group 1/2 difference Day 35 no Group 1/3 difference Day 44 - Group 1/4 difference
Bar Exam Results * * * Consistent anxiety & symptom initial elevations
Study 2 Exam Prep 228 premed undergraduate students Preparing for important midterm exam Staggered randomized diary design Week before and week after Day (Exam Centered) Group 1 (n = 64) Group 2 (n = 22) Group 3 (n = 26) Group 4 (n = 27) Group 5 (n = 27) Group 6 (n = 25) Group 7 (n = 25) Total N = 218 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10 11 Exam Day
Study 2 Exam Prep Complete morning (AM) and evening (PM) measures Positive (vigorous) and negative (anxious) mood (McNair et al., 1971) Physical symptoms (headache, couch, cold, etc.) Study time
Exam Prep Study Results * * Morning anxiety & symptom initial elevations Also, evening vigor & studying initial elevations
Study 3 College Life 870 undergraduates 85% recruited as roommate pairs 4 time point panel study Randomly assigned to 3 groups October December February April Group 1 X X X X Group 2 X X X Group 3 X X Roommates were randomly assigned to report on themselves or their roommates first
Study 3 College Life Complete bimonthly measures Self and roommate Positive (vigorous) and negative (anxious) mood Physical symptoms (headache, couch, cold, etc.) K10 mental distress screener (Kessler et al., 2002) Roommate measures included to test potential phenomenological mechanism If so, roommates should not show the effect
College Life Study Results Anxiety, vigor, & symptom initial elevations Also, self and roommate mental distress effects Phenomenological mechanism not clearly supported
Study 4 Exam Prep 2 141 undergraduate students Preparing for midterm exam 2 conditions (test conversational norm mechanism) 1-study condition 2 weeks of daily diaries 2-study condition 1 week of diaries for one study 1 week of diaries for another study Told first study was currently full and could participate in separate study in the meantime until spots opened in one week
Study 4 Exam Prep 2 2-study condition split into two groups 2a completed both studies 2b did not enroll in first study, waited for second Complete evening (PM) measures Positive (vigorous) and negative (anxious) mood Physical symptoms (headache, couch, cold, etc.) Study time
Exam Prep Study 2 Results 1-study and 2a-study conditions did not differ Lack of evidence for conversational norm mechanism Replicate anxiety and symptom initial elevations between 1-study and 2b-study conditions
New Analyses Study 2 (Exam Prep) & Study 3 (College Life) Additional measures Other moods Other mental health measures Other behaviors Additional mechanism Physiological
Study 2 Exam Prep Additional measures Depression, anger, fatigue, low arousal positive mood (McNair et al., 1971) Alcohol urges (Bohn et al., 1995) Alcohol frequency and duration (DQFI; Straus & Bacon, 1953)
Study 2 Exam Prep ** * *** *** *** * ** * ** * *
Study 3 College Life Additional measures Mental health screen (PHQ9; Kroenke et al., 2001) Depression symptoms (CESD; Radloff, 1977) Anxiety symptoms (Zung, 1971) Moods depression Expected effects of alcohol (CEOA; Fromme et al., 1993) Desired effects of alcohol (SAML; Simpson et al., 2004) Drinking problems (RAPI; White & Labouvie, 1989) Drinking amoung/frequency (AUDIT; Saunders et al., 1993)
College Life Study Additional Results Mental Health & Mood Initial elevation for screener measures
College Life Study Additional Results Alcohol Initial elevation for expectancies and problems, but desired effects or actual behavior
Another Possible Mechanism Physiological salivary cortisol Is cortisol elevated at the first assessment compared to others? Morning peak Overall trajectory Study 2 (Exam Prep) Subsample of 80 individuals
Initial Elevation in Cortisol Salivary cortisol assessment four times per day At waking, 30 after waking, prior to lunch, prior to bed Day (Exam Centered) Group 1 (n = 64) Group 2 (n = 22) Group 3 (n = 26) Group 4 (n = 27) Group 5 (n = 27) Group 6 (n = 25) Group 7 (n = 25) Total N = 218 -8 -7 -6 -5 -4 -3 -2 -1 c 0 1 2 3 c 4 5 c 6 7 8 9 10 11 c c c c c c c c c c c c c c c c c c c Exam Day
Diurnal Cortisol Results No differences in: Peak (AM) Trajectory First assessments do not appear to: Induce reactivity Evoke emotional reappraisal Follow-up assessments do not appear to be comparatively therapeutic 0 = Matched Follow-up Days 1 = First Days
Discussion The attenuation effect does not appear to be attenuation, but rather an initial elevation Observed in all four studies Studies 1, 2, & 4 decreases in anxiety despite an approaching exam Particularly notable in Study 1 Group 4 s anxiety post exam Not just negative reports, but positive also Not just internal states, but also behaviors (to an extent) Pattern holds across: Designs intensive/panel, within/between Contexts acute/chronic stress, self/other
Discussion Minimal support for potential mechanisms Reactivity (Study 2 cortisol) Therapeutic process (Study 3) Conversational norms (Study 4) But should not rule them out Other mechanisms Learning (practice) familiarity might lead to less extreme reporting Meaning-making meaning of responses is changed based on those previously reported
Implications Initial elevation bias is: Likely pervasive across many types of assessments Problematic for both cross-sectional and longitudinal designs Cross-sectional First report is the only report Longitudinal Trajectories are contaminated Problematic for both correlational and experimental designs Correlational False conclusions following national/world events Experimental Biased and underpowered effects Cutoffs for screening measures likely committing many false- positives
Why it is an Issue Even in Experiments XB YB X Y YT XT
Why it is an Issue Even in Experiments Correlation between elevation biases, r = 0.80 Correlation between elevation biases, r = 0.00
Conclusions Awareness of initial elevation bias is the first step Can adjust for it analytically if planned for in the study design More study is needed to understand what causes it And potentially how to experimentally mitigate it (e.g., question framing) Experiments are not immune
Thank You! Collaborators Pat Shrout Niall Bolger Turu Stadler Joy McClure Grace Jackson Joy Xu Fred Clavel Marci Gleason Masumi Iida NYU & Columbia Couples Labs Grant #R01-AA017672
