Challenges in Life Satisfaction Scale Assessments
Statistical analyses often assume linear scale, interpersonal comparability, and intertemporal comparability, challenging the reporting function in life satisfaction assessments. Cognitive interviews with UK residents shed light on issues like priming and representativeness.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Evidence against the simple validity of life satisfaction scales from cognitive interviews Mark Fabian, Caspar Kaiser, Sofia Panasiuk, Sabina Funk, Caroline Brett IARIW, London, 2024 Presented by Arthur Grimes (Motu Research & Victoria University of Wellington) 1
The challenge Statistical analyses of life satisfaction data often assume: 1) linear scale (though some do not e.g. use of GOL) 2) interpersonal comparability 3) intertemporal comparability To test these assumptions, we need to understand the reporting function: process by which individuals make life evaluations & map them to a response scale Investigates: scope (domains), the ranking problem, calibration of response Challenges include: adaptation, scale norming, ceiling & floor effects, Is the focus micro or macro (aggregated) applications? concerns may differ 2
The reporting function (Oswald 08) Life Satisfaction Assessments 45o Scale Responses 3
The reporting function - formally ???represents an ordered vector of possible lives ? for individual ? at time ? is a respondent s actual life ? ?? ? ?? = {1,2, ,?, ,?} denotes the life satisfaction response category , relative to ???, into ? The reporting function, ? ??, maps ? ??: ?? ;??? ? ??= ? ??? ?? 4
Cognitive interviewing - process 100 UK residents (93 completed) Objective is conceptual saturation - diverse sample, not representative Recruitment is purposive & snowballing Non-representative sample very useful for raising issues, but not for answering how widespread or concerning they are comments imply a sample with higher-than-average education ( asymptotic , domains , linear scale , my sense of agency , locus of control , ) Issues of priming? (for answers in discussion, not for life satisfaction) 5
Scope and ranking problem Scope: Time frame varies across people & life-stage Aspects of life broadly rational Ranking: how to weight different life aspects most are capable of doing so 6
Calibration E.g. strict vs generous graders 7
Adaptation Involves real changes in feelings/evaluations E.g. recovery from divorce 8
Scale norming Observed changes in response category reflect changes in reporting style not changes in life satisfaction Also Reference point effects: A change in response style that leads to a change in feelings - e.g. gratitude (positive effect) or being newly classed as a disadvantaged minority (negative) 9
Scope, ranking & calibration Randomly alternate between ONS life satisfaction question/Cantril Ladder What is the distribution of life satisfaction in the sample? Misses opportunity to compare the two measures (to come??) What determines your life satisfaction? (Discussion) Do emotions/events affect the answer (Mostly no) Time frame considered? (Varied: Immediate to whole life) Consider family & society or self only? (Mostly self; some wider, especially family) Meaning of satisfied & best/worst possible (Mostly realistic) hat does x/10, y/10, mean to you? (Varied answers) 10
Scale use is rarely linear (does it differ by starting point?) 11
Difficult & existential questions It s like, Whoa, is it a 5, is it a 6, 7? But if I m really honest, I think what could make my life better? I don t actually know, Mark. I really have no clue. [Ceiling effect: person answered <10] - are ceiling effects real? Excess 10 phenomenon (Brul & Veenhoven, SIR, 2017) I don t go about life thinking of it in terms of a scale and where I am on that scale and I don t think I d want to. It s tricky, because having to put a number to it, to me it s a sort of an alien thing. We can t measure satisfaction by numbers. What s the point of answering that question? It s sort of meaningless. 12
Macro vs micro level validity? Source: World Happiness Report, 2018 (Gallup Poll Cantril ladder)
Implications UK average life satisfaction = 7.7 (ONS) Can it rise much higher? Ceiling effects (if they exist) suggest not by much Could artificially favour policies that have greatest effects on people with low S But that would be consistent with most SWFs (between utilitarian & Rawlsian) Raises issues about using LS as a diagnostic tool for individuals Is this a problem of bias or noise? Supplement with multi-dimensional SWB measures in same survey (WHO-5, P Q9, )? 14
Further thoughts Are these problems for analysing macro-level policies (eg funding mental health)? Perhaps the heterogeneity washes out at the macro level (as per macroeconomics) Complicated further by lack of representative sample in this study (future work) Use other methods to test if identified issues matter: Use GOL to test linearity of impacts of covariates on LS And test for consistency across different population groups (including by age, and other covariates) Use separate life satisfaction steps to predict actions (eg migration): test linearity, homogeneity Use longitudinal data to test for adaptation & scale-norming when LS is used as a predictor 15
Final thought The paper makes us think and question, as all good papers should do ! THANKS FOR AN INSIGHTFUL & ENJOYABLE READ 16
