Qualitative Approach in Cognitive Interview Data Analysis
Understanding the importance of a qualitative approach in analyzing cognitive interview data is crucial for uncovering patterns and recurring themes. This involves delving deeper into the why behind the responses rather than just naming and classifying data. Using methods like standardised coding schemes and qualitative analysis can shed light on cognitive processes during interviews, aiding in survey design and data interpretation.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Analysing and interpreting cognitive interview data: a qualitative approach
Presentation structure background aims and objective of QT design of cognitive interviews review of methods of analysis NatCen approach issues for discussion
Background aims and objectives of QT does test question meet measurement objectives if not, what problems arise implications of problems for survey design of cognitive interviews probe sheet interviewing techniques sampling strategy
Review of methods of analysis little written on analysis of cognitive interview data two main methods cited: standardised coding scheme qualitative analysis standardised coding scheme approaches are documented (usually) qualitative analysis descriptions are often scant on detail
Standardised coding schemes can be used to as stand alone question appraisal tool (e.g. QAS) can incorporate some elements of behaviour coding interviewer has a problem reading the question or recording the answer focus on cognitive Q & A model comprehension/communication recall/computation bias/sensitivity (judgement issues) response category plus logical issues
Issues with standardised schemes PROS lend themselves to presenting data quantitatively (x Rs had this problem) perceived (by some) to be more robust process is replicable useful in cross-national/ cross cultural settings, where standardisation important CONS need a lot of detailed codes under each main heading (particularly comprehension) time consuming loose context: why did R interpret Q in that way? lend themselves to presenting data quantitatively (x Rs had this problem)
Qualitative approach Just naming and classify what is out there is usually not enough. We need to understand the patterns, the recurrences, the whys. As Kaplan (1964) remarks, the bedrock of inquiry is the researcher s quest for repeatable regularities . Miles & Huberman (1994)
Approaches to qualitative analysis Ethnographic accounts detailed thick descriptions of cultures or organisations Life histories analysed as individual cases or mined for common themes Content analysis identifies content and context of documents, often involves counting (not strictly qual) Grounded theory generates analytical categories and the links between them through an iterative process of collecting and analysing data
Approaches to qualitative analysis Narrative analysis examines how a story is told and the intention of the teller Conversation analysis examines the structure of (usually) naturally occurring conversations Discourse analysis focuses on how knowledge is produced through the use of language Interpretative analysis attempts to present and re-present the world of those studied, by identifying and describing substantive themes, and searching for patterns between them
Key stages of the analytical process Data management identifying themes sorting and reducing data Generation of findings describing classifying finding linkages and patterns identifying explanations Characteristicsof good analysis system Remains grounded in the data Transparent data reduction process Facilitates and displays ordering Permits within and between case analysis
The analytical hierarchy in qualitative research Seeking wider applications Explanatory analysis Developing explanations Detecting patterns of association Establishing typologies Descriptive analysis Identifying elements & dimensions ? Summarising / synthesising data Sorting data Data management Tagging data Identifying initial concepts / themes Primary data Data collection
Thematic analysis - purposes and principles Structured display of data by theme (Q) across all cases Creating categories and classifying data within them* Demonstrating range and diversity Using examples to illustrate and amplify Must be comprehensive Labelling and categorising must be valid
Carrying out thematic analysis - NatCen approach Familiarisation with data Identification of factors highlight, summarise, provisionally label Categorisation is this a different manifestation of that is this a subset of that is this of the same order as that Iterative process of refinement Start close to the data - become more abstract and interpretative Must be comprehensive Aim is analytical coherence
Looking for explanations Informed by: hunches and hypotheses reflections during fieldwork and analysis other research or theories Process involves: detailed within case analysis comparison between cases repeated interrogation of the data milking data moving back and forth between cases searching for rival explanations Expect multiplicity Must be comprehensive
Summary of NatCen approach Detailed notes made on interviews Notes reviewed Chart set up Notes charted (chart revised) Charts reviewed Data interpreted Findings emerge Recommendations made Report written
Example chart Case details Q1 Emp status Q2 Job title Q3 No. hrs wrk last wk Q4 No. employees DC001, M, 52yrs, Self-employed Answer Comp Response Other issues DC002, F, 24, pt temporary worker & ft student DC003, F, 36, ft employee DC004, M 68, retired, self- employed (pt)
Issues to consider better documentation of qualitative analysis approach integration of code frames within thematic approach development of best practice for analysis of cognitive data cognitive interview data as one component of testing strategy collaborate findings using other data sources (e.g. split ballot experiments)
