Understanding Experiments and Threats to Validity
Explore the world of experimental research, internal and external validity, threats to validity, and the importance of proper study design in ensuring the credibility of research results. Learn about confounding variables, threats to internal validity such as environmental factors and group comparisons, as well as threats over time like history and instrumentation changes. Dive into testing effects, carry-over effects, and experimental mortality affecting internal validity, along with challenges in generalizing findings across participants due to selection bias and other external validity threats.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
EXPERIMENTS & THREATS TO VALIDITY
Overview Experimental Research Definition and terms Characteristics Types Definition of internal & external validity Threats to internal validity Threats to external validity Threats to both
Validity (of a study) Internal validity Is there only one unambiguous explanation? Have you eliminated confounds? Can the changes in the DV be attributed to the manipulation of IV? External validity Do the results generalize? Can the results apply to populations, situations, measurement procedures which were not part of original study?
Threats to Internal Validity Confound variables 1. Threats for all studies: Environmental variables E.g., Time of day, room size, room tempt, noise 2. Threats when comparing groups Assignment bias (e.g., non-equivalent control group) Groups are not equal (e.g., personality characteristics or experiences)
Threats to Internal Validity 3. Threats over time History Outside events occur Maturation Systematic changes in people (e.g., age) Instrumentation Changes in measurement, scales, or criteria Diffusion of treatment Communication across groups can interfere with manipulation of IV
Threats to Internal Validity 3. Threats over time (continued) Testing effects (can happen outside within-subjects designs too) Changes due to testing Carry-over effects: fatigue, practice, and context effects Statistical regression Regression towards the mean extreme scores become less extreme Experimental Mortality (i.e., attrition) Differential dropout rates between groups
Threats to External Validity 1. Generalizing across participants Selection bias Bias in sampling procedure College students Volunteer bias Participant characteristics Cross-species generalization
Threats to External Validity 2. Generalizing across features of study Novelty effects Do participants respond differently in the lab? Multiple treatment effects Did participation in earlier treatment affect outcomes? Experimenter characteristics Would we get the same results if we used a different experimenter?
Threats to External Validity 3. Generalization across features of measures Sensitization Assessment influences participants Makes them more aware of thoughts, feelings, or behavior Time of measurement Would we get the same result if we measured 1 week after intervention vs. 5 weeks?
Balancing Internal & External Validity We want both, but usually a trade-off Higher internal validity associated w/ greater control Greater control associated w/ lower external validity We have to decide what s most important Depends on purpose of study Another reason why it is the accumulation of information that is important! (i.e., multiple studies)
Threats to internal & external validity Experimental bias (i.e., expectancy effect) Experimenter expectations influence results Not necessarily intentional
Threats to internal & external validity Reactivity Participants change behavior because they know they are participating in a study Demand characteristics Cues make hypothesis obvious Influence participants responses/behavior (e.g., subject effect) The good participant Apprehensive attitude The negative participant
Reducing bias, reactivity & other issues Standardize or automate experiment Blind Single- or double-blind studies Conducting a preliminary study
Reducing bias, reactivity, & other issues Cover stories Add other unrelated DVs Use involving activities Habituation Indirect measures Using placebos
