The Basics of Experiments in Social Sciences

undefined
 
Stephen
Benard
Department of
Sociology
Indiana
University
 
LABORATORY
EXPERIMENTS IN THE
SOCIAL SCIENCES:
AN INTRODUCTION
 
A sample question
What is an experiment?
Basics of experimental design
What can we learn from experiments?
Ethics of experiments
 
OVERVIEW
 
Just a small sample of the:
Questions
Experimental designs
Independent and dependent variables
Many, many possibilities
AN (ENCOURAGING) DISCLAIMER
 
If we notice someone in need:
Are we more likely to help when alone, or in the
presence of others?
“Diffusion of responsibility”
SAMPLE QUESTION: WHAT PREDICTS
HELPING IN AN EMERGENCY?
 
ARE PEOPLE LESS LIKELY TO HELP
OTHERS WHEN IN A GROUP?
 
Challenging to study through observation
Emergency events are rare and hard to predict
May vary in countless ways
Many alternative explanations
People in groups less likely to notice
More groups at busier times of the day – less time
Unhelpful people more likely to travel in groups
 
Would be useful to repeatedly observe responses
to 
the same 
emergency under 
different 
condition
E.g., when many or few people observe the
emergency
Could be staged in a laboratory
(Darley and Latane 1968 in 
JPSP
)
Laboratory discussion group
One person appears to have a seizure
Manipulate number of people present
Measure proportion who helped, speed
STUDYING HELPING IN AN EXPERIMENT
 
Does violent media make people aggressive, or do aggressive
people prefer violent media (Bandura, Ross, and Ross 1961)?
Does intergroup 
contact
 reduce or exacerbate intergroup 
conflict
(Sherif 1958)?
Does positive mood make people more altruistic, or are more
altruistic people happier (Isen et al 1978)?
Do our attitudes determine our behavior, or does our behavior
determine our attitudes (Festinger and Carlsmith 1959)?
Does the gender/race/age/criminal record/other characteristic
of a job applicant affect the likelihood of being hired (e.g., Pager
2003)?
Is support for a policy determined by the content of the policy, or
the identity of the party supporting it (Cohen 2003)?
Does lack of control over our environment turn us into conspiracy
theorists (Whitson and Galinsky 2008)?
Does the status of an author’s institution affect their chances of
having an article accepted (Peters and Ceci 1982)?
A FEW MORE EXAMPLES
 
Identifying causes
Addressing alternative explanations
Identifying moderators and mediators
Examining hard-to-observe or rare events
WHY CONDUCT AN EXPERIMENT?
 
Manipulation of the independent variable
Random assignment to condition
Controlled measurement
WHAT IS AN EXPERIMENT?
THREE PRINCIPLES
 
In experiments you must 
manipulate
 an
independent variable (IV)
This creates 2 (or more) levels of the IV
The levels of the IV are called 
conditions
Conditions identical except for the
manipulated IV
E.g., number of people present when an
emergency occurs
MANIPULATION
 
How do we distinguish the effects of our IV
from extraneous variables?
Perhaps personal interest in helping others
confounded with group size
Experimenter places people into experimental
conditions by 
chance
Equal likelihood of being in each condition
Individual differences cancel out
RANDOM ASSIGNMENT
 
Before Random
Assignment
 
After Random
Assignment
Colors symbolize
any
 differentiating
attribute among
the individuals
(e.g., personal
interest in helping
others)
RANDOM ASSIGNMENT
 
Before choosing
 
Systematic
error
Colors symbolize
any
 differentiating
attribute among
the individuals
(e.g., personal
interest in helping
others)
WHAT IF PEOPLE CHOSE THEIR
CONDITION?
 
Systematically observe changes in the
dependent variable as a function of changes
in the independent variable
Important to avoid bias in recording the DM
Participant blind to hypotheses
Experimenter blind to hypotheses
Experimenter blind to condition
 
CONTROLLED MEASUREMENT
 
Experimental setting: a laboratory discussion
group
Simulate an emergency (seizure)
Manipulate number of other people present in
group
E.g., zero vs. three
Randomly assign participants to the “alone”
condition or the “group” condition
Measure proportion helping, time to help
A SIMPLIFIED HELPING STUDY
(BASED ON DARLEY AND LATANE 1968)
 
Two condition, treatment-control design
Similar to medical study with placebo
Simplest possible design
Often very effective, but also limited
Additional treatment conditions
Factorial designs
EXPERIMENTAL DESIGN
 
Perhaps group size has a non-linear effect
Add additional condition with 6 total group members
Sometimes it is useful to have a “baseline”
condition
E.g., a study of whether a text is evaluated more
positively when the author is a man vs. a woman
May wish to compare to a condition with no author
information
Is it that men receive a boost relative to the baseline,
or women receive a penalty?
ADDITIONAL TREATMENT CONDITIONS
 
Multiple IVs
Every combination
of every level of IV
Interaction effects
Predict an interaction
Or evaluate generality
FACTORIAL DESIGNS
A 2 x 2 Factorial Design
 
Between-subjects design
: Each participant is
exposed to 
one 
level of the independent
variable
E.g., study of helping
 
 
 
BETWEEN VS. WITHIN-SUBJECTS
DESIGNS
 
Within-subjects 
design: Each  participant
exposed to multiple levels of the dependent
variable
E.g. Text evaluation study
More efficient
But possibly easier to guess hypotheses
Requires 
counterbalancing
Rarely possible in high-impact designs
BETWEEN VS. WITHIN-SUBJECTS
DESIGNS
 
How do we know 
operational 
independent variable
accurately measures 
theoretical 
dependent variable?
E.g., positive mood
How do we know the manipulation had the expected
effect on participants?
Manipulation Checks
 
INDEPENDENT VARIABLES
 
DEPENDENT VARIABLES
(AND SOME BROAD GENERALIZATIONS)
 
High degree of control provides high 
internal validity
Experiments provide the strongest possible evidence
for causality
But, 
external validity 
of laboratory experiments is
often criticized
Settings don’t always resemble “real world”
Participants don’t resemble other populations
Samples are generally non-random
Small samples, at least by survey data standards
Participants are often college undergraduates
Participants are often WEIRD: Western, educated,
industrialized, rich, democratic
WHAT CAN WE LEARN FROM
EXPERIMENTS?
 
Mundane Realism: the extent to which an
experiment looks like the real world
Experimental realism: the extent to which
experience is psychologically real and
important to participants
Rarely come to a lab for a group discussion
 
MUNDANE VS EXPERIMENTAL REALISM
 
Should 
not 
generalize
directly from an
experiment to a real
world situation
Experiments test
theories
Theory bridges
empirical studies and
the real world
See Zelditch, 1969, “Can
you really study an army
in the laboratory”
GENERALIZING FROM…?
 
Criticism that findings
won’t generalize
Often explicitly or
implicitly signal possible
scope conditions
These can be tested to
further refine the
theory
 
Example: College
students discriminate
against women in hiring
simulation
Maybe more than real
managers: less
experience
Maybe less than real
managers: more
egalitarian
SCOPE CONDITIONS
 
Useful to think of different methods as
complementary, not competing
Survey data
May have high external validity, but limited ability to
show causality
Experiments
High internal validity, but limited generality
 
CONVERGENT VALIDITY
 
Three core principles for all research
Respect
Beneficience
Justice
Deception
Necessary to test some hypotheses
But should be used only as a last resort
And fully explained to participants
Debriefing
 
EXPERIMENTAL ETHICS:
 
Experiments are excellent for answering
questions about causality, exploring alternative
explanations, and examining rare or hard to
observe events
Many different types and approaches to
experiments, can (must) be tailored to the
research question
Facilitate systematic replication and theory
development
Strengths/weaknesses complement other
methods
 
SUMMARY
undefined
 
Stephen
Benard
Department
of Sociology
Indiana
University
 
THANK YOU!
Slide Note
Embed
Share

Delve into the world of experiments in social sciences with a focus on experimental designs, ethical considerations, and intriguing questions like predicting helping behavior in emergencies. Explore how studying helping behavior in experiments can shed light on complex human interactions and behaviors.

  • Experiments
  • Social Sciences
  • Helping Behavior
  • Emergencies
  • Laboratory

Uploaded on Sep 10, 2024 | 1 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. LABORATORY LABORATORY Stephen Benard Department of Sociology Indiana University EXPERIMENTS IN THE EXPERIMENTS IN THE SOCIAL SCIENCES: SOCIAL SCIENCES: AN INTRODUCTION AN INTRODUCTION

  2. OVERVIEW A sample question What is an experiment? Basics of experimental design What can we learn from experiments? Ethics of experiments

  3. AN (ENCOURAGING) DISCLAIMER Just a small sample of the: Questions Experimental designs Independent and dependent variables Many, many possibilities

  4. SAMPLE QUESTION: WHAT PREDICTS HELPING IN AN EMERGENCY? If we notice someone in need: Are we more likely to help when alone, or in the presence of others? Diffusion of responsibility

  5. ARE PEOPLE LESS LIKELY TO HELP OTHERS WHEN IN A GROUP? Challenging to study through observation Emergency events are rare and hard to predict May vary in countless ways Many alternative explanations People in groups less likely to notice More groups at busier times of the day less time Unhelpful people more likely to travel in groups

  6. STUDYING HELPING IN AN EXPERIMENT Would be useful to repeatedly observe responses to the same emergency under different condition E.g., when many or few people observe the emergency Could be staged in a laboratory (Darley and Latane 1968 in JPSP) Laboratory discussion group One person appears to have a seizure Manipulate number of people present Measure proportion who helped, speed

  7. A FEW MORE EXAMPLES Does violent media make people aggressive, or do aggressive people prefer violent media (Bandura, Ross, and Ross 1961)? Does intergroup contact reduce or exacerbate intergroup conflict (Sherif 1958)? Does positive mood make people more altruistic, or are more altruistic people happier (Isen et al 1978)? Do our attitudes determine our behavior, or does our behavior determine our attitudes (Festinger and Carlsmith 1959)? Does the gender/race/age/criminal record/other characteristic of a job applicant affect the likelihood of being hired (e.g., Pager 2003)? Is support for a policy determined by the content of the policy, or the identity of the party supporting it (Cohen 2003)? Does lack of control over our environment turn us into conspiracy theorists (Whitson and Galinsky 2008)? Does the status of an author s institution affect their chances of having an article accepted (Peters and Ceci 1982)?

  8. WHY CONDUCT AN EXPERIMENT? Identifying causes Addressing alternative explanations Identifying moderators and mediators Examining hard-to-observe or rare events

  9. WHAT IS AN EXPERIMENT? THREE PRINCIPLES Manipulation of the independent variable Random assignment to condition Controlled measurement

  10. MANIPULATION In experiments you must manipulate an independent variable (IV) This creates 2 (or more) levels of the IV The levels of the IV are called conditions Conditions identical except for the manipulated IV E.g., number of people present when an emergency occurs

  11. RANDOM ASSIGNMENT How do we distinguish the effects of our IV from extraneous variables? Perhaps personal interest in helping others confounded with group size Experimenter places people into experimental conditions by chance Equal likelihood of being in each condition Individual differences cancel out

  12. RANDOM ASSIGNMENT Colors symbolize any differentiating attribute among the individuals (e.g., personal interest in helping others) Before Random Assignment R After Random Assignment Small crowd Large crowd Experimental Groups

  13. WHAT IF PEOPLE CHOSE THEIR CONDITION? Colors symbolize any differentiating attribute among the individuals (e.g., personal interest in helping others) Before choosing C Systematic error Small crowd Large crowd Self-selected Groups

  14. CONTROLLED MEASUREMENT Systematically observe changes in the dependent variable as a function of changes in the independent variable Important to avoid bias in recording the DM Participant blind to hypotheses Experimenter blind to hypotheses Experimenter blind to condition

  15. A SIMPLIFIED HELPING STUDY (BASED ON DARLEY AND LATANE 1968) Experimental setting: a laboratory discussion group Simulate an emergency (seizure) Manipulate number of other people present in group E.g., zero vs. three Randomly assign participants to the alone condition or the group condition Measure proportion helping, time to help

  16. EXPERIMENTAL DESIGN Two condition, treatment-control design Similar to medical study with placebo Simplest possible design Often very effective, but also limited Additional treatment conditions Factorial designs

  17. ADDITIONAL TREATMENT CONDITIONS Perhaps group size has a non-linear effect Add additional condition with 6 total group members Sometimes it is useful to have a baseline condition E.g., a study of whether a text is evaluated more positively when the author is a man vs. a woman May wish to compare to a condition with no author information Is it that men receive a boost relative to the baseline, or women receive a penalty?

  18. FACTORIAL DESIGNS Multiple IVs Every combination of every level of IV Interaction effects Predict an interaction Or evaluate generality Similarity Similarity to to Similar No help Victim Victim Not No help No help Group Large Size Small Help A 2 x 2 Factorial Design

  19. BETWEEN VS. WITHIN-SUBJECTS DESIGNS Between-subjects design: Each participant is exposed to one level of the independent variable E.g., study of helping

  20. BETWEEN VS. WITHIN-SUBJECTS DESIGNS Within-subjects design: Each participant exposed to multiple levels of the dependent variable E.g. Text evaluation study More efficient But possibly easier to guess hypotheses Requires counterbalancing Rarely possible in high-impact designs

  21. INDEPENDENT VARIABLES How do we know operational independent variable accurately measures theoretical dependent variable? E.g., positive mood How do we know the manipulation had the expected effect on participants? Manipulation Checks

  22. DEPENDENT VARIABLES (AND SOME BROAD GENERALIZATIONS) Examples Examples Difficulty/ Difficulty/ Cost Cost Sensitivity to Sensitivity to Social Social Desirability Desirability Participant Participant Engagement Engagement Useful Useful for/When for/When Reported attitudes & emotions, vignettes Low High Low Goal is measure internal state Verbal Help victim, donate to charity High Low High Goal is predict behavior Behavioral Choose partner, agree to something Moderate Moderate High (pre- decision) Commitment of more interest than behavior Behavioroid fMRI, cortisol, heart rate Very High Very Low High (but possible discomfort) Biological mediators/ moderators Physiological

  23. WHAT CAN WE LEARN FROM EXPERIMENTS? High degree of control provides high internal validity Experiments provide the strongest possible evidence for causality But, external validity of laboratory experiments is often criticized Settings don t always resemble real world Participants don t resemble other populations Samples are generally non-random Small samples, at least by survey data standards Participants are often college undergraduates Participants are often WEIRD: Western, educated, industrialized, rich, democratic

  24. MUNDANE VS EXPERIMENTAL REALISM Mundane Realism: the extent to which an experiment looks like the real world Experimental realism: the extent to which experience is psychologically real and important to participants Rarely come to a lab for a group discussion

  25. GENERALIZING FROM? Should not generalize directly from an experiment to a real world situation Experiments test theories Theory bridges empirical studies and the real world See Zelditch, 1969, Can you really study an army in the laboratory Theory Complex phenomena to be explained Provide evidence for or against a theory Provide explanation for real world phenomena Real World Experiment

  26. SCOPE CONDITIONS Criticism that findings won t generalize Often explicitly or implicitly signal possible scope conditions These can be tested to further refine the theory Example: College students discriminate against women in hiring simulation Maybe more than real managers: less experience Maybe less than real managers: more egalitarian

  27. CONVERGENT VALIDITY Useful to think of different methods as complementary, not competing Survey data May have high external validity, but limited ability to show causality Experiments High internal validity, but limited generality

  28. EXPERIMENTAL ETHICS: Three core principles for all research Respect Beneficience Justice Deception Necessary to test some hypotheses But should be used only as a last resort And fully explained to participants Debriefing

  29. SUMMARY Experiments are excellent for answering questions about causality, exploring alternative explanations, and examining rare or hard to observe events Many different types and approaches to experiments, can (must) be tailored to the research question Facilitate systematic replication and theory development Strengths/weaknesses complement other methods

  30. Stephen Benard Department of Sociology Indiana University THANK YOU!

More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#