Understanding Experimental Psychology: Designs and Control Issues in Research
This content delves into the basics of experimental research, focusing on different types of experimental designs and control issues. It covers between-group and within-group designs, discussing advantages and disadvantages, as well as methods for controlling for nonequivalence. The importance of random assignment and matching in research design is highlighted, along with the challenges and considerations involved in matching subjects effectively.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Psychology 3450W: Experimental Psychology Fall, 2019 Professor Delamater
Experimental Research: Basic Experimental Designs and Control Issues 1. Basic Types of Experimental Designs 2. Control Issues 3. Bias in Experimental Research Lets look at each of these in more detail
Between & Within Subject Designs Between Subject (Group) Design Different subjects participate in the different groups of the study. Within Subject (Group) Design The same subjects participate in each condition of the study. Between Subject Design Within Subject Design Experimental Group S1 S2 S3 S4 Control Group S5 S6 S7 S8 Experimental S1 S2 S3 S4 Control S1 S2 S3 S4
Between & Within Subject Designs Between Subject (Group) Design Different subjects participate in the different groups of the study. Between Subject Design Experimental Group S1 S2 S3 S4 Control Group S5 S6 S7 S8 2 Disadvantages: 1. Large N required (compared to within group designs) 2. Potential for nonequivalence
Between & Within Subject Designs Methods of controlling for nonequivalence 1. Random Assignment works for large numbers of subjects because subject variables will be equally distributed across the different groups of the study. - block randomization ensures = n/group 2. Matching (on particular subject variables) - used when sample size is low - matching variable is known about - matching variables can be measured easily Find Matched Pairs (trios, quartets, etc) and then randomly assign to the different groups. e.g., Effects of caffeine on color perception. Match gender, vision, art major or not, etc.
Between & Within Subject Designs Problems with matching 1. We may not have identified the most critical variable to match. variables of interest. In this case, we do the best we can and make sure that the different variables we ve identified as important ones are equally distributed across the groups. 2. It can be difficult to find enough subjects that are matched on all
Between & Within Subject Designs Within Subject (Group) Design The same subjects participate in each condition of the study. Within Subject Design Experimental Control S1 S2 S3 S4 S1 S2 S3 S4 2 Advantages: 1. Small N required (compared to between group designs) 2. Statistically more sensitive at detecting effects 3. No nonequivalence problem
Between & Within Subject Designs Within Subject (Group) Design The same subjects participate in each condition of the study. Within Subject Design Experimental Control S1 S2 S3 S4 S1 S2 S3 S4 Main Disadvantage: performance in other conditions. Types of Sequence Effects: a. Progressive Effect b. Carry-over Effect Sequence (or Order) Effects Performance in one condition may influence
Between & Within Subject Designs Progressive Effect This refers to a general effect of experiencing multiple treatment conditions (e.g., fatigue, practice). Carry-over Effect This refers to a specific effect of one treatment condition upon another. Different orders of exposure to the different treatment conditions could lead to different results.
Between & Within Subject Designs Progressive Effect This refers to a general effect of experiencing multiple treatment conditions (e.g., fatigue, practice). Carry-over Effect This refers to a specific effect of one treatment condition upon another. Different orders of exposure to the different treatment conditions could lead to different results. e.g., Expose subjects to an Easy and a Hard problem and measure the time it takes for them to complete each task. The order in which they could be exposed to these problems is either: 1. Easy Hard, or 2. Hard Easy Suppose it takes 6 s to solve the easy problem and 10 to solve the difficult, and that this does not depend upon order. No carry over effect.
Between & Within Subject Designs e.g., Expose subjects to an Easy and a Hard problem and measure the time it takes for them to complete each task. The order in which they could be exposed to these problems is either: 1. Easy Hard, or 2. Hard Easy Suppose it takes 6 s to solve the easy problem and 10 to solve the difficult, and that this does not depend upon order. No carry over effect. But, suppose solving the easy problem makes it easier to solve the difficult problem, whereas doing the difficult problem first makes it harder to solve the easy problem. Easy (6s) Hard (8s) Hard (10s) Easy (12s) In this situation the easy task is performed more rapidly than the hard task when they are the first task performed. But when they are the 2nd task performed, the hard problem is performed more rapidly than the easy one. This is an example of a Carry-Over Effect (specific orders matter!).
Between & Within Subject Designs Counterbalancing A method used to control for Sequence Effects Types of counterbalancing procedures: 1. Complete counterbalancing (N!) 2. Partial counterbalancing (Latin Square) 3. Reverse counterbalancing 4. Block randomization If subjects are exposed to each condition of the study only once and the number of conditions is not too large, then a complete counterbalancing procedure should be used. e.g., suppose we wish to have subjects rate 3 different types of wine (A, B, & C). We would need a total of 3 * 2 * 1 = 6 subjects, or multiples of 6.
Between & Within Subject Designs Complete Counterbalancing N! will determine the number of orders Order of exposure to the wines We could run 6 people, 12, 18, etc See how this controls for progressive effects To assess a difference between the wines we average over all the As and compare to Bs and also Cs. Each wine is tasted in each sequential position of the sequence an equal number of times. Therefore, if there was a general progressive effect, this would apply equally to each wine. Any difference in wine rating could not be explained in this way. 1st, 2nd, 3rd A B C A C B B A C B C A C A B C B A Subject 1: Subject 2: Subject 3: Subject 4: Subject 5: Subject 6:
Between & Within Subject Designs Partial Counterbalancing Latin Square Design Order of exposure to the wines Suppose there are 4 wines to rate. This requires a minimum of 4 * 3 * 2 * 1 = 24 subjects. That may be too many. So we can use a Latin Square partial counterbalancing procedure which requires a minimum of 4 subjects (or multiples of 4). Two Criteria for the Latin Square: (1) each condition occupies each position once, and (2) each condition is preceded and followed by every other condition. 1st, 2nd, 3rd, 4th A B D C B C A D C D B A D A C B Subject 1: Subject 2: Subject 3: Subject 4: The rule is: A, B, X, C, X-1, D, X-2, etc where X is the final condition
Between & Within Subject Designs Partial Counterbalancing Reverse Counterbalancing Suppose each condition is to be assessed twice. ABCDDCBA will be the sequence. See how this controls for progressive effects.
Between & Within Subject Designs Partial Counterbalancing Block Randomization Suppose each condition is to be assessed many times (not just once or twice). ABCD | CADB | ACBD | etc In each block of ratings each condition appears once, and the order in that block is randomized. See how this controls for progressive effects.
Between & Within Subject Designs The main problem with counterbalancing is that carry-over effects are not adequately dealt with. Recall the Easy vs Hard problem we considered earlier: Easy (6s) Hard (8s) Hard (10s) Easy (12s) In this case, the pattern of results differ depending upon the order in which the two conditions are experienced. Averaging over the two Easy conditions shows no difference compared to the two Hard conditions. Thus, we would mistakenly conclude that there is no difference in these two cases. The best thing to do here is to SWITCH to a between group design. This design avoids the problem entirely because each subject is only tested once and there is no carry-over problem. This particular example is sometimes referred to as asymmetric transfer.
Between & Within Subject Designs How do we distinguish between an overall effect, a progressive effect, and carry-over effects? Answer: Evaluate the data in different ways. Order of exposure to the wines To? Assess? an? Overall? Effect 1st, 2nd, 3rd 20 15 A B C A C B B A C B C A C A B C B A Subject 1: Subject 2: Subject 3: Subject 4: Subject 5: Subject 6: Mean?Rating XA = XB = XC = 10 5 0 A B C Type? of? Wine Looks like wine A is preferred to wine B which is preferred to wine C. Of course, statistics would need to be done to confirm this.
Between & Within Subject Designs How do we distinguish between an overall effect, a progressive effect, and carry-over effects? Answer: Evaluate the data in different ways. Order of exposure to the wines To? Assess? a? Progressive? Effect 1st, 2nd, 3rd 25 20 A B C A C B B A C B C A C A B C B A X1 X2 X3 Subject 1: Subject 2: Subject 3: Subject 4: Subject 5: Subject 6: Mean?Rating 15 10 5 0 Pos? 1 Pos? 2 Pos? 3 Position?in? the?Sequence Ratings are progressively getting smaller, independent of wine. This suggests there is a progressive effect at work. No differences across positions would indicate no progressive effect.
Between & Within Subject Designs How do we distinguish between an overall effect, a progressive effect, and carry-over effects? Answer: Evaluate the data in different ways. Order of exposure to the wines To? Assess? Carry-Over? Effects 1st, 2nd, 3rd 30 Wine? A Wine? B Wine? C 25 Mean?Rating A B C A C B B A C B C A C A B C B A Subject 1: Subject 2: Subject 3: Subject 4: Subject 5: Subject 6: 20 15 10 5 0 Position?1 Position?2 Position?3 Position?in? the?Sequence The same pattern of ratings appear in each position. Thus, there is no carry-over effect.
Between & Within Subject Designs How do we distinguish between an overall effect, a progressive effect, and carry-over effects? Answer: Evaluate the data in different ways. Order of exposure to the wines To? Assess? Carry-Over? Effects 1st, 2nd, 3rd 30 Wine? A Wine? B Wine? C 25 Mean?Rating A B C A C B B A C B C A C A B C B A Subject 1: Subject 2: Subject 3: Subject 4: Subject 5: Subject 6: 20 15 10 5 0 Position?1 Position?2 Position?3 Position?in? the?Sequence Different patterns of ratings appear in each position. Thus, there are carry-over effects. Time to switch to a between-subjects experimental design to get rid of this contamination.
Between & Within Subject Designs Experimental Designs and Control Issues in Developmental Research Cross-sectional design This is a between-group experimental design in which distinct groups of people of different ages are assessed at the same time on some independent variable or set of independent variables. at different ages in life. e.g., examine how IQ is a function of age by comparing people who are problem caused by the fact that people cannot possibly be randomly assigned to the different age groups of the study. So, for instance, does IQ really change with age or are there other differences experienced by the distinct age groups that may account for any apparent differences. People of different generations have different life experiences, for instance, and have different culturally accepted ways of living, different attitudes, etc. Control Issue: Cohort effects. There is a potential non-equivalence
Between & Within Subject Designs Experimental Designs and Control Issues in Developmental Research Longitudinal design This is a within-subject experimental design in which the same set of people are assessed at different ages. and administer the test repeatedly over many years. e.g., examine how IQ is a function of age, but follow the same people for different reasons, so it would be difficult to have a uniform population of people at all the desired time points. Control Issue: Sequence Effects. With multiple measures taken on the same individual over many time points one does not know if repeated testing itself may affect the results. Counterbalancing is not possible in this procedure. Control Issue: Attrition. Many people may not persist in the experiment Note: These potential problems don t invalidate the research, but, at the same time, any potential limitations in the research procedures should be understood by the investigator.
Between & Within Subject Designs Bias in Experimental Research Experimenter Bias This is where experimenter expectations may influence the way in which the experiment is conducted or analyzed and this could affect the results either in an unconscious or conscious way. e.g., Clever Hans The famous horse who allegedly could count Can the horse count and perform simple math operations? Or, unconscious bias?
Between & Within Subject Designs Bias in Experimental Research Experimenter Bias This is where experimenter expectations may influence the way in which the experiment is conducted or analyzed and this could affect the results either in an unconscious or conscious way. e.g., Clever Hans The famous horse who allegedly could count task and collecting the data. 2. Double blind technique: Neither the experimenter nor the subject knows which condition of the experiment they are in. Thus, any conscious or unconscious effects the experimenter may exert on the study cannot be applied differentially to the different groups of the study. Controls: 1. Use computerized, automated, methods of administering the
Between & Within Subject Designs Bias in Experimental Research Subject Bias This is where the subject s expectations may influence the manner in which they behave that can confound the experiment. e.g., If the subject is sensitive to the demand characteristics of the experiment, then they might infer the true intent of the study. If they understand the true intent of the study, then they may behave in a manner in which they perceive to be acceptable to the experimenter, or they may defy the experimenter and purposely behave in a way to contradict the way they think the experimenter wishes them to behave. In short, if the subject knows what the experiment is about, this can cause the subject to behave unnaturally, and this could adversely affect the results. conceal the demand characteristics of the study. 2. Deception: If the true intent of the study is disguised sufficiently, then subjects will not be able to infer the true intent of the study. For ethical reasons, subjects would have to be informed about the study s true intent at the end of the study. More later. 3. Field Research: If experimenter is collecting data by observing people in a public situation, people do not realize they are part of a study and will behave naturally. Controls: 1. Double blind technique: This works here as well, providing that they effectively
