Detecting and Predicting Differential Item Functioning Using the MIMIC Model

Explore how the Multiple Indicators Multiple Causes (MIMIC) model can be applied to detect and predict potential biases in assessments, particularly between genders. Research questions include investigating attitudinal factors that may influence differential item functioning. The model incorporates covariates on items and latent traits, helping to reveal group differences and predict item biases. Practical significance is assessed based on the signed area under the Item Response Theory (IRT) model.

• MIMIC model
• Item bias
• Differential item functioning
• Gender differences
• Attitudinal factors

lei_mo Follow

Uploaded on Oct 05, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Application of the MIMIC model to detect and predict differential item functioning Kevin Krost, M.A. Josh Cohen, Ph.D. Virginia Tech, Educational Research and Evaluation

2. Research Questions RQ1 Is there differential item functioning between gender on this assessment? RQ2 What attitudinal factors predict or mediate differential item functioning and/or gender differences among mathematics?

3. Differential Item Functioning Differences in probability of endorsing an item between two groups after controlling for ability Can indicate item bias Difficult to explain and determine cause

4. MIMIC Model Generalized SEM which incorporates covariates on items and latent traits Family and link function for items Effect of grouping variable on latent trait indicates group differences Items indicate DIF Covariates can predict item and mediate DIF

5. MIMIC Model Proven to be related to the 2- parameter normal ogive IRT model (McDonald, 1967) Effect sizes based on signed area under IRT for practical significance (Raju, 1988) Signed Area = - i i

6. MIMIC Model Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Q27 Q28 Q29 Q30 Q31 Q32 Q33 Q34 Q35 Q36 Q37 Q38 Q39 Q40 Q41 Q42 Q43 Q44 Q45 Q46 Q47 Q48 Q49 Q50 Q51 Q52 logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit Ability 1 Gender Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 Q23 Q24 Q25 Q26 logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit

7. Scale Purification Items used to measure latent trait might exhibit DIF and should be removed from scale (Wang, Shih, & Yang, 2009) Iterative procedure Items detected as exhibiting DIF removed from scale and modeled again Time-consuming!

8. Purified Model Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Q27 Q28 Q29 Q30 Q31 Q32 Q33 Q34 Q35 Q36 Q37 Q38 Q39 Q40 Q41 Q42 Q43 Q44 Q45 Q46 Q47 Q48 Q49 Q50 Q51 Q52 logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit Ability 1 Gender Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Bernoulli Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 Q23 Q24 Q25 Q26 logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit logit

9. Predictive Model

10. Predictive Model Interested in items exhibiting largest DIF based on effect size Enter all covariates to predict item and mediate DIF Based on results, drop non- significant covariates to create more parsimonious model Drop all non-significant variables or one at a time?

11. Judgment Time Within items, interested in largest absolute drop in effect size from model to model Most interested when grouping variable becomes non-significant After most parsimonious models obtained, interested in covariates significantly predicting item Again, interested in full mediation

12. Judgment Time Depends on your interest as a researcher Substantive, focused on relationship of covariates and mediation of DIF Professional/test developer, interested in determining if DIF indicates item bias or simply spurious Both, depending on research interests

13. Any questions? THANK YOU!