Exploring Multilevel Theory and Analysis: Ongoing Debates and Future Directions
Delve into the world of multilevel modeling with insights on disciplines, key debates, and ongoing discussions. Discover the nuances between B and W variances, top-down effects, and more, offering a comprehensive view of this complex analytical approach. Unravel traditional and contemporary perspectives to pave the way for future advancements in multilevel theory and analysis.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Multilevel Theory and Analysis Ongoing Debates and Future Directions March 30, 2022 Zhen Zhang, PhD Department of Management and Organizations Southern Methodist University
Agenda A review of multilevel modeling traditions and key insights Two debates that have perplexed new learners An integrated example Future directions with more advanced methods Q & A (30 minutes) 2
Multilevel Modeling (MLM) Disciplines and traditions Educational, developmental, and applied psychology (Klein & Kozlowski, 2000; Snijders & Bosker, 1999; Raudenbush & Bryk, 2002) Districts, schools, classrooms, and students Companies, departments, groups, and employees Econometrics and strategic management Yearly observations nested under firms Fixed-effect (FE) modeling = within-firm effects (Certo, Withers, & Semadeni, 2017) 3
Key Insights of MLM Between (B) vs. Within (W) variances Level 1 has both B and W variances (Preacher et al., 2010) Same idea in the WABA technique (Dansereau et al., 1984) Potentially different B vs. W effects E.g., amount of exercise and risk of an heart attack Unit-mean centering is pivotal for partitioning these effects Cross-level interaction B (W W): the most interesting? The econometrics tradition does not conceptualize cross-level interaction in the same way as OB/HR does 4
Ongoing Debate 1 Is there a real top-down effect? The answer depends on how seriously you d theorize the B variance of the outcome Psych Safety Climate Collective Efficacy Team level Creative Self-efficacy Individual level Within-team level Creative Self-efficacy 5
Ongoing Debate 1 This is not an issue for educational research When predicting student achievement, there is not any construct for collective achievement (classroom average achievement) In strategic management, there is no construct called average firm performance across 10 years of panel data In OB/HR research, collective constructs have been introduced and remained prominent in the literature for the past 30 years (team personality, team turnover, etc.) Referent-shift measures were often not used 6
Ongoing Debate 1 The actual relationships Team Collective Efficacy Psych Safety Climate Creative Self-efficacy Relative Efficacy Within-Team Perceived Climate Should we theorize purely at B level and/or W level? 7
Ongoing Debate 2 Can we just combine (and not pay attention to) the B and W effects? For example, GEE purposefully estimates population average effects (McNeish & Kelley, 2019) Using Mplus type=complex; estimator=MLR, or STATA cluster- robust SEs, these single level analyses produce an estimate that is the weighted average of B- and W-level coefficients (the weight is ICC1; again, refer to WABA) My colleagues and I (Preacher et al., 2010; Zhang et al., 2009) propose the decomposition-first approach Understanding that B and W level relationships often vary in magnitude and even in signs, if you take the conflation-first approach (e.g., McNeish, Stapleton, & Silverman, 2017), you will lose the opportunity to identify differential relationships at W vs. B levels 8
When does the conflation-first approach make sense? ICC1 values of endogenous variables are close to zero When ICC1 > 0, but the B and W effects do not differ similar to the Hausman specification test for FE vs. RE models When the level-2 units/groups are arbitrary (experiments, gender comparisons, weekdays, etc.) 9
Experimental studies: Arbitrary level-2 units Randomized Controlled Trials (RCTs) Key differences in data structure and statistical inference Only treatment vs. control groups, and no other groups to generalize to Similarly, gender comparison (women vs. men) studies do not use MLM to generalize to another group In ESM analysis, Monday Friday effects do not need to generalize Echoing the MLM logic, RCTs only show the average treatment effect (Deaton & Cartwright, 2018) RCTs can not predict how an individual subject reacts to the intervention 10
An Integrated Example Conceptual Model Team Diversity Team level Individual level Creative Self-efficacy Creative Performance Extraversion 11
An Integrated Example How was it tested? Team Diversity Team level Individual level Creative Self-efficacy Creative Performance Extraversion 12
Why we cant switch predictor and moderator In single level analysis, switching does not hurt much But, the predictor determines the intended literature to contribute In MLM, switching is problematic The predictor s and moderator s main effects are at different levels Multi-stage sampling was used to collect data First, draw a random sample of level-2 units Within each level-2 unit, cover a high proportion of unit members Given a level-2 unit s attribute, we examine a within-level variable s effect; not vice versa Level-2 attributes dominate the within-unit relationships 13
The Model that Should Be Tested Average Creative Performance Team Collective Efficacy Team Diversity Team level Within-team level Creative Self-efficacy Creative Performance Perceived Team Diversity Extraversion 14
Future Directions Cross-lagged panel modeling (CLPM) Maxwell and Cole (2007, Psych Methods) 15
Future Directions Random-Intercept cross-lagged panel modeling (RI-CLPM; Hamaker, Kuiper & Grasman, 2015, Psych Methods) https://www.statmodel.com/download/RI-CLPM%20Hamaker%20input.pdf 16
Future Directions Dynamic SEM using intensive longitudinal data (time series analysis with N > 1) Explicitly partitioning W vs. B effects Both random intercepts and random slopes are of substantive interest Further sources to explore: 2019 workshop by Mo Wang and Zhen Zhang hosted by Sun Yat-sen Business School (https://www.sohu.com/a/326542186_694649) Zhen Zhang s talk in 2016 in the CARMA video library (https://carmattu.com) Zyphur et al. s 3-paper series on general panel data modeling 17
References Certo, S. T., Withers, M., & Semadeni, M. (2017). A tale of two effects: The theoretical and empirical implications of longitudinal relationships in strategy research. Strategic Management Journal, 38, 1536-1556. Dansereau, F., Alutto, J. A., & Yammarino, F. J. (1984). Theory testing in organizational behavior: The variant approach. Englewood Cliffs, NJ: Prentice Hall. Deaton, A., & Cartwright, N. (2018). Understanding and misunderstanding randomized controlled trials. Social Science and Medicine, 210, 2-21. Klein, K. J., & Kozlowski, S. W. J. (2000). From micro to meso: Critical steps in conceptualizing and conducting multilevel research. Organizational Research Methods, 3, 211-236. McNeish, D. & Kelley, K. (2019). Fixed effects models versus mixed effects models for clustered data: Reviewing the approaches, disentangling the differences, and making recommendations. Psychological Methods, 24, 20-35. McNeish, D., Stapleton, L. M., & Silverman, R.D. (2017). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22, 114-140. Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15, 209-233. Snijders, T. A., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. Thousand Oaks, CA: Sage. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage. Zhang, Z., Zyphur, M. J., & Preacher, K. J. (2009). Testing multilevel mediation using hierarchical linear models: Problems and solutions. Organizational Research Methods, 12, 695-719. 18
Thank you! Questions? zhenzhang@smu.edu 19
