Meta-Analyzing Individual Participant Data from Studies with Complex Survey Designs: A Tutorial on Using the Two-Stage Approach for Data from Educational Large-Scale Assessments

Autor/inn/en	Brunner, Martin; Keller, Lena; Stallasch, Sophie E.; Kretschmann, Julia; Hasl, Andrea; Preckel, Franzis; Lüdtke, Oliver; Hedges, Larry V.
Titel	Meta-Analyzing Individual Participant Data from Studies with Complex Survey Designs: A Tutorial on Using the Two-Stage Approach for Data from Educational Large-Scale Assessments
Quelle	In: Research Synthesis Methods, 14 (2023) 1, S.5-35 (31 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Brunner, Martin) ORCID (Keller, Lena) ORCID (Stallasch, Sophie E.) ORCID (Kretschmann, Julia) ORCID (Hasl, Andrea) ORCID (Hedges, Larry V.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1759-2879
DOI	10.1002/jrsm.1584
Schlagwörter	Leitfaden; Meta Analysis; Surveys; Research Design; Educational Research; Achievement Tests; Foreign Countries; International Assessment; Secondary School Students; Data Analysis; Sampling; Regression (Statistics); Effect Size; Gender Differences; Socioeconomic Status; Correlation; Individual Characteristics; Program for International Student Assessment + Suchen Sie Ihr Suchwort? Meta-analysis; Metaanalyse; Survey; Umfrage; Befragung; Forschungsdesign; Bildungsforschung; Pädagogische Forschung; Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Ausland; Sekundarschüler; Auswertung; Regression; Regressionsanalyse; Geschlechterkonflikt; Socio-economic status; Sozioökonomischer Status; Korrelation; Personality characteristic; Personality traits; Persönlichkeitsmerkmal
Abstract	Descriptive analyses of socially important or theoretically interesting phenomena and trends are a vital component of research in the behavioral, social, economic, and health sciences. Such analyses yield reliable results when using representative individual participant data (IPD) from studies with complex survey designs, including educational large-scale assessments (ELSAs) or social, health, and economic survey and panel studies. The meta-analytic integration of these results offers unique and novel research opportunities to provide strong empirical evidence of the consistency and generalizability of important phenomena and trends. Using ELSAs as an example, this tutorial offers methodological guidance on how to use the two-stage approach to IPD meta-analysis to account for the statistical challenges of complex survey designs (e.g., sampling weights, clustered and missing IPD), first, to conduct descriptive analyses (Stage 1), and second, to integrate results with three-level meta-analytic and meta-regression models to take into account dependencies among effect sizes (Stage 2). The two-stage approach is illustrated with IPD on reading achievement from the Programme for International Student Assessment (PISA). We demonstrate how to analyze and integrate standardized mean differences (e.g., gender differences), correlations (e.g., with students' socioeconomic status [SES]), and interactions between individual characteristics at the participant level (e.g., the interaction between gender and SES) across several PISA cycles. All the datafiles and R scripts we used are available online. Because complex social, health, or economic survey and panel studies share many methodological features with ELSAs, the guidance offered in this tutorial is also helpful for synthesizing research evidence from these studies. (As Provided).
Anmerkungen	Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01