The Computational Reproducibility of Psychology Learning and Teaching Research

Background:

Research is computationally reproducible when independent analysts can use the underlying data to reproduce the original results. Some research in psychology is computationally reproducible, although much is not.

Objective:

To assess the computational reproducibility of research published in Teaching of Psychology (ToP), Psychology Learning and Teaching (PLaT), and Scholarship of Teaching and Learning in Psychology (SoTL-P).

Method:

We identified key claims in 60 papers published with open data in ToP, PLaT, and SoTL-P between 2017 and 2025. We then sought to reproduce 101 results supporting these claims.

Results:

We exactly reproduced 73 of the 101 results. For a further 23 results, the substantive interpretations of our results matched those of the original researchers. The majority of the reproductions were performed relatively quickly by a psychology undergraduate with 2 years of relevant experience.

Conclusion:

Psychology learning and teaching research appears more computationally reproducible than research in other psychology sub-fields. However, reproducibility is undermined by several factors, including a lack of open data and code.

Teaching Implications:

Computational reproducibility activities can be incorporated into undergraduate research methods classes, effectively address many research methods learning outcomes, and help students develop both subject-specific and transferable employability skills.