Using StatHand to Train Structural Awareness and Promote the Development of Statistic Selection Skills

Psychology students struggle to recall, recognise or explain how they would select appropriate statistics for common research designs. These selection skills are underpinned by structural awareness, which is the ability to look past the surface (or topic) features of a research design and focus instead on its deep structural characteristics. Although most psychology undergraduates display limited structural awareness, it can be trained. In this pre-registered experiment, we designed and evaluated a novel method of training structural awareness. This training method made use of StatHand, a free iOS and web application, in scaffolded activities designed to highlight how the structural (but not surface) characteristics of a research design determine the selection of an appropriate statistical analysis. Bayesian analyses clearly indicated that this training was effective. Specifically, trained undergraduate psychology students (n = 50) out-performed an un-trained control group (n = 52) on five measures of structural awareness (performance on two sets of triad judgement tasks, two sets of explanation tasks and a scenario generation task) immediately following training, and again one week later (ð = 0.71 to 1.60). At both time points, the trained students also showed greater selection skills than the un-trained control students (ð = 0.52 and 0.57). Finally, on five of these six outcome measures, the trained students showed no decrease in performance between the two time points. Educators are encouraged to consider how they can adapt our methods and materials for deployment in a classroom context or online activities.