An experimental evaluation of StatHand: A free application to guide students’ statistical decision making

Quantitative research methods underpin psychological literacy and evidence based practice in psychology. Despite this, many students struggle to identify appropriate statistics for different types of research questions and data types. StatHand (see https://stathand.net) is a free application that facilitates this statistical decision making process by prompting students to focus systematically on each structural characteristic of their research design. 217 undergraduate psychology students were randomised to use one of four decision making aids: StatHand on an iPad, a familiar textbook, a familiar paper decision tree, or the textbook and decision tree combined. Participants were then asked to identify suitable statistics for five research scenarios. Students assigned to use StatHand demonstrated higher decision making accuracy than users of the three alternative aids (δ = .50 to .64). StatHand users also experienced lower cognitive load, higher confidence in the accuracy of their decisions and greater satisfaction with their assigned aid than one or more of the other groups. However, it took the StatHand users longer to make their decisions. Finally, there was strong evidence to support the hypothesis that StatHand is instructionally efficient, and that its use requires less effort to promote higher performance relative to the other three aids (δ = .49 to .70). StatHand can be incorporated into a variety of classroom learning activities, and educators are encouraged to consider how they can use it most effectively.