PREreview of "Controlled experiment finds no detectable citation bump from Twitter promotion"

This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/10044712. This review reflects comments and contributions from Melissa Chim, Allie Tatarian, Martyn Rittman, Pen-Yuan Hsing. Review synthesized by Stephen Gabrielson. Selected journal articles were tweeted from one of several Twitter accounts with a large number of followers. The altmetrics and citations of these papers were compared with a set of control papers for a study period of three years. While altmetrics saw an increase immediately after tweeting, there was no statistically significant increase in citations for the study papers versus the controls by the end of the study period. Major comments: I would like to see a more explicit acknowledgement that this experiment was conducted with only ecological papers - the results are written as if the conclusions apply to scientific research broadly and not one specific discipline. For example, disciplinary differences in citation politics and mechanisms may have a big impact on the effects of social media dissemination. Would social media affect citations of monographs in the humanities the same as ecology papers? If the authors believe their findings can be generalised to other domains, I'm happy for them to make that argument, too. I like the authors' methodical approach to the study. It's well-designed and takes into account weaknesses of previous similar studies. I appreciate how thoroughly the authors explained their criteria for choosing articles. It's a shame that it is statistically under-powered to detect citation changes in WoS/Scopus, but that is an interesting result in itself and sets parameters for future studies. From my perspective, the current discussion section of this paper (1) summarises the key learnings from the experiment, (2) acknowledges that social media engagement is useful beyond paper citation counts, and (3) a "wistful" commentary on the value of social media dissemination of research. These points are worthwhile. However, I'd like to see a deeper, constructive dissection into the limitations of this experiment in the discussion section. In addition to mostly ecologist accounts tweeting ecology papers, there are various potential minor issues which could be tackled in future studies. I'd love to see a discussion of them. While the authors state that the dataset collected from this experiment is shared in the Supplementary Materials, I...