Automating Archives: The challenges of investigating large-scale web archival infrastructures

In this paper we reflect on the motivations and methodological challenges of investigating the World’s largest web archive, the Internet Archive’s Wayback Machine (IAWM). Using a mixed methods approach, we report on a pilot project centred around documenting the inner-workings of ‘Save Page Now’ (SPN) - an Internet Archive tool that allows users to initiate the creation and storage of ‘snapshots’ of web resources in perpetuity by the Wayback Machine. By improving our understanding of SPN and its role in shaping the Wayback Machine, this work both reveals the myriad ways in which the tool is being used, but also highlights the challenges of designing and operationalising a study of the mostly obscured and dynamic processes that support this information infrastructure at scale. In doing so, the paper welcomes and invites further discussion surrounding investigations of hidden processes that enable large-scale and ‘semi-public’ digital infrastructures. The project is currently in the early data analysis and write-up phase.