Web Archiving Many Voices: Documenting COVID-19 and Marginalized Communities at Arizona State University

This week’s post was written by Shannon Walker, University Archivist, Arizona State University.

The COVID-19 pandemic gives us, as memory workers, the unusual opportunity to document a crisis as it is occurring. As with many institutions, University Archives staff at Arizona State University quickly recognized the need to document a rapidly unfolding history and its impact on our institution and community through the use of our web archiving tools.

Several factors highlighted the urgency of the task. Firstly, this was likely a once in a lifetime occurrence. We did not have time to develop a new process or procedure for “how to document a pandemic.” Secondly the crisis itself has a temporal nature, it was important to capture sites that were continuously updating, knowing that in six months or a year they might no longer refer to COVID-19 (we hope!). 

Deciding What to Capture

While there is no official mission statement for the web archiving program @ ASU, we have developed a draft of general guidelines for prioritizing the websites we capture, preserve and make available for research. Essentially:

The purpose of the Web Archiving Program at the ASU Library is to develop best practices for capturing born-digital, web-based information produced by the University. The first priority is identifying sites that complement the University Archives’ Collection Development Policy. The Web Archiving Program will also attempt to capture web-based publications for the broader ASU umbrella, especially as mandated by the University Archives Records Management Program, as well as current events and specialized collections.

In addition to being guided by our internal guidelines, we also had in mind ASU’s Charter

ASU is a comprehensive public research university, measured not by whom it excludes, but by whom it includes and how they succeed; advancing research and discovery of public value; and assuming fundamental responsibility for the economic, social, cultural and overall health of the communities it serves.

So, with both principles guiding our web archiving selection criteria we sought to address some of the following questions:

  • As we document COVID-19 at ASU, how can we be inclusive?
  • As we select sites to capture, do they document the more prominent voices on campus or marginalized communities or both?
  • Who might we be excluding? Can excluded groups be included through web archiving?
  • When a researcher is using these materials 10 years down the road what will they see? What will they not see?

Creating the COVID-19 @ ASU Response collection

The initial phase of this effort included identifying, reviewing and capturing our own institution’s websites. We wanted to document the School’s official response as well as the experiences of employees and students. We sought out and captured seeds from the Office of the President, the Office of the Provost and ASU Now, one of our campus news resources. We also captured a site from the University’s governing body, the Arizona Board of Regents

Additionally, we sought to document the impact of the pandemic on the student experience. We were able to identify seeds from the Admissions Office, Financial Aid, University Housing, and Greek Life (fraternities/sororities). In addition, a few individual schools created lists of resources for their students. Many of these lists focused on the mental, emotional, and financial consequences of the pandemic, recognizing the impact on the whole student, far beyond the logistics of classes and technology.

As we were identifying seeds to capture we were fortunate to locate a few that provided the opportunity to document the voice and experiences of underrepresented groups on campus. We say fortunate because we know that marginalized groups do not always have a prominent presence on the website, but they are an important part of what we hoped to document. Some of the sites included in this group were the Alliance of Indigenous Peoples, ASU’s American Indian Policy Institute, the International Student and Scholars Center, and the American Indian Student Support Services. 

Importantly, many of these sites addressed issues of equity and the digital divide, which was a significant issue for students here at Arizona State University as we went to online classes. 

Challenges and Unexpected Discoveries

As with any project, there were challenges that made us profoundly aware of the limits of web archiving. For one, not all campus experiences are captured on published campus websites. We know that many of the raw experiences of students, staff and faculty were being documented on social media sites. However, we chose not to capture these sites because of privacy concerns.  In addition, some website captures did not go well. After reviewing them, we could see that elements of the original site were not captured. We continue to work on improving those crawls. 

As an added bonus to our efforts, we were able to identify and collect materials in other digital formats (namely PDF or JPG) that further document COVID-19 at ASU. An additional unexpected discovery was that a few of the sites we targeted for crawls were already being well captured by the Internet Archive. In that case, we needed to figure out how to point to them as part of our curated collection.


Our efforts to identify, capture and collect websites related to COVID-19 at Arizona State University are only a part of the pandemic story for future researchers. We hope that the sites we chose, and were successfully able to capture, will present a varied and diverse perspective considering the limits of web archiving technology. It is not a perfect tool but it is timely and nimble, and becoming an increasingly important part of our toolkit as we seek to expand the narrative of our school’s history.