By Susan Paterson, SAA Web Archiving Section Vice-Chair and liaison librarian at Koerner Library at the University of British Columbia Vancouver.
The Web Archiving Section hosted another successful Coffee Chat which featured Tom Smyth, Program Manager of the Web and Social Media Preservation Program within the Digital Preservation Division at Library and Archives Canada (LAC). Tom discussed the evolution of the program which began in December 2005. The collection exceeds 100 TB of content and includes over 2.64 billion assets from wide ranging collections such as Government of Canada websites, thematic and rapid response collections.
At the start of the LAC’s National Web Archive Program, collection activities focused on the Government of Canada domain, provincial and federal elections, Canada’s experience in the Summer and Winter Olympics and Paralympics and commemorative events such as state funerals and the War of 1812. In 2013, the program expanded to include thematic collections and events-based collections such as COVID-19, the 2022 trucking convoy protest in Ottawa, and current Canadian perspectives from the war in Ukraine. The program conducts rescue and preservation harvesting and ensures that federal government websites are preserved, pending decommission or content removal to another website.
So who does all this work at LAC? The web archiving team consists of digital librarians and archivists that bring their own unique perspectives to web archival curation. On the technical side, the program’s senior crawl engineers are a critical component in tackling complex quality control issues. Tom and his team ensure that data curation aligns with user requirements and includes a digital humanities perspective with the purpose of building datasets for data historians, researchers and scholars. Tom used the following example to explain how the the digital librarians and digital archivists approach collection curation: “We ask the question, ‘Twenty years from now, when a digital historian sits down to write about the history of COVID-19 and its impact on Canada, what kind of data and sources do they wish they would have?’ This influences how we select for curation.” COVID-19 has demonstrated that web archiving is a key resource for documenting history and the pandemic has influenced how they collect materials.
LAC’s COVID-19 web archive consists of over 2000 resources, 16 TB of data and over 478 million objects, including 34 newspapers in both official languages representing various political and regional perspectives from coast to coast – and it continues to grow. The collection includes social media and over 4 million tweets concentrating on COVID-19 dialogue and its impact on Canadians.
Tom discussed the “black hole of quality control” (QC) and described the importance of using a methodological approach when conducting QC. He explained the importance of both a framework and a balance as QC can be a never ending project. For more on LAC’s approach to quality control, you can view one of the 2021 IIPC presentations from Tom Smyth and Patricia Klambauer The Black Hole of Quality Control: Toward a Framework for Managing QC Effort to Ensure Value
The importance of web archive finding aids was a key thread throughout the talk and later in the discussion. As Tom explained, finding aids enable researchers to see at a glance whether the collection or datasets are helpful in addressing their research questions. Ideal finding aids would include practical information such as a master seedlist, how many seeds for each theme, resource distribution, type of resource, language, QC status, and metadata specifications.
The Prime Minister of Canada’s website (www.pm.gc.ca) was used as an example to describe why web archiving is so important. Back in 2006, LAC had only five days to ensure that all of outgoing Prime Minister Paul Martin’s website was captured before the site was replaced with the website of newly elected Prime Minister Stephen Harper. Tom underlined the importance of being proactive and the need to have good working relationships with partners and government departments such as the Privy Council Office, which supports the Office of the Prime Minister.
The Truth and Reconciliation Commission Web Archive (TRC) is another example of collaboration between LAC and other institutions. LAC worked jointly with the University of Winnipeg Library, the University of Manitoba Libraries and the National Centre for Truth and Reconciliation (NCTR) to develop this collection. The relationship with Indigenous Peoples is a concentration for LAC and a constant subject of collection. All of these collections will be publicly findable in the Government of Canada Web Archive, which is planned to be released in the fall of 2022.
Tom discussed the ethical and legal considerations of web archiving. The Library and Archives of Canada Act (S.C. 2004, c. 11, s.8(2)) empowers LAC to collect pertinent web content. Steps are taken to ensure the copyright owner is informed that their site will be archived and takedown requests are respected. Even with LAC’s extensive web archiving experience and resources, some websites just can’t be crawled due to their structure and the need for human input.
LAC is a client of the Internet Archive and uses the Archive-IT software. All of the WARCS are transferred back to LAC for local digital preservation via LTO tape.
Tom closed his talk on a hopeful note. Our efforts today will help us to reduce the likelihood of what Vint Cerf has termed a “digital black hole” and “the forgotten century.” The global community of web archivists, through their concerted efforts, are making every effort to ensure that nationally relevant materials are captured, preserved and made available for future generations.