“The barriers to website and social media archiving solutions in the higher education market is limited budget, time constraints and lack of resource. And, during this period, these challenges are only exacerbated” David Clee, MirrorWeb.
In this post, MirrorWeb and Arkivum sit down to discuss the barriers to website and social media archiving solutions for universities and higher education markets.
Joining us in the post is David Clee (CEO, MirrorWeb), Sinead McKeown (VP of Product Management, Arkivum) and Sean Rippington (Digital Archives Officer at the University of St Andrews), thank you to all for their support and contributions to this article.
Read on to learn more about the 7 barriers to website archiving and social media archiving and how you can overcome them.
Barriers behind archiving and how to overcome them
1. Lack of Budget and Resources
If it wasn't hard enough already to obtain budget and resource, the Covid-19 pandemic has turned the dial up, freezing business activity and procurement. Whilst this is true, there's been an uplift in purchases made in digital technology, helping to shield businesses through this difficult period and protect their future.
MirrorWeb's David Clee says that, amongst organisations in the heritage, museums and higher education markets, “The initial reaction is that digital platforms are expensive and that web and social archiving is also expensive. There's a view that an organisation may need external funding and an expert to manage the process.”
It is an assumption held by many organisations, but the cost will largely depend on the size of the dataset (e.g. single or multiple websites) and the frequency of archiving.
As an example, for financial clients, the cost is likely to be more expensive as they have to archive every day. If archiving takes place once a year or once a month, dependent on your requirements, the price would be cheaper for customers.
The availability of affordable yet scalable, web and social media archiving solutions mean organisations are now able to decrease internal resource costs and overheads needed to maintain infrastructure. For example, outgoings for local servers, hardware upgrade cycles, and growing storage requirements on much more manageable with an archiving solution.
A lack of budget can also mean a lack of resources to have the capacity to engage with digital archiving services - which is why MirrorWeb created the heritage and compliance service to help businesses during this period, making access to digital archiving solutions easier and ensuring long-term preservation is guaranteed.
2. Lack of Awareness
A lack of awareness of new solutions has made website and social media archiving difficult for the higher education markets, as highlighted by Arkivum’s Sinead McKeown:
“I’ve heard stories of librarians having to archive their websites and tweets by copying and pasting them into their library management system, and then reference them back via their online catalogue. What else could you do when Twitter first emerged, when there wasn’t an available solution to implement?”
The lack of available digital archiving solutions has been influenced by a community in which there have been many discussions about problems but a lack of tangible solutions.
“The danger, and I’ve heard this from archivists, IT managers and record managers, is that the conversation only gets as far as talking. It’s easy, especially in this market, to talk about problems and to identify problems, even to analyse some of the problems, without knowing quite how to solve them,” says Arkivum’s Sinead McKeown.
It's still only a recent development that cloud-native web and social media archiving services have become available in the market. Therefore, it's essential that businesses reevaluate these archaic and time-consuming processes and become aware of more accurate and efficient solutions, ensuring the long-term preservation of their digital data.
3. Limited Viable Archiving Solutions
Thanks to emerging partnerships such as MirrorWeb and Arkivum's, there are now vendors thar answer a full spectrum of digital archiving needs. However, partnerships in this space are sparse which ultimately impacts customers as they are unable to deploy an end-to-end solution. As MirrorWeb's David Clee states, “Many customers are struggling twofold, they're not only unaware that website and social media archiving platforms exist but they're also unaware of the benefits when archiving vendors come together to join their core offerings.”
“From the latest ARA Conference, something apparent was that archivists didn’t know it was possible to archive Instagram, or Facebook, or Twitter, and to capture the metadata around each of those platforms and each of those interactions.”
He adds that MirrorWeb and Arkivum’s hybrid, end-to-end digital archiving solution makes this easier than ever: “Whether it’s websites, blogs, social media channels, Twitter, or even YouTube, we can capture it, and this will make up the rich tapestry of history.”
To help universities during these uncertain times MirrorWeb has created a Heritage and Compliance web archiving service. Simply click here to find out more and learn how University of Westminster and Cardiff University are using the MirrorWeb Platform to preserve their digital identity and web records.
4. Meeting Big Data Requirements
These are complex information-driven organisations and house a lot of research data, etc., and so a concern amongst heritage, museums and higher education markets is whether a digital archiving solution can meet their big data requirements.
Sean Rippington from the University of St Andrews raises common questions within such organisations. These include whether available solutions can cope with large websites, the regularity of crawls and the cost of storage, and if this data can then be available for practical use.
This is demonstrated in the museum sector. “The thing everybody’s talking about in the museum sector is huge image libraries that are massively difficult to manage,” says MirrorWeb’s Sam Roberts, as he goes on to outline the solution: “To me, that needs a cloud solution, and that’s basically where you need to be, to make them indexed, to make them searchable, to make them useful.”
We have found cloud-native archiving to be a scalable, cost-effective solution for our customers, and no matter the size of your website or social media accounts - whether you are The National Archives with over 5,000 website and an archive of 120 TB or have a small website or social account - we can meet your data requirements.
5. Convincing Senior Management
“The people with the budget and the problem include the IT Director, the Information Governance Officer, the Data Protection Officer, and the board of trustees. It’s those people with the pain problem,” says Arkivum’s Sinead McKeown.
But, as highlighted by Sean Rippington, Digital Archives Officer at the University of St Andrews, it can be difficult raising awareness amongst people in such senior management positions about the need for web and social archiving - but as digital becomes increasingly critical to organisations, awareness surrounding the fragility of their digital assets has been established, engaging senior management to take digital preservation action.
There are also a number of resources being produced to drive the message home, for example, MirrorWeb contributed to the DPC Executive Guide in 2019, which acted as a helpful resource for practitioners to persuade senior executives to take digital preservation action.
6. Sensitive Data
Sean Rippington, Digital Archives Officer at the University of St Andrews, raises questions about the legal restrictions of social media, including copyright, GDPR and the ethical concerns of capturing so much web data which may include student data.
This is in most part due to complications of personal data regulations and platform restrictions, which can make it difficult for institutions to produce a useful policy for their researchers and research support staff.
These form the basis of ongoing discussions for many markets, not only higher education. Collectively we seem to be calling for more open data at one end of the spectrum whilst calling to protect personal data at the other.
Although there is no clear answer due to the complexities of GDPR and individual business circumstances. It's clear that when organisations capture data to meet compliance requirements and have a legitimate interest to do so, they're operating completely in the spirit of the regulation. MirrorWeb’s solution was built with this in mind, ensuring every archive is legally admissible and signed with a digital signature that's GDPR compliant, CMA compliant, along with many other regulatory requirements.
7. Support Moving to a New Provider
Whether organisations are managing digital archiving in-house or using an external service provider, they want to know there is a clear strategy, the associated costs and how best to proceed should they need to move to a different digital archiving provider.
This is something MirrorWeb has enjoyed great success with for The National Archives when we helped them move to a new web and social media archive provider in two weeks. That was over 5,000 websites from 1996 to the present, as well as tweets and videos from government social media accounts, with the data footprint of the archive being over 120 TB in 2018.
A robust archiving provider will also use the industry-standard WARC file format, ensuring archives are immutable and tamperproof. This also means they can be transferred over to an alternative supplier should this be a requirement in the future.
So, whether you need to move a lot of or a small amount of archive data, the principles of moving the data remain the same, and no matter your requirements, we will be able to support your move to a new provider.
{{cta('eee985bc-26e2-4740-9a03-c36696fe28b6')}}
An end-to-end digital archiving solution
Taking a lead from other successful technology sectors, MirrorWeb and Arkivum identified that their solutions were complimentary, bringing different archiving specialisms together to provide an end-to-end digital archiving solution.
Together, we're proud to say we're able to offer the most comprehensive data lifecycle management solution and service for website and social media archiving in the higher education markets.
The WARCs created are passed seamlessly and automatically via an API to Arkivum’s Perpetua system, preserving and future-proofing the web and social media content for all time.
The partnership now covers safeguarding, digital preservation, compliances, records management and integration, and data discovery, filling a major gap for institutions seeking digital preservation as a comprehensive service with open-standards and a no-vendor-lock-in philosophy.
To help universities during this difficult period, MirrorWeb are currently offering a Heritage and Compliance web archiving service helping organisations drive down costs whilst remaining compliant. Simply click below to find out more!
