Digital Preservation

Digital Preservation Technology Watch Reports pass 171,000 Downloads

At the beginning of this month the new series of Digital Preservation Coalition Technology Watch Reports passed 171,000 downloads: substantially up from the 100,000 reported in May 2015: these are downloads by real users excluding robots etc.

The new series was launched publicly in February 2012 with Preserving Email by Chris Prom and there are now 12 titles published since that date. All have proved very popular: Digital Forensics and Preservation now heads the group with over 35,000 downloads, followed by Preserving Email with over 34,000, and Preserving Moving Picture and Sound with over 19,000.

The reports are published by the Digital Preservation Coalition (DPC) in association with Charles Beagrie Ltd as editors and Neil Beagrie as Principal Investigator and managing editor of the series. The series is intended as an advanced introduction to specific issues for those charged with establishing or running services for long term access.  They identify and track developments in IT, standards and tools which are critical to digital preservation activities. All are released as peer-reviewed open-access publications after a preview period of exclusive access to DPC members.

The DPC Technology Watch Report Series publications are freely available online from the DPC website at: http://www.dpconline.org/advice/technology-watch-reports

Digital Preservation Handbook Update February 2016

Originally published in 2001 as a paper edition, ‘Preservation and Management of Digital Materials: a Handbook’ was the first attempt in the UK to synthesise the diverse and burgeoning sources of advice on digital preservation.  Demand was so great that in 2002, a free online edition of the Handbook was published by the newly established Digital Preservation Coalition.

After more than a decade, in which digital preservation has been transformed, the Handbook remains among the most heavily used area of the DPC website.

Funders and organisations are collaborating on re-designing, expanding and updating the Handbook so it can continue to grow as a major open-access resource for digital preservation. The DPC and Charles Beagrie Ltd have been engaged on a major re-working of the Digital Preservation Handbook for release as a new edition over 2015/2016. The National Archives (our Gold Sponsor) working together with other stakeholders including Jisc, the British Library, and The Archives and Records Association (our Silver Sponsors), and the National Records of Scotland (our Bronze Sponsor) is supporting the Digital Preservation Coalition in updating and revamping the Handbook. Many individuals and organisations are also contributing to this work through book sprints, peer review, project and advisory boards.

The revision, guided by the user feedback and consultation (see Report on the Preparatory User Consultation on the 2nd Edition of the Digital Preservation Handbook), is modular and being undertaken over a two year period to March 2016.

We have provided updates at regular intervals to inform the community on progress with the project and with this final February update we are delighted to announce a number of key developments.

 

Publication Schedule

The 2nd edition of the Handbook had a partial “soft launch” in October 2015 and approximately 2/3rds is online and publicity accessible at http://www.dpconline.org/advice/preservationhandbook

This partial release will be further enhanced by additional functionality when a new platform for the website focused on ‘responsive design’ is brought on stream by the DPC in 2016. This will provide an updated design and improved user experience on mobile and tablet devices, compared to the current site templates that are optimised for viewing on a desktop screen. We will also add the facility to generate PDFs. In the interim some functionality and content will remain “works in progress” but the community have gained early access to a significant new resource.

The remaining 14 sections to complete the Handbook have now been written, edited and are in peer review (see Handbook contents page for coming soon sections). We are aiming to complete this work and revise content for publication by the end of March 2016. The Handbook is now live so we will need to close and update section by section for these 14 remaining updates, hopefully in the final week of March and/or early April 2016. Watch this space for future announcements!

NRS joins funding group

The Digital Preservation Coalition was delighted to announce this month that The National Records of Scotland (NRS) had come on board as a ‘Bronze Sponsor’ for the eagerly anticipated second edition of the ‘Digital Preservation Handbook’. As of February 2016, with the addition of the NRS we have raised 93% of estimated funding required for the Handbook revision. We have prioritised content creation, scaled back some events, and adjusted budgets to ensure completion within a very tight funding profile.

Slideshare from Handbook Workshop at DCDC15

A workshop on the Digital Preservation Handbook was run at the DCDC15 conference in early October. Powerpoint slides from the Handbook presentation are now available on Slideshare. They provide a detailed overview of the new edition Handbook and work in progress. To date, there have been over 2,000 views of the slides.

New Technology Watch report: Preserving Social Media

The Digital Preservation Coalition (DPC) and Charles Beagrie Ltd, with the UK Data Service, release the latest in their series of Technology Watch Reports today.

Preserving Social Media  by Sara Day Thomson of the DPC, provides guidance for researchers wanting to access social media for research purposes, the institutions who support them, and all organisations with a need to preserve social media data.

The report describes the landscape of archiving social media, identifying the challenges associated with this task, and just some of the strategies which might be adopted in attempting to provide long-term access to such voluminous and unwieldy content.

This newest addition to the Technology Watch Series was commissioned by the UK Data Service with sponsorship from the Economic and Social Research Council (ESRC) as part of their Big Data Network Support initiative.

‘The current ownership framework around social media data is very restrictive-mostly because of platform terms of service and developer agreements as well as the exclusive access of commercial data resellers,’ explains author Sara Day Thomson. ‘However, a number of strategies and case studies provide useful and legal avenues for ensuring long-term access to this valuable content.’

The report lays out a number of approaches to the preservation of social media data-a valuable resource currently at relatively high risk of disappearance if not actively addressed. For both small and large scale needs, this report applies methods to curate and archive user-generated content captured through platforms APIs. Many of these methods derive from the work of a handful of organisations at the forefront of this new field. Though the report addresses a number of significant challenges, it focuses on new developments and growing motivation across disciples to ensure that future generations have access to social media created today.

The preservation of social media has a wide appeal and this report is likely to be of interest not only to DPC members, but many organisations throughout the digital preservation community who face the challenge of keeping user generated content through social media accessible in the future.

‘Preserving Social Media’ is the latest in the series of popular DPC Technology Watch Reports which support the Digital Preservation Coalition’s objectives and provide advice on how to manage high-value and vulnerable digital resources beyond the limits of technological obsolescence.

Personal Digital Archiving DPC Technology Watch Report now publicly available

The Digital Preservation Coalition, Gabriela Redwine and Charles Beagrie Ltd are delighted to announce the public release of the latest DPC Technology Watch Report ‘Personal Digital Archiving’, written by Gabriela Redwine, Digital Archivist at the Beinecke Rare Book and Manuscript Library at Yale University.

This free peer-reviewed report is aimed at individuals who are concerned about how best to manage and preserve their own personal digital archives, as well as professionals who advise people on how to select and best preserve such digital content.

‘The term personal digital archiving refers to how individuals manage or keep track of their digital files, where they store them, and how these files are described and organised’, explained Gabriela Redwine. ‘People keep personal archives for many different reasons and the ubiquity of personal computing devices and the ease with which files can be duplicated often means that the same digital files can exist in multiple locations simultaneously.’

The report provides an overview of the key issues related to personal digital archiving, arguing for the importance and urgency of preserving personal files, while also acknowledging the difficulty of managing digital files that include a combination of digitised and born-digital materials. There is a short introduction to the role of cultural heritage organisations, in the history of personal digital archiving, as well as current initiatives, which sets the stage for resources and recommendations for individuals who want to be proactive about saving their own digital materials.

‘Our Technology Watch Reports are available open access as a distinctive contribution to the dissemination of good practice in digital preservation, but are framed directly around the needs of our members who help suggest topics and help us turn ideas into sustained and significant advice that addresses their requirements, explained William Kilbride of the DPC.

Neil Beagrie, managing editor of the Technology Watch Report series on behalf of the DPC added, “Personal archiving is where digital preservation most directly affects non-specialists as well as curators. This title in the Technology Watch Report series is likely to be of interest not only to DPC members and the digital preservation community but also to a wider public internationally.’

Personal Digital Archiving’ is published by the DPC in association with Charles Beagrie Ltd. Neil Beagrie, Director of Consultancy at Charles Beagrie Ltd, was commissioned to act as principal investigator for, and managing editor of, this Series in 2011. He has been further supported by an Editorial Board drawn from DPC members and peer reviewers who comment on text prior to release: William Kilbride (Chair), Janet Delve (University of Portsmouth), Marc Fresko (Inforesight), Sarah Higgins (University of Aberystwyth), Tim Keefe (Trinity College Dublin), and Dave Thompson (Wellcome Library).

12 slideshares for Xmas: 20 years in digital preservation

I have just posted the final instalment of a personal selection of 12 presentations drawn from events and topics over the last 20 years in digital preservation, which I hope will be of interest.

They are taken from events on four different continents including the first iPres conference and cover themes such as personal archiving, research data management, e-journals, the digital preservation lifecycle model, national and institutional strategies and collaboration, costs/benefit/economic impacts of digital preservation, the establishment of the Digital Preservation Coalition, and the development of the online Digital Preservation Handbook. I hope there will be something in there for everyone.

There are accompanying blog narratives which set the presentations into context and the powerpoint presentations themselves on Slideshare. Details and web links to them are as follows:

2014 – The Value and Impact of Research Data Infrastructure (economic impact), presentation to the Preservation and Archiving Special Interest Group (PASIG), Karlsruhe Germany    slides     narrative

2013 – Maintaining a Vision: how mandates and strategies are changing with digital content (changes and responses), keynote presentation to Screening the Future conference, London UK slides     narrative

2010 – Keeping Research Data Safe (digital preservation costs and benefits), presentation to KB Experts Workshop on Digital Preservation Costs, The Hague Netherlands          slides     narrative

2007 – Digital Preservation: Setting the Course for a Decade of Change (evolution or revolution?), keynote presentation to the Belgian Association for Documentation (ABD-BVD), Brussels Belgium              slides     narrative

2005 – Digital Preservation and Curation Summing up + Next Steps (setting curation and research agenda for2005-2015), conclusions to Warwick II Workshop, Warwick UK             slides     narrative

2005 – Plenty of Room at the Bottom? Personal Digital Libraries and Collections, keynote presentation to European Conference on Research and Advanced Technology for Digital Libraries (ECDL), Vienna Austria   slides     narrative

2004 – eScience and Digital Preservation, presentation to Association for Information Science and Technology (ASIST) conference, Rhode Island USA                  slides     narrative

2004 –  The JISC Continuing Access and Digital Preservation Strategy 2002-5(covering UK Higher Education sector and partners), presentation to the JISC-CNI conference, Brighton UK slides  narrative

2004 –Digital Preservation, e-journals and e-prints, presentation at private workshop 1st iPres conference, Beijing China                 slides     narrative

2004  –  The Digital Preservation Coalition (DPC), Its History, Programme, Rationale ,and Structure, set of 4 linked presentations to DPC Forum, London UK              slides     narrative

2001 – Preservation Management of Digital Materials (the Digital Preservation Handbook) presentation to Digital Preservation Workshop/State Library, Melbourne Australia         slides     narrative

1998 – Preserving Digital Collections: current methods and research (digital preservation lifecycle model), presentation to the Society of Archivists annual conference, Sheffield UK             slides     narrative

This is a baker’s dozen as there is a also bonus presentation from 2015 on slideshare covering the latest work on The Digital Preservation Handbook (new edition for full release in March 2016).

The background and narrative blog for this personal selection of presentations is also available.

SlideShare: The Value and Impact of Research Data Infrastructure

This slideshare, The Value and Impact of Research Data Infrastructure, was given at the Preservation and Archiving Special Interest Group (PASIG) meeting in September 2014 held at Karlsruhe, Germany. It is the final instalment of 12 presentations I have selected to mark 20 years in Digital Preservation. It demonstrates the value of preservation and re-use of research data.

Between 2011 and 2014, Charles Beagrie Ltd and John Houghton completed three major studies on the economic value and impact of the Archaeology Data Service, the British Atmospheric Data Centre, and the Economic and Social Research Data Service, and a synthesis of the three studies. In these studies, we developed and refined qualitative and quantitative methodologies to measure the value and impact of research data and associated services and tools.

This combination of methods has broken new ground in approaches to assessing the value and impact of major research data services and provided a strong evidence base and compelling outcomes.  In a recent review of the international state of the art as regards the relationships between large-scale science facilities and innovation performance, our work was one of 3 studies highlighted to UK Department of Business, Innovation and Skills as being particularly good examples of ‘good practice’ in the measurement of economic impacts.

The presentation focuses on these studies, with the study of the Archaeology Data Service given as a detailed example. It has a UK Focus but the research and lessons are international. These studies are also three of the few quantitative studies of the value and impact of digital preservation currently available.

A fourth study on the value and impact of the EMBL European Bioinformatics Institute has since been completed by Charles Beagrie Ltd and John Houghton and should be available in 2016.

New Resources page on Charles Beagrie Website

We have produced a new resources pages on our website describing all the outputs we have produced which are publicly available and accessible on open access to students and practitioners interested in our work. Areas described include Cost/Benefit, Impact, Technology Watch, Digital Preservation Policies and Strategies. Conference presentations, and other digital preservation resources. These are linked either to outputs on our website or on the websites of clients and partners. An extract of the page is shown below.

Breaking News: Digital Preservation Handbook Update October 2015

Originally published in 2001 as a paper edition, ‘Preservation and Management of Digital Materials: a Handbook’ was the first attempt in the UK to synthesise the diverse and burgeoning sources of advice on digital preservation. Demand was so great that in 2002, a free online edition of the Handbook was published by the newly established Digital Preservation Coalition.

After more than a decade, in which digital preservation has been transformed, the Handbook remains among the most heavily used area of the DPC website.

Funders and organisations are collaborating on re-designing, expanding and updating the Handbook so it can continue to grow as a major open-access resource for digital preservation. The DPC and Charles Beagrie Ltd have been engaged on a major re-working of the Digital Preservation Handbook for release as a new edition over 2015/2016. The National Archives (our Gold Sponsor) working together with other stakeholders including Jisc, the British Library, and The Archives and Records Association (our Bronze sponsors), is supporting the Digital Preservation Coalition in updating and revamping the Handbook. Many individuals and organisations are also contributing to this work through book sprints, peer review, project and advisory boards.

The revision, guided by the user feedback and consultation (see Report on the Preparatory User Consultation on the 2nd Edition of the Digital Preservation Handbook), is modular and being undertaken over a two year period to March 2016.

We have provided updates at regular intervals to inform the community on progress with the project and with this October update we are delighted to announce a number of key developments.

Publication Schedule

We are pleased to share the news that a critical mass of content has been prepared and peer reviewed and the project board has agreed we should release a majority of the Handbook.  DPC members have already seen the emerging revised 2nd Edition of the Handbook on the members’ private area and this has been switched to the public side of the DPC website. This partial release will be further enhanced by additional functionality when a new platform for the website focused on ‘responsive design’ is brought on stream by the DPC early in 2016. This will provide an updated design and improved user experience on mobile and tablet devices, compared to the current site templates that are optimised for viewing on a desktop screen. We will also add the facility to generate PDFs. We hope to complete remaining sections of the Handbook for a formal full publication release of the Handbook by March 2016. In the interim some functionality and content will remain “works in progress” but the community will gain early access to a significant new resource.

ARA joins funding group

The Digital Preservation Coalition was delighted to announce in September that The Archives and Records Association (ARA) had come on board as a ‘Bronze Sponsor’ for the eagerly anticipated second edition of the ‘Digital Preservation Handbook’. As of Oct 2015, with the addition of the ARA we have raised 87% of estimated funding required for the Handbook revision and continue working to complete it.

Section Illustrations and icons

We are using graphics available from digitalbevaring.dk (http://digitalbevaring.dk/about-us/) for main sections of the Handbook. They have kindly worked in collaboration with us to develop new illustrations when we have identified topics in the Handbook requiring new graphics for illustrations or icons.

New resources icon designs were received over the summer from digitalbevaring.dk  and the interim versions have been replaced in the Handbook. These are the new set:            

 

They are embedded now in all the Resources and Case Studies sections of the Handbook. It means there is now a consistent style to the Handbook with the icons and section heading illustrations sharing the same design, something we all felt was desirable. We are very pleased with the results and overall look that is now in place, and with the collaboration with digitalbevaring.dk that has added a lot to the visual appeal of the Handbook.

Multi-media

Multi-media resources where relevant have been selected and embedded in the Handbook. Selection has focussed on short, high-quality videos that can add significant value to experience and content.

Handbook Workshop at DCDC15

A workshop on the Digital Preservation Handbook was run at the DCDC15 conference in early October. Powerpoint slides from the Handbook presentation are now available on Slideshare. They provide a detailed overview of the new edition Handbook and work in progress.

 

SlideShare: How institutional mandates and preservation strategies are changing with digital content

This slideshare, Maintaining a Vision: how mandates and strategies are changing with digital content, is one I like and is a keynote given to the 2013 Screening the Future conference in London.

It is the penultimate of 12 presentations I have selected to mark 20 years in Digital Preservation. The final one to come will be published in December 2015.

My brief  for this conference keynote was to focus on how  institutional responses to collection and preservation mandates are realized and stretched by the digital…do existing institutions just ‘go digital’ but otherwise claim ‘business as usual’ [or not]?

The Talk had an AV focus given the nature of the conference but I think the messages will be of broad interest. It was in three parts:

The Changes: covering how digital content (including AV content) has changed the nature of typical collections across sectors; how it has shifted the scale of available content; and how content has fragmented and the number of content creators proliferated.

The Responses: covering how we have seen in response the growth of cross-sectoral preservation exchange (different sectoral membership of the DPC; Technology Watch Reports; the national coalitions worldwide such as nestor, NCDD, NDSA, etc); the development of shared services and outsourcing (e.g. digital preservation services in the cloud); and in some cases a range of cross-sector mergers (particularly of national archives and national libraries).

Conclusions:

What is changing? We are seeing multi-media permeating sectoral boundaries; greater shared interests and convergence of interests across different sectors; and a massive shift in the scale and management of digital media.

The responses?  We are seeing new alliances and partnerships; digital preservation exchange across sectors; some mergers and partnerships across established boundaries; and more shared services and outsourcing.

Finally, if you want to know the answer to the question “When was the beginning of the Digital Age” posed in previous posts, the answer is here in slide 8:

 

Keeping Research Data Safe (KRDS)

Keeping Research Data Safe (KRDS), a workshop presentation from 2010 available now on Slideshare, is the ninth of 12 presentations I have selected to mark 20 years in Digital Preservation. The remaining two to come will be published at monthly intervals over November and December 2015.

This presentation was given as part of the KB Experts Workshop on Digital Preservation Costs, held at The Hague in the Netherlands in 2010.

Although very small in terms of budget, the KRDS projects were terrific examples of collaboration to achieve influential results and the pleasure and value of working with colleagues from many disparate fields and organisations. I’ve selected it as an example of doing great things on small budgets if you have the right people, and for its influence on subsequent work both by me (e.g. impact studies) and on the field generally. For me, in terms of personal follow-up and later projects, the costs element of KRDS has been less important than the benefits side which has led to a series of project on impact with John Houghton (more on this in the final Slideshare in December).

The KB requested a briefing document on each cost model presented at the workshop in the form of responses to their set questions. I have reproduced mine for the KRDS presentation below – it captures lots of interesting context for the slides. I have added links to the KRDS Factsheet and KRDS costs data survey to it.

THE KEEPING RESEARCH DATA SAFE MODEL

Outline:

1. General presentation of the cost model

What is the purpose of the cost model?  The KRDS model aims to support the costing of digital preservation of research datasets and assessment of the benefits of preservation. A significant proportion of its work is also focussed on identification of preservation cost data sources and methods which could support any model. It is currently primarily a set of tools and methods to construct a localised model rather than a pre-developed generic costing tool. Further information on findings from the KRDS projects is available in the KRDS Factsheet.

Who are the users? – The primary audience is research organisations in the UK but organisations in other countries and sectors can adopt parts of the model and its methodologies.

What preservation strategies does it handle? – It can accommodate any preservation strategy or service strategy (e.g. outsourcing or shared services as well as preservation in-house).

What is the target data? – Research data from the sciences, social sciences, or arts and humanities.

What time perspective does it cover? – Any time period.

2. What method is the cost model based on?

What reference is the model based on?  – The model uses OAIS with extensions and adaptations by the project team.

What financial principles is it based on? – It is modelled to adopt the Transparent Approach to Costing (TRAC) a full economic costs (FEC) model approved by UK research funders and universities.

Which costing approach have you adopted?– We use an activity based costing approach supported by a Benefits Taxonomy for assessing benefits.

What implementation have you chosen? – N/A

3. Which challenges do you currently see in relation to cost modelling?

Special issues – General cost model challenges? –

Primarily a lack of good quality preservation cost data from a range of different types of archive and data types (see our KRDS costs data survey) which can be used to underpin and develop models.

Secondly an excessive focus on costs (rather than cost/benefits) and also sometimes a too limited focus on costs of preservation strategies rather than preservation service costs as a whole.

Occasional over-reliance on research project or start-up cost data which will not be representative of operational preservation costs.

The degree of confidence that can be placed in results from cost models. How reliable is any cost prediction for a model and how does that change over time or other variables?

4. What are the opportunities for standardisation of cost models and collaboration between projects?

Possible standardisation and alignment of cost models? – I think cost models always need to be tailored to some degree to different audiences/sectors and prospects for standardisation and alignment may be variable. Some areas e.g. digital storage costs may be more promising than others.

Collaboration? – I can see beneficial opportunities for both formal and informal partnerships between projects and organisations. There may be opportunities for European and international collaboration.

5. What are your initial comments and feedback on the draft decision tree appended below?

A decision tree could start much earlier and involve different decisions on the cost model itself e.g. scope of activities, level of detail, and sources of data.

6. Please provide a short one paragraph biography for yourself

Neil Beagrie is director of consultancy at Charles Beagrie and principal investigator for the JISC Keeping Research Data Safe project which has investigated the costs and benefits of digital preservation for research data. He is an experienced senior consultant and an internationally recognised expert with extensive experience in information management, digital preservation, and developing access to digital collections.

« Prev - Next »