Scholarly Communication

Digital Preservation Handbook Update February 2016

Originally published in 2001 as a paper edition, ‘Preservation and Management of Digital Materials: a Handbook’ was the first attempt in the UK to synthesise the diverse and burgeoning sources of advice on digital preservation.  Demand was so great that in 2002, a free online edition of the Handbook was published by the newly established Digital Preservation Coalition.

After more than a decade, in which digital preservation has been transformed, the Handbook remains among the most heavily used area of the DPC website.

Funders and organisations are collaborating on re-designing, expanding and updating the Handbook so it can continue to grow as a major open-access resource for digital preservation. The DPC and Charles Beagrie Ltd have been engaged on a major re-working of the Digital Preservation Handbook for release as a new edition over 2015/2016. The National Archives (our Gold Sponsor) working together with other stakeholders including Jisc, the British Library, and The Archives and Records Association (our Silver Sponsors), and the National Records of Scotland (our Bronze Sponsor) is supporting the Digital Preservation Coalition in updating and revamping the Handbook. Many individuals and organisations are also contributing to this work through book sprints, peer review, project and advisory boards.

The revision, guided by the user feedback and consultation (see Report on the Preparatory User Consultation on the 2nd Edition of the Digital Preservation Handbook), is modular and being undertaken over a two year period to March 2016.

We have provided updates at regular intervals to inform the community on progress with the project and with this final February update we are delighted to announce a number of key developments.

 

Publication Schedule

The 2nd edition of the Handbook had a partial “soft launch” in October 2015 and approximately 2/3rds is online and publicity accessible at http://www.dpconline.org/advice/preservationhandbook

This partial release will be further enhanced by additional functionality when a new platform for the website focused on ‘responsive design’ is brought on stream by the DPC in 2016. This will provide an updated design and improved user experience on mobile and tablet devices, compared to the current site templates that are optimised for viewing on a desktop screen. We will also add the facility to generate PDFs. In the interim some functionality and content will remain “works in progress” but the community have gained early access to a significant new resource.

The remaining 14 sections to complete the Handbook have now been written, edited and are in peer review (see Handbook contents page for coming soon sections). We are aiming to complete this work and revise content for publication by the end of March 2016. The Handbook is now live so we will need to close and update section by section for these 14 remaining updates, hopefully in the final week of March and/or early April 2016. Watch this space for future announcements!

NRS joins funding group

The Digital Preservation Coalition was delighted to announce this month that The National Records of Scotland (NRS) had come on board as a ‘Bronze Sponsor’ for the eagerly anticipated second edition of the ‘Digital Preservation Handbook’. As of February 2016, with the addition of the NRS we have raised 93% of estimated funding required for the Handbook revision. We have prioritised content creation, scaled back some events, and adjusted budgets to ensure completion within a very tight funding profile.

Slideshare from Handbook Workshop at DCDC15

A workshop on the Digital Preservation Handbook was run at the DCDC15 conference in early October. Powerpoint slides from the Handbook presentation are now available on Slideshare. They provide a detailed overview of the new edition Handbook and work in progress. To date, there have been over 2,000 views of the slides.

New report: The Value and Impact of the European Bioinformatics Institute

We are pleased to announce a new report: The Value and Impact of the European Bioinformatics Institute.

In 2015, Charles Beagrie Ltd  was commissioned by the European Bioinformatics Institute (EMBL-EBI), to study and analyse its economic and social impact.

The EMBL- EBI, located on the Wellcome Genome Campus in Hinxton, near Cambridge in the UK, manages public life science data on a very large scale, making a rich resource of genome information freely available to the global life science community.

The full report published today presents the results of the quantitative and qualitative study of the Institute, examining the value and impact of its work. The report highlights key findings, including that EMBL-EBI data and services made commercial and academic R&D significantly more efficient. This benefit to users and their funders is estimated, at a minimum, to be worth £1 billion per annum worldwide – equivalent to more than 20 times the direct operational cost of EMBL-EBI.

A press release with further information is available on the EMBL-EBI website at http://www.ebi.ac.uk/about/news/press-releases/value-and-impact-of-the-european-bioinformatics-institute

The Full Report is available online in printable format at http://www.beagrie.com/EBI-impact-report.pdf

A short Executive Summary version of the report is available online in printable format at http://www.beagrie.com/EBI-impact-summary.pdf

12 slideshares for Xmas: 20 years in digital preservation

I have just posted the final instalment of a personal selection of 12 presentations drawn from events and topics over the last 20 years in digital preservation, which I hope will be of interest.

They are taken from events on four different continents including the first iPres conference and cover themes such as personal archiving, research data management, e-journals, the digital preservation lifecycle model, national and institutional strategies and collaboration, costs/benefit/economic impacts of digital preservation, the establishment of the Digital Preservation Coalition, and the development of the online Digital Preservation Handbook. I hope there will be something in there for everyone.

There are accompanying blog narratives which set the presentations into context and the powerpoint presentations themselves on Slideshare. Details and web links to them are as follows:

2014 – The Value and Impact of Research Data Infrastructure (economic impact), presentation to the Preservation and Archiving Special Interest Group (PASIG), Karlsruhe Germany    slides     narrative

2013 – Maintaining a Vision: how mandates and strategies are changing with digital content (changes and responses), keynote presentation to Screening the Future conference, London UK slides     narrative

2010 – Keeping Research Data Safe (digital preservation costs and benefits), presentation to KB Experts Workshop on Digital Preservation Costs, The Hague Netherlands          slides     narrative

2007 – Digital Preservation: Setting the Course for a Decade of Change (evolution or revolution?), keynote presentation to the Belgian Association for Documentation (ABD-BVD), Brussels Belgium              slides     narrative

2005 – Digital Preservation and Curation Summing up + Next Steps (setting curation and research agenda for2005-2015), conclusions to Warwick II Workshop, Warwick UK             slides     narrative

2005 – Plenty of Room at the Bottom? Personal Digital Libraries and Collections, keynote presentation to European Conference on Research and Advanced Technology for Digital Libraries (ECDL), Vienna Austria   slides     narrative

2004 – eScience and Digital Preservation, presentation to Association for Information Science and Technology (ASIST) conference, Rhode Island USA                  slides     narrative

2004 –  The JISC Continuing Access and Digital Preservation Strategy 2002-5(covering UK Higher Education sector and partners), presentation to the JISC-CNI conference, Brighton UK slides  narrative

2004 –Digital Preservation, e-journals and e-prints, presentation at private workshop 1st iPres conference, Beijing China                 slides     narrative

2004  –  The Digital Preservation Coalition (DPC), Its History, Programme, Rationale ,and Structure, set of 4 linked presentations to DPC Forum, London UK              slides     narrative

2001 – Preservation Management of Digital Materials (the Digital Preservation Handbook) presentation to Digital Preservation Workshop/State Library, Melbourne Australia         slides     narrative

1998 – Preserving Digital Collections: current methods and research (digital preservation lifecycle model), presentation to the Society of Archivists annual conference, Sheffield UK             slides     narrative

This is a baker’s dozen as there is a also bonus presentation from 2015 on slideshare covering the latest work on The Digital Preservation Handbook (new edition for full release in March 2016).

The background and narrative blog for this personal selection of presentations is also available.

Breaking News: Digital Preservation Handbook Update October 2015

Originally published in 2001 as a paper edition, ‘Preservation and Management of Digital Materials: a Handbook’ was the first attempt in the UK to synthesise the diverse and burgeoning sources of advice on digital preservation. Demand was so great that in 2002, a free online edition of the Handbook was published by the newly established Digital Preservation Coalition.

After more than a decade, in which digital preservation has been transformed, the Handbook remains among the most heavily used area of the DPC website.

Funders and organisations are collaborating on re-designing, expanding and updating the Handbook so it can continue to grow as a major open-access resource for digital preservation. The DPC and Charles Beagrie Ltd have been engaged on a major re-working of the Digital Preservation Handbook for release as a new edition over 2015/2016. The National Archives (our Gold Sponsor) working together with other stakeholders including Jisc, the British Library, and The Archives and Records Association (our Bronze sponsors), is supporting the Digital Preservation Coalition in updating and revamping the Handbook. Many individuals and organisations are also contributing to this work through book sprints, peer review, project and advisory boards.

The revision, guided by the user feedback and consultation (see Report on the Preparatory User Consultation on the 2nd Edition of the Digital Preservation Handbook), is modular and being undertaken over a two year period to March 2016.

We have provided updates at regular intervals to inform the community on progress with the project and with this October update we are delighted to announce a number of key developments.

Publication Schedule

We are pleased to share the news that a critical mass of content has been prepared and peer reviewed and the project board has agreed we should release a majority of the Handbook.  DPC members have already seen the emerging revised 2nd Edition of the Handbook on the members’ private area and this has been switched to the public side of the DPC website. This partial release will be further enhanced by additional functionality when a new platform for the website focused on ‘responsive design’ is brought on stream by the DPC early in 2016. This will provide an updated design and improved user experience on mobile and tablet devices, compared to the current site templates that are optimised for viewing on a desktop screen. We will also add the facility to generate PDFs. We hope to complete remaining sections of the Handbook for a formal full publication release of the Handbook by March 2016. In the interim some functionality and content will remain “works in progress” but the community will gain early access to a significant new resource.

ARA joins funding group

The Digital Preservation Coalition was delighted to announce in September that The Archives and Records Association (ARA) had come on board as a ‘Bronze Sponsor’ for the eagerly anticipated second edition of the ‘Digital Preservation Handbook’. As of Oct 2015, with the addition of the ARA we have raised 87% of estimated funding required for the Handbook revision and continue working to complete it.

Section Illustrations and icons

We are using graphics available from digitalbevaring.dk (http://digitalbevaring.dk/about-us/) for main sections of the Handbook. They have kindly worked in collaboration with us to develop new illustrations when we have identified topics in the Handbook requiring new graphics for illustrations or icons.

New resources icon designs were received over the summer from digitalbevaring.dk  and the interim versions have been replaced in the Handbook. These are the new set:            

 

They are embedded now in all the Resources and Case Studies sections of the Handbook. It means there is now a consistent style to the Handbook with the icons and section heading illustrations sharing the same design, something we all felt was desirable. We are very pleased with the results and overall look that is now in place, and with the collaboration with digitalbevaring.dk that has added a lot to the visual appeal of the Handbook.

Multi-media

Multi-media resources where relevant have been selected and embedded in the Handbook. Selection has focussed on short, high-quality videos that can add significant value to experience and content.

Handbook Workshop at DCDC15

A workshop on the Digital Preservation Handbook was run at the DCDC15 conference in early October. Powerpoint slides from the Handbook presentation are now available on Slideshare. They provide a detailed overview of the new edition Handbook and work in progress.

 

20 years in DP: eScience and Digital Preservation 2004

eScience and Digital Preservation, presentation to Association for Information Science and Technology (ASIST) conference November 2004, Rhode Island USA, available now on Slideshare is the sixth of 12 presentations I’ve selected to mark 20 years in Digital Preservation. The remainder will be published at monthly intervals over 2015.

It is closely related to the previous slideshare for May on the Jisc continuing access and digital preservation strategy but focuses just on the science component.

This is one I wasn’t able to present in person but it was kindly delivered by Gail Hodge.

My brief for the presentation was “thoughts or citations you have for the impact of e-science, particularly the GRID, on information management, particularly archiving, preservation and long-term access.”

It is a short presentation of 15 slides covering collection-based science, the Grid, data publishing, and the background and rationale for the Digital Curation Centre (just launched two weeks before in the UK).

It is a snapshot in time and of key issues in 2004 – interesting to contrast with what one would write 10 years on and ponder on progress made.

Reflections on the 2nd Digital Preservation Handbook Book Sprint 18-19 May 2015

Another rewarding but exhausting couple of days! We completed a two day book sprint in Kew earlier this week focussing on developing more new content for the release of the next edition of the Digital Preservation Handbook that is being funded by The National Archives, the British Library, and Jisc. Really pleased with the outputs and progress we made.

This is now the second book sprint we have held and we have been able to build on the sterling work at the first sprint held in October last year.

A group of 9 people Neil Beagrie (Charles Beagrie Ltd), Glenn Cumisky (British Museum), Matt Faber (Jisc), Stephen Grace (University of East London), Alex Green (The National Archives), William Kilbride (DPC), Gareth Knight (London School of Hygiene & Tropical Medicine), Sharon McMeekin (DPC), and Paul Wheatley (DPC), met up over two days to progress sections of the content for the new “ Getting Started” and “Organisational Activities” sections of the Handbook (as identified in the Draft Outline of the 2nd Edition of the Digital Preservation Handbook). We also progressed some sub-sections of “Technical Solutions and Tools” left over from Book Sprint 1. The venue for the sprint was kindly provided by The National Archives in their Kew building.

We completed draft sections for:

Getting Started

Creating digital materials

Acquisition and appraisal

Retention and review

Preservation

Metadata and documentation

Access

Information Security

Persistent Identifiers

We covered more topics than the first sprint so were occasionally thinly spread: as a cautionary note we may need to review our draft content carefully to ensure the final outputs have the breadth and depth of perspective we aim for:  what I have read so far has been terrific although inevitably it will need some more content adding and final polishing.

The revision has been guided by the user feedback and consultation (see Report on the Preparatory User Consultation on the 2nd Edition of the Digital Preservation Handbook) in short to keep the Handbook text practical, concise, and accessible with more detail available in the case studies and further reading.

We used a different tool from book sprint 1 and successfully adopted Google Docs for our collaborative writing.

A two-day book sprint was very intense but few could have spared more time away from the workplace, and a tight-deadline helped everyone focus on the tasks in hand.

We followed a process of scoping contents for a specific section, brainstorming key points for inclusion, writing, and then review.

Participants were also able to see the substantial emerging Handbook content that is already in the DPC content management system together with the excellent illustrations re-used with permission from digitalbevaring.dk. In addition Google Docs was pre-populated with any relevant text from the previous Handbook, marked in red so it was easily identifiable for review, retention, deletion, amendment or addition/replacement  as needed. The Google Docs were also pre-populated with all case studies and external resources relevant to those sections identified during desk research for the new edition of the Handbook.

The after work drinks in the Tap on the Line and group dinner at Café Mamma were enjoyed by all and allowed everyone to relax and socialise outside the event itself. Next time I will try to remember to take photos for the report!

In June the draft text will be the focus for detailed editorial review, additions, arrangement, proof-reading and input to the DPC content management system. Based on the 1st book sprint that will be at least a two month process after which we will look for peer review to be completed by around the end of September.

It is great to see so much more of the new Handbook there in preliminary form after the sprint. With the contents of the first sprint, supplementary work, and its peer review, there is now substantial draft content emerging for the 2nd edition of the Handbook.

Digital Preservation Technology Watch Reports pass 100,000 Downloads

Last month the new series of Digital Preservation Coalition Technology Watch Reports passed 118,842 downloads: these are downloads by real users excluding robots etc.

The new series was launched publicly in February 2012 with Preserving Email by Chris Prom and there are now 10 titles published since that date. All have proved very popular: Preserving Email still heads the group with over 29,000 downloads (but has been available for longest), followed by Digital Forensics and Preservation with over 25,000, and Preserving Moving Picture and Sound with over 15,000.

The reports are published by the Digital Preservation Coalition (DPC) in association with Charles Beagrie Ltd as editors and Neil Beagrie as Principal Investigator and managing editor of the series. The series is intended as an advanced introduction to specific issues for those charged with establishing or running services for long term access.  They identify and track developments in IT, standards and tools which are critical to digital preservation activities. All are released as peer-reviewed open-access publications after a preview period of exclusive access to DPC members.

The DPC Technology Watch Report Series publications are freely available online from the DPC website at: http://www.dpconline.org/advice/technology-watch-reports

Digital Preservation of e-journals and e-prints, presentation at 1st iPRES conference, Beijing China 2004

Digital Preservation of e-journals and e-prints presentation 1st iPRES conference Beijing China2004 available now on Slideshare is the fourth of 12 presentations I’ve selected to mark 20 years in Digital Preservation. The remainder will be published at monthly intervals over 2015.

This is one of the four presentations I gave as part of the very first iPRES conference at the Chinese National Academy of Sciences in July 2004.

iPRES was conceived in 2004 by the Chinese Academy of Science and Electronic Information for Libraries, as a forum to exchange ideas and expertise in digital preservation between China and Europe. Since then, it has expanded to attract delegates from around the world. For me it was a great founding conference with wonderful hospitality: there can be few better places in the world to visit if you like archaeology, culture and food.

I have chosen the e-journals preservation presentation from the conference as it is a thread of work that I have returned to many times over the last decade (including a DPC Technology Watch report in 2013).

It was also the most intriguing, as I gave an additional invited private seminar on the topic of preservation and continuing access for international STM e-journals to Chinese colleagues during the conference. At the time tensions were rising over Taiwan: it was probably the only time I sensed digital preservation was felt to be (and probably could be funded as) a national security issue….

 

Public release of the OAIS Introductory Guide (2nd Edition) DPC Technology Watch Report

We are pleased to announce the public release of the OAIS Introductory Guide (2nd Edition), the latest report in the Digital Preservation Coalition (DPC) Technology Watch Series. Written by Brian Lavoie of OCLC Research, and published by the DPC in association with Charles Beagrie Ltd, this free peer-reviewed report  looks back on the development, features, and impact of the Open Archival Information System (OAIS) Reference Model, one of the core standards of digital preservation.

Research Scientist at OCLC, Brian observes that perhaps “the most important achievement of the OAIS in this history is that it has become almost universally accepted as the lingua franca of digital preservation”

Emphasising its flexibility and conceptual nature, the report describes the OAIS, its core principles and functional elements, as well as the information model which support long-term preservation, access and understandability of data – highlighting the in-built level of abstraction which makes it such a widely applicable foundation resource for digital preservation.

Brian adds “it is possible to identify a few limitations associated with the OAIS’s impact,” generally associated with the very conceptual nature of the model, and goes on to recommend that the digital preservation community would certainly “benefit from a careful assessment of where more precise and authoritative definitions of OAIS concepts and relationships would accelerate progress in achieving robust, widely applicable, and interoperable digital preservation solutions.”

The Introduction to OAIS was the first of the DPC Technology Watch reports, and although it was first published a decade ago it has remained popular. The second edition updates and expands this first report, providing an excellent introduction to the OAIS for those new to digital preservation and resource for practitioners wishing to re-acquaint themselves with the basics of the model, supplemented by the wisdom of a decade of research, development and implementation.

Sarah Higgins of the Department of Information Studies at Aberystwyth University praises the report, calling it “a much needed and important update. It lays out both the content of the second edition of the OAIS Reference Model, and the results of over a decade of research and development that can trace its roots to OAIS. The tools and processes for practical implementation of digital preservation and measuring their success are expertly explained and evaluated. The report will be invaluable to both established and new entrants to the digital preservation profession who need to understand the basic concepts of an OAIS and the tools available to them. This clear and comprehensive report will be embedded as core reading for Aberystwyth University students studying Digital Curation or Digital Information Management at Master’s level”.

OAIS Introductory Guide (2nd Edition) is the latest in the state of the art Technology Watch Reports that give an advanced introduction to ensuring that high-value and vulnerable digital resources can be managed beyond the limits of technological obsolescence.

Read Brian Lavoie’s Technology Watch Report ‘OAIS Introductory Guide (2nd Edition)’ now.

Reflections on the Digital Preservation Handbook Book Sprint 28-29 October 2014

What a terrific couple of days! We completed a two day book sprint in London last week focussing on developing new content for the first release of the next edition of the Digital Preservation Handbook that is being funded by The National Archives, the British Library, and Jisc. Really pleased with the outputs and progress we made.

A group of 11 people Matthew Addis (Arkivum), Neil Beagrie (Charles Beagrie Ltd), Stephanie Davidson (West Yorkshire Archive Service), Michael Day (British Library), Matt Faber (Jisc), Chris Fryer (Parliamentary Archives), Anna Henry (the Tate Gallery), William Kilbride (DPC), Ed Pinsent (ULCC), Virginia Power (Jisc), Susan Thomas (Bodleian Library Oxford), met up over two days to progress sections of the content for the new “Technical Solutions and Tools” chapter of the Handbook (as identified in the Draft Outline of the 2nd Edition of the Digital Preservation Handbook). Accommodation for the sprint was kindly provided by the Jisc in their central London offices via the good offices of Neil Grindley.

We have completed draft sections for:

  • Tools (including guidance on Tool Registries)
  • Media and Storage
  • File Formats
  • Digital Forensics

In addition a content outline was agreed for the “Getting Started” sub-section of the Introduction.  Alongside this work, other sections including the Background, How to Use the Handbook, Definitions and Concepts, Acronyms and Initials, and References have been partially revised as we went.

The revision has been guided by the user feedback and consultation (see Report on the Preparatory User Consultation on the 2nd Edition of the Digital Preservation Handbook) in short to keep the Handbook text practical, concise, and accessible with more detail available in the case studies and further reading.

This was the first book sprint for all bar one of the participants. We learnt a lot about the strengths and weaknesses of “Booktype” the open source software we used that had been developed to help support this type of activity, eventually settling on using it in parallel with collaborative text tools such as Google Docs to get the best from each approach. A two-day book sprint was very intense but few could have spared more time away from the workplace, and as one participant said a tight-deadline helped everyone focus on the tasks in hand.

At the end of the sprint the challenge was set to aim to make the new content available within 3 months – we hope sufficient additional sections to create a ready critical mass, potentially the complete Tools and Solutions Chapter of the Handbook can be readied and transferred to the DPC website and reviewed for release in the New Year.

Next »