Beagrie.com

The Warwick3 Workshop: Digital Preservation and Curation Summing up + Next Steps available now on Slideshare is the eighth of 12 presentations I have selected to mark 20 years in Digital Preservation. The remainder will be published at monthly intervals over 2015.

I have chosen it as it briefly allows us to look back at aspirations and achievements in Digital Preservation over a 20 year period from the very first (and seminal) Warwick 1 workshop held in 1995 to today. The first Warwick workshop considered the Long Term Preservation of Electronic Materials and a UK response to the final report of the RLG/CPA Task Force on Digital Archiving. Two further Warwick workshops followed in 1999 and 2005 to review progress and set a forward agenda.

The two-day workshop that took place over 7 – 8 November 2005 at the University of Warwick aimed for the first time to address digital preservation issues for both scientific data and cultural heritage and to map out a future research agenda for them. Sponsored by JISC, the Digital Curation Centre (DCC), the British Library and the Council for the Central Laboratory of the Research Councils (CCLRC), the invitation-only event drew a wide range of national and international experts to explore the current state of play with a view to shaping future strategy. The slides are from my summing up and conclusions at the workshop close.

Part of my conclusions (slides 12-13), outlined the recommendations of the previous Warwick workshop held in 1999 and reviewed the progress that had been made in implementing them over the subsequent five years with a very subjective level of achievement √ (some) to √ √ √ (good) as follows:

Raise awareness

√ √ √ DPC advocacy, EU council, UNESCO, CODATA, ICSTI, NSF,RCUK

Encourage cross-sectoral communication

√ √ Established Digital Preservation Coalition 2001 – now 27 members

Develop guidelines

√ √ Preservation Management Handbook, Curation Manual, Cornell tutorial

Preservation Centre/Network of centres

√ √ Digital Curation Centre, British Library, The National Archives

Certification criteria

√ RLG/NARA checklist (TRAC)

Checklist to determine complexity and cost

√JISC 04/04 funding programme (LIFE project, assessment tool project)

New research – emulation, dynamic data

√Camileon project, JISC 04/04 programme, DCC research agenda

So how have we done 10 years further on?  Overall, OK I think with the caveat progress in digital preservation can take a long time. Perhaps I would raise the achievement levels if doing this exercise again in 2015 for “Encourage cross-sectoral communication”, “Checklist to determine complexity and cost”, and “New research”. However I would probably move Raise Awareness down one level. The others would probably be about the same. How about you?

Plenty of Room at the Bottom? Personal Digital Libraries and Collections, my keynote presentation to the European Conference on Research and Advanced Technology for Digital Libraries (ECDL), 2005, in Vienna Austria available now on Slideshare is the seventh of 12 presentations I’ve selected to mark 20 years in Digital Preservation. The remainder will be published at monthly intervals over 2015.

This presentation represents a thought piece and call to arms to focus more on the collection and preservation of personal digital archives. It was given as a keynote to ECDL but also formed the core of my Banks Lecture at the University of Texas in April 2006 on Preservation and Access for Personal Digital Archives and Literary Papers.

Many of the ideas in the presentation were developed in greater detail in an article in D-Lib June 2005 Plenty of Room at the Bottom? Personal Digital Libraries and Collections, in my contribution to the Memories for Life project (c.2004-2006) and our publication in the Royal Society Interface Journal in June 2006 Memories for Life: a review of the science and technology, and in my initial work as Principal Investigator on the Digital Lives research project involving the British Library and UCL. It is an area of interest I had to leave behind on departing the BL and focussing full-time on consultancy. However it has been great to be editor on behalf of the DPC for the forthcoming Technology Watch Report by Gabriela Redwine on Preserving Personal Digital Archives that should be released later this year on the DPC website.

Over recent years this area has blossomed with an annual conference since 2010 on Personal Digital Archiving and many special collections and research projects developed in libraries. We are beginning to see mass market shared services for lifelogging and personal collection emerging but the key focus of growth currently seems to be on health data. Broader issues though for the public are still surfacing: there has been growing publicity around digital legacy issues for social media and even guidance from the Law Society in the UK on digital legacy and executors. It remains a fascinating area for digital preservation.

eScience and Digital Preservation, presentation to Association for Information Science and Technology (ASIST) conference November 2004, Rhode Island USA, available now on Slideshare is the sixth of 12 presentations I’ve selected to mark 20 years in Digital Preservation. The remainder will be published at monthly intervals over 2015.

It is closely related to the previous slideshare for May on the Jisc continuing access and digital preservation strategy but focuses just on the science component.

This is one I wasn’t able to present in person but it was kindly delivered by Gail Hodge.

My brief for the presentation was “thoughts or citations you have for the impact of e-science, particularly the GRID, on information management, particularly archiving, preservation and long-term access.”

It is a short presentation of 15 slides covering collection-based science, the Grid, data publishing, and the background and rationale for the Digital Curation Centre (just launched two weeks before in the UK).

It is a snapshot in time and of key issues in 2004 – interesting to contrast with what one would write 10 years on and ponder on progress made.

Another rewarding but exhausting couple of days! We completed a two day book sprint in Kew earlier this week focussing on developing more new content for the release of the next edition of the Digital Preservation Handbook that is being funded by The National Archives, the British Library, and Jisc. Really pleased with the outputs and progress we made.

This is now the second book sprint we have held and we have been able to build on the sterling work at the first sprint held in October last year.

A group of 9 people Neil Beagrie (Charles Beagrie Ltd), Glenn Cumisky (British Museum), Matt Faber (Jisc), Stephen Grace (University of East London), Alex Green (The National Archives), William Kilbride (DPC), Gareth Knight (London School of Hygiene & Tropical Medicine), Sharon McMeekin (DPC), and Paul Wheatley (DPC), met up over two days to progress sections of the content for the new “ Getting Started” and “Organisational Activities” sections of the Handbook (as identified in the Draft Outline of the 2nd Edition of the Digital Preservation Handbook). We also progressed some sub-sections of “Technical Solutions and Tools” left over from Book Sprint 1. The venue for the sprint was kindly provided by The National Archives in their Kew building.

We completed draft sections for:

Getting Started

Creating digital materials

Acquisition and appraisal

Retention and review

Preservation

Metadata and documentation

Access

Information Security

Persistent Identifiers

We covered more topics than the first sprint so were occasionally thinly spread: as a cautionary note we may need to review our draft content carefully to ensure the final outputs have the breadth and depth of perspective we aim for:  what I have read so far has been terrific although inevitably it will need some more content adding and final polishing.

The revision has been guided by the user feedback and consultation (see Report on the Preparatory User Consultation on the 2nd Edition of the Digital Preservation Handbook) in short to keep the Handbook text practical, concise, and accessible with more detail available in the case studies and further reading.

We used a different tool from book sprint 1 and successfully adopted Google Docs for our collaborative writing.

A two-day book sprint was very intense but few could have spared more time away from the workplace, and a tight-deadline helped everyone focus on the tasks in hand.

We followed a process of scoping contents for a specific section, brainstorming key points for inclusion, writing, and then review.

Participants were also able to see the substantial emerging Handbook content that is already in the DPC content management system together with the excellent illustrations re-used with permission from digitalbevaring.dk. In addition Google Docs was pre-populated with any relevant text from the previous Handbook, marked in red so it was easily identifiable for review, retention, deletion, amendment or addition/replacement  as needed. The Google Docs were also pre-populated with all case studies and external resources relevant to those sections identified during desk research for the new edition of the Handbook.

The after work drinks in the Tap on the Line and group dinner at Café Mamma were enjoyed by all and allowed everyone to relax and socialise outside the event itself. Next time I will try to remember to take photos for the report!

In June the draft text will be the focus for detailed editorial review, additions, arrangement, proof-reading and input to the DPC content management system. Based on the 1st book sprint that will be at least a two month process after which we will look for peer review to be completed by around the end of September.

It is great to see so much more of the new Handbook there in preliminary form after the sprint. With the contents of the first sprint, supplementary work, and its peer review, there is now substantial draft content emerging for the 2^nd edition of the Handbook.

Digital Preservation of e-journals and e-prints presentation 1^st iPRES conference Beijing China2004 available now on Slideshare is the fourth of 12 presentations I’ve selected to mark 20 years in Digital Preservation. The remainder will be published at monthly intervals over 2015.

This is one of the four presentations I gave as part of the very first iPRES conference at the Chinese National Academy of Sciences in July 2004.

iPRES was conceived in 2004 by the Chinese Academy of Science and Electronic Information for Libraries, as a forum to exchange ideas and expertise in digital preservation between China and Europe. Since then, it has expanded to attract delegates from around the world. For me it was a great founding conference with wonderful hospitality: there can be few better places in the world to visit if you like archaeology, culture and food.

I have chosen the e-journals preservation presentation from the conference as it is a thread of work that I have returned to many times over the last decade (including a DPC Technology Watch report in 2013).

It was also the most intriguing, as I gave an additional invited private seminar on the topic of preservation and continuing access for international STM e-journals to Chinese colleagues during the conference. At the time tensions were rising over Taiwan: it was probably the only time I sensed digital preservation was felt to be (and probably could be funded as) a national security issue….

What a terrific couple of days! We completed a two day book sprint in London last week focussing on developing new content for the first release of the next edition of the Digital Preservation Handbook that is being funded by The National Archives, the British Library, and Jisc. Really pleased with the outputs and progress we made.

A group of 11 people Matthew Addis (Arkivum), Neil Beagrie (Charles Beagrie Ltd), Stephanie Davidson (West Yorkshire Archive Service), Michael Day (British Library), Matt Faber (Jisc), Chris Fryer (Parliamentary Archives), Anna Henry (the Tate Gallery), William Kilbride (DPC), Ed Pinsent (ULCC), Virginia Power (Jisc), Susan Thomas (Bodleian Library Oxford), met up over two days to progress sections of the content for the new “Technical Solutions and Tools” chapter of the Handbook (as identified in the Draft Outline of the 2nd Edition of the Digital Preservation Handbook). Accommodation for the sprint was kindly provided by the Jisc in their central London offices via the good offices of Neil Grindley.

We have completed draft sections for:

• Tools (including guidance on Tool Registries)
• Media and Storage
• File Formats
• Digital Forensics

In addition a content outline was agreed for the “Getting Started” sub-section of the Introduction.  Alongside this work, other sections including the Background, How to Use the Handbook, Definitions and Concepts, Acronyms and Initials, and References have been partially revised as we went.

The revision has been guided by the user feedback and consultation (see Report on the Preparatory User Consultation on the 2nd Edition of the Digital Preservation Handbook) in short to keep the Handbook text practical, concise, and accessible with more detail available in the case studies and further reading.

This was the first book sprint for all bar one of the participants. We learnt a lot about the strengths and weaknesses of “Booktype” the open source software we used that had been developed to help support this type of activity, eventually settling on using it in parallel with collaborative text tools such as Google Docs to get the best from each approach. A two-day book sprint was very intense but few could have spared more time away from the workplace, and as one participant said a tight-deadline helped everyone focus on the tasks in hand.

At the end of the sprint the challenge was set to aim to make the new content available within 3 months – we hope sufficient additional sections to create a ready critical mass, potentially the complete Tools and Solutions Chapter of the Handbook can be readied and transferred to the DPC website and reviewed for release in the New Year.

A big thank-you from Neil Beagrie and William Kilbride to everyone who contributed to the recent audience research survey or who  commented on the potential contents outline for the new edition of the Digital Preservation Handbook.

Following that work, the DPC and Charles Beagrie Ltd are delighted to announce the release two important documents which will form the foundations of the new edition of the DPC Digital Preservation Handbook: the results of a major survey into audience needs, an the first full outline of content.

‘We are very keen to make sure that the new edition of the handbook fits with people’s actual needs so we were very encouraged by the substantial response to the consultation document which we sent out before summer’ explained Neil Beagrie who is editor and lead author of the new edition of the handbook. ‘We estimate that the digital preservation community represented on the JiscMail list numbers around 1500 people in total: and there were 285 responses to the survey.’

‘It a very large sample of the community but it’s also re-assuringly diverse.  There’s a strong representation from higher education and public sector agencies but there’s also a sizeable group from industry, from charities as well as museums and community interest groups.  When asked if they would use the handbook, not a single respondent said no.’

‘The survey has directly informed the contents of the new handbook’, explained William Kilbride, Execuitve Director of the DPC.  ‘We started with an idea of the gaps and the many parts that had become outdated since the original handbook was published.  So we invited users to tell us what they wanted and how they wanted it – both in terms of content and presentation.  The project team has responded thoughtfully to these requests so I am confident that the resulting list of content is tailored to people’s needs. But we remain open to suggestions and comments’

‘This will help ensure that the handbook remains relevant for many years to come.’

The two documents are available as follows:

• Report on the Preparatory User Consultation on the 2nd Edition of the Digital Preservation Handbook

• Draft Outline of the 2nd Edition of the Digital Preservation Handbook

A colleague has pointed out that our synthesis report for Jisc on the Value and Impact of Data Sharing and Curation has had over 3,900 downloads since April 2014. You can see the stats and access the report here on the Jisc Repository.

It is great to see that there is a very high level of interest in the topic and report. I’m not sure how that figure compares, but if you have done work for Jisc you should now be able to search or browse the Jisc repository and see the download stats for your own work. Potentially, access to the Jisc repository stats is going to be very useful for those involved in REF or needing to demonstrate their  impact to their institutions and other stakeholders.

We are pleased to announce the publication of Preserving eBooks, the latest in the series of Digital Preservation Coalition (DPC) Technology Watch Reports. Written by Portico’s Amy Kirchhoff and Sheila Morrissey, and published in association with Charles Beagrie Ltd as managing editors, the report discusses the current developments and issues with which public, national and higher education libraries, publishers, aggregators and preservation institutions must contend to ensure long-term access to eBook content.

Archive Services Product Manager for Portico in the USA, Amy explains that “an increasingly ‘digital native’ population with new expectations such as efficient automated search, retrieval and re-use of information, as well as cost pressures on the production and storage of new publications, have made the eBook as a mode of publication a fact on the ground for the foreseeable future.”

With this in mind, the report examines legal questions about the use, re-use, sharing and preservation of eBook objects; format issues, including the sometimes tight coupling of eBook content with particular hardware platforms; the embedding of digital rights management artefacts in eBook files to restrict access to them; and the diverse business ecosystem of eBook publication, with its associated complexities of communities of use and, ultimately, expectations for preservation.

Sheila adds that “while large-scale digitization of print books has created valuable and widely used digital surrogates for those books that are being put to uses impossible with print books, it has also introduced certain quality assurances issues, and has also embroiled institutions in legal entanglements arising from both the eBook’s similarity to, and difference from, its print source.”

Collections Management Officer at the University of North Carolina, Luke Swindler, admires the way the report “sketches the salient issues at levels and in terms that its varied audiences can understand,” and goes on to observe that “a major strength of the report is the recognition of and close attention to the coupling of e-book content with its corresponding software (including intellectual property and digital rights management restrictions) and the hardware/platform envelope.”

While Preserving eBooks will be well received by libraries, scholars and publishers, the report also includes generic lessons in this field of interest for the wider digital preservation community, covering relevant legal, economic and service issues.

You can read Amy Kirchhoff and Sheila Morrissey’s report Preserving eBooks by downloading it from the DPC website here.

We are pleased to announce that The National Archives (TNA) has published our new guidance on Cloud Storage and Digital Preservation, with five accompanying case studies.

The Guidance and case studies have been created for TNA to address questions archivists have raised about digital preservation and cloud storage. The guidance is written by a Charles Beagrie team comprising of Neil Beagrie, Paul Miller, and Andrew Charlesworth.

The Guidance is now available to download here.

Of particular interest to many archivists will be the experience of our case studies, which are available as separate PDFs from the url above. These are as follows: Dorset History Centre, Parliamentary Archives, Tate Gallery, University of Oxford, and the Archives and Records Council Wales Digital Preservation Consortium.

To accompany the publication of the Guidance, we held a webinar for archivists on digital preservation and the cloud on 13May 2014, the recording of which will be accessible soon on TNA’s website. A further announcement will be made when that is available.