Beagrie.com

I am pleased to report that Charles Beagrie Ltd will be the lead contractor for Keeping Research Data Safe 2: a  new JISC-funded study of the identification of long-lived digital datasets for the purposes of cost analysis.

The study aims to build on the work from the original Keeping Research Data Safe consultancy and is being undertaken by a consortium consisting of 4 partners involved in the original  study (University of Cambridge, Charles Beagrie Ltd, OCLC, and University of Southampton) and 4 new partners (the Archaeology Data Service, University of Oxford, UK Data Archive, and University of London Computer Centre) with significant data collections and interests in preservation costs. All the partners bring considerable relevant expertise, knowledge and resources to the project.

The new study will identify and analyse sources of long-lived data and develop longitudinal data on associated preservation costs and benefits. We believe these outcomes will be critical to developing preservation costing tools and cost benefit analyses for justifying and sustaining major investments in repositories and data curation.

The project will utilise the Keeping Research Data Safe cost framework as a tool for organising and scoping its work. We will undertake a combination of desk research; a data survey; analytical work with national and disciplinary digital archives that have existing historic cost data for preservation of digital research data; and interaction with digital archives in research universities who have little or no historic cost data but a strong interest in this study and identifying criteria and metrics for capturing cost data going forward.

A project website will be available shortly and regular updates on the study will be posted to this blog.

An interesting emerging digital preservation development is the Unified Digital Formats Registry (UDFR) combining efforts from the UK National Archive’s PRONOM service and Harvard University’s Global Digital Formats Registry (GDFR).

THE GDFR website notes in April 2009 the GDFR initiative joined forces with the UK National Archives’ PRONOM registry initiative under a new name – the Unified Digital Formats Registry (UDFR). The UDFR will support the requirements and use cases compiled for GDFR and will be seeded with PRONOM’s software and formats database. A new website is being constructed for the UDFR and will be available at www.udfr.org.

To quote from the UDFR Proposal and Roadmap:

” There are two major efforts underway to create a format registry with complimentary strengths and weaknesses. PRONOM, created by The National Archives (TNA) in the UK, has a strong technological base, and has been building a database of original information about various digital formats. PRONOM at this point however is owned and maintained by a single organization, making it vulnerable to changes in that institution. The Global Digital Formats Registry (GDFR) effort, hosted by Harvard University, has developed a model for a registry based on shared governance, cooperative data contribution, and distributed data hosting. However, GDFR is technically less far along in development, and has not yet begun database building.

Given the paucity of resources in the digital preservation community it would be highly unfortunate if these efforts were to compete for resources. Therefore a group of involved and interested institutions have agreed to join together to create a single shared formats registry drawing on the individual strengths of the two existing efforts. The initiative would:

• be technically based on the existing PRONOM system and database;

• create a community governance model for the registry involving all institutions willing to contribute to its development;

• develop a mechanism for the distribution of the registry data in such a way as to support local extensions and additions to the database;

• develop both technical and organizational support for distributed input to the registry, including some form of quality vetting of contributed data.”

Further details of the proposal are available from the GDFR website.

Some years ago (February 2006) Chris Rusbridge, director of the DCC,wrote a great article in Ariadne entitled “Excuse Me… Some Digital Preservation Fallacies?” . The aim of the article was to challenge some potential “digital preservation urban myths” , a number of common assertions about digital preservation that had begun to worry him.

One of the assumptions Chris challenged was that a large number of commercially-orientated file formats become rapidly obsolete and inaccessible. He had been unable to find any good examples where a large amount of content is completely inaccessible today.

Memories of Chris’s article came back to me when  reading some recent press reports in the BBC and national newspapers mentioning that:

“Britain’s National Archive estimates that it holds enough information to fill about 580,000 encyclopaedias in formats that are no longer widely available.”

So should I start emailing Chris now? I’m afraid not. I remember that 580,000 figure from an a 2007 TNA press release. Going back to it, you can see that this figure of 580,000 encyclopaedias was intended as an approximation for the TNA’s combined paper and digital records. A growing proportion of this is digital and to quote the press release ” in some instances, applications that support older file formats are no longer commercially available”.

Sorry Chris no email yet 🙂

The Dutch National Library (the KB), The Ligue des Bibliothèques Européennes de Recherche (Association of European Research Libraries – LIBER), and the Dutch Digital Preservation Coalition (NCDD) are holding a digital preservation workshop titled e-Merging New Roles and Responsibilities in the European Landscape on 17 April at the KB, The Hague, Netherlands.

The workshop aims to develop a basic understanding of the issues presented by long-term digital curation and preservation of resources which are (to be) deposited in institutional and subject-based repositories – both within research institutions and research communities. It will highlight the state of the art in digital curation and will cover best practices, including possibilities for outsourcing.

I will be chairing the afternoon session on “Policy, preconditions and costs: opportunities and pitfalls in long-term digital preservation”  with Marcel Ras, Head of the e-Depot at the KB. Attendees registering for the workshop have the opportunity to  list a specific question or problem they would like to see covered, so the session content will be tailored to your suggestions! For further information see the workshop webpages linked above.

Digital Preservation has been tipped as an emerging technology to watch by a leading IT magazine.

Yesterday’s ComputerWeekly has an  article in its IT Management section on How to beat the recession using underutilised technology by Michael Pincher. It focuses on how IT vendors can look at emerging technologies and customer requirements to innovate and begin to buck the recession.

Its an interesting article looking at overlooked areas of corporate innovation, key markets, “hype cycles”, and emerging technologies.

The emerging technologies section particularly caught my eye mentioning that digital preservation is a growth area in data management. In addition related issues such as regulatory compliance technologies, content management and repositories, infrastructure protection, storage management, and risk management are highlighted.

The list of emerging technologies is provided to give food for thought and help advise on business and innovation potential in the marketplace. The content of the article however should be of interest to a much wider readership and I highly recommend reading it.

Readers of the blog may be interested in the article Digital Archivists in Demand which appeared in the Fresh Starts column of business section of the New York Times on Saturday in both print and online editions. This is a monthly column covering emerging jobs and job trends.

The piece focusses on careers for digital asset managers, digital archivists and digital preservation officers and how demand for them is expanding. It features Jacob Nadal, the preservation officer at the University of California, Los Angeles and Victoria McCargar, a preservation consultant in Los Angeles and a lecturer at U.C.L.A. and San José State University.

Vicky McCargar estimates that 20,000 people work in the field today — plus others in related areas — and she expects that to triple over the next decade, assuming that economic conditions stabilise before long.

US rates of pay for Digital Archivists are also cited in the article. Digital asset managers at public facilities would do well to make $70,000 a year. Salaries for their corporate counterparts are generally higher. Consultants who can make recommendations on systems can make $150 an hour.Those who manage them in the commercial sector once they’re up and running make from the $70,000’s up to $100,000 a year.

Despite the higher pay in the corporate world Jacob Nadal outlines the case for working in the public sector: “Public-sector institutions just strike me as far, far cooler. They have better collections, obviously, and they are innovative, connected and challenging in ways that seem more substantial to me.”

It is good to see that mainstream newspapers are beginning to see digital archiving as an emerging career path. I have given short seminars on digital preservation and curation to students on the Information Studies courses at UCL over the last couple of years. I always emphasis to them that not only is it intellectually challenging field but a very good career option for those with a traditional archive or library training and an interest in electronic information.

JSTOR and Ithaka have recently announced the merger of their organisations. The new combined enterprise will be called Ithaka and will be dedicated to helping the academic community use digital technologies to advance scholarship and teaching and to reducing systemwide costs through collective action.

JSTOR was founded in 1995 by The Andrew W. Mellon Foundation as a shared digital library to help academic institutions save costs associated with the storage of library materials and to vastly improve access to scholarship. Today, more than 5,200 academic institutions and 600 scholarly publishers and content owners participate in JSTOR.

Ithaka was started in 2003 with funding from the Mellon Foundation and The William and Flora Hewlett Foundation and Stavros S. Niarchos Foundation. It is probably best known for incubating and hosting Portico its digital preservation service for e-journals and e-books. Ithaka is also the organisational home to NITLE, a suite of services supporting the use of technology in liberal arts education and has produced a number of influential reports including the 2007 “University Publishing in A Digital Age” and the 2008 “Sustainability and Revenue Models for Online Academic Resources.”

The merger makes sense in containing expenses at a time when endowments are under severe pressure. JSTOR and Ithaka already work closely together (for example over the Portico service) and share a common history, values, and a fundamental purpose. During 2008, the Ithaka-incubated resource Aluka was integrated into JSTOR as an initial step, further strengthening ties between the organisations. JSTOR will now join Portico and NITLE as a coordinated set of offerings made available under the Ithaka organisational name. In addition to JSTOR, Portico, and NITLE, Ithakas existing research and strategic services groups will be important parts of the enterprise.

Kevin Guthrie will remain President of Ithaka and Michael Spinella from JSTOR will become Executive Vice-President. The board will be composed of Ithaka and JSTOR Trustees, with Henry Bienen, President of Northwestern University, serving as Chairman and Paul Brest, President of the Hewlett Foundation as Vice Chairman.

I have previously blogged (see Research Data Canada) on work by The Canadian Research Data Strategy Working Group.

Its report “Stewardship of Research Data in Canada: A Gap Analysis” is now available. Using the data lifecycle as a framework, the report examines Canada’s current state versus an ‘ideal state’ based on existing international best practices across 10 indicators. The indicators include: policies, funding, roles and responsibilities, standards, data repositories, skills and training, accessibility, and preservation.

The analysis reveals significant barriers to the access and preservation of research data ’” barriers that could have a serious impact on the future of Canadian research and innovation if not addressed. For example, large amounts of data are being lost because of the woefully inadequate number of trusted data repositories in Canada.

The report summarises gaps for Canadian research data across the data lifecycle as follows:

Data Production

• Priority is on immediate use, rather than potential for long-term exploitation.
• Limited funding mechanisms to prepare data appropriately for later use.
• Few research institutions require data management plans.
• No national organization that can advise and assist with application of data standards.

Data Dissemination

• Lack of policies governing the standards applied to ensure data dissemination.
• Researchers unwilling to share data, because of lack of time and expertise required.
• Some policies require certain types of data be destroyed after a research project is over.

Long-term Management of Data

• Lack of coverage and capacity of data repositories.
• Preservation activities in repositories are not comprehensive.
• Limited funding for data repositories in Canada.
• Few incentives for researchers to deposit data into archives.

Discovery and Repurposing

• Most data rests on the hard drives of researchers and is inaccessible by others.
• Per per view and licensed access mechanisms are common where data are available.
• Many researchers are reluctant to enable access to their data because they feel it is their intellectual property.

The gap analysis will be extremely familar to many – reflecting difficulties recognised and responded to in many different countries such as the USA (Datanets), Australia (ANDS), and the UK (UKRDS feasibility study). It is pleasing to see the report cite the UK and USA as two countries that are seen internationally to be leading responses to these challenges.

It is reported that in the last several months, the Canadian Research Data Strategy Working Group has also made progress on a number of other fronts. Three Task Groups have been established to support efforts in addressing the gaps identified in the analysis. The Task Groups are:

1. Policies, funding and research;

2. Infrastructure and services; and

3. Capacity (skills, training, and reward systems). The Capacity Task Group is currently developing a workshop on data management for researchers, which it hopes to begin offering in 2009.

The next steps for the Working Group are to develop an action plan and an engagement strategy to involve senior leaders from the various institutions represented on the Working Group.

I was pleased to see that the International Blue Ribbon Task Force has issued its Interim Report on the economic issues for digital preservation brought on by the data deluge in the Information Age and the use that the interim report makes of the research undertaken by the LIFE and Keeping Research Data Safe studies.

The following press release appears on the UC San Diego website:

A blue ribbon task force, commissioned late last year to identify sustainable economic models to provide access to the ever-growing amount of digital information in the public interest, has issued its interim report. The report calls the current situation urgent, and details systemic pitfalls in developing economic models for sustainable access to digital data.

There is no time to waste, according to the new report from the Blue Ribbon Task Force on Sustainable Digital Preservation and Access, launched by the National Science Foundation and the Andrew W. Mellon Foundation in partnership with the Library of Congress, the Joint Information Systems Committee of the United Kingdom, the Council on Library and Information Resources, and the National Archives and Records Administration.

A recent study by the International Data Corporation (IDC) said that in 2007, the amount of digital data began to exceed the amount of storage to retain it, and will continue to grow faster than storage capacity from here on. The IDC study predicts that by 2011, our ‘digital universe’: consisting of digitally-based text, video, images, music, etc.: will be 10 times the size it was in 2006.

Although not all of this data should be preserved, digital data within the public interest: digital official and historical documents, research data sets, YouTube videos of presidential addresses, etc.: must be retained to maintain an accurate and complete ‘digital record’ of our society. Such digital information is now part of what is known as cyberinfrastructure, an organized aggregate of computers, networks, data, storage, software systems, and the experts who run them that is vital to our life and work in the Information Age.

‘NSF and other organizations, both national and international, are funding research programs to address these technical and cyberinfrastructure issues,’ said Lucy Nowell, Program Director for the Office of Cyberinfrastructure at the National Science Foundation. ‘This is the only group I know of that is chartered to help us understand the economic issues surrounding sustainable repositories and identify candidate solutions.’

While storage and technological issues have been at the forefront of the discussion on digital information, relatively little focus has been on the economic aspect of preserving vast amounts of digital data fundamental to the modern world.

‘The long-term accessibility and use of valuable digital materials requires digital preservation activities that are economically sustainable: in other words, provisioned with sufficient funding and other resources on an ongoing basis to achieve their long-term goals,’ said Brian Lavoie, a co-chair of the task force and a research scientist with OCLC, an international library service and research organization headquartered in Dublin, Ohio. ‘Economically sustainable digital preservation is a necessary condition for securing the long-term future of our scholarly and cultural record.’

‘Access to data tomorrow requires decisions concerning preservation today,’ said Fran Berman, director of the San Diego Supercomputer Center at the University of California San Diego, and also a co-chair on the task force. ‘The Blue Ribbon Task Forces interim report represents a year of testimony and investigation into the economic models supporting current practice in digital preservation and access across sectors.’

The interim report traces the contours of economically sustainable digital preservation, and identifies and explains the necessary conditions for achieving economic sustainability. The report also synthesizes current thinking on this topic, including testimony from 16 leading experts in digital preservation representing a variety of domains. In reviewing this synthesis, the task force identified a series of systemic challenges that create barriers to long-term, economically viable solutions. Some of these challenges include:

• Inadequacy of funding models to address long-term access and preservation needs. Funding models for efforts that incorporate digital access and preservation are often not persistent: they may be ‘one time’ efforts subsequently abandoned as more critical short-term priorities emerge.
• Confusion and/or lack of alignment between stakeholders, roles, and responsibilities with respect to digital access and preservation. Often, those who create and use digital information are not responsible for serving as stewards to support preservation and access. Consequently, the costs may not be shared, which can lead to inadequate economic models for sustainability.
• Inadequate institutional, enterprise, and/or community incentives to support the collaboration needed to reinforce sustainable economic models. Digital preservation and access require long-range planning and support, as well as agreement on formats, standards and use models, and hardware/software compatibility.
• Complacency that current practices are ‘good enough.’ The urgency of developing sustainable economic models for digital information is not uniformly appreciated. There is general agreement that leadership and competitiveness, if not institutional survival, in the Information Age depends on the persistent availability of digital information, making preservation of that information an urgent priority.
• Fear that digital access and preservation is too big to take on. There is general agreement that in its entirety, digital preservation is a big problem, incorporating technical, economic, regulatory, policy, social, and other aspects. But it is not insurmountable. Digital access and preservation may be as manageable as including a ‘data bill’ as an explicit and fixed part of an institutions business model. Successes depend on making sustainable digital access and preservation a persistent ‘line item’ on the part of stakeholders.

Continuing its work for a second and final year, the Blue Ribbon Task Force on Sustainable Digital Preservation and Access will issue its final report in late 2009 proposing practical recommendations for sustainable economic models to support access and preservation for digital data in the public interest.

To view the complete BRTF-SDPA Interim Report, click here.

For a complete list of BRTF-SDPA members, click here.

I am pleased to forward the announcement that the final report for the UK Research Data Service (UKRDS) Feasibility Study Project has been submitted and an International Conference on the UKRDS Feasibility Study will be held at The Royal Society, London on Thursday, 26 February 2009.

Booking for this international conference of senior policymakers, funders, scientists, IT managers, librarians and data service providers has now opened: Attendance at the conference is free. Places are limited, so early booking is advised.

The UKRDS feasibility study was commissioned to explore a range of models for the provision of a national infrastructure for digital research data management. It has brought together key UK stakeholders, including the Research Councils, JISC, HEFCE, British Library, Research Information Network, Wellcome Trust, researchers, and university IT and library managers, and it builds on the work of the UK’s Office of Science and Innovation e-infrastructure group. It also takes into account international developments in this area.

The UKRDS final report is due to be released soon and makes important recommendations for investment in this key part of the UK national e-infrastructure.

The study has been funded by HEFCE as part of its Shared Services programme, with additional support from JISC, Research Libraries UK (RLUK) and the Russell Group IT Directors (RUGIT). It has been led by the London School of Economics, with Serco Consulting as lead consultants supported by Charles Beagrie Limited and Grant Thornton as sub-contractors.