Clever Recordkeeping Metadata Project Research Method and Findings

The research methodology was designed within an action-research framework where a close alignment between the practical development of tools and active reflection on each stage of their development iteratively informs both the further development of the tools and also identifies challenges and issues to be addressed in an ongoing fashion.

The research involved the initial development of a proof-of-concept prototype to demonstrate that metadata re-use is possible and illustrate the business utility of recordkeeping metadata.  From that initial proof of concept, the project intended to develop a more robust demonstrator available for wider dissemination.

First Iteration: Development of Proof of Concept Prototype

The first iteration of the CRKM Project investigated a simple solution for demonstrating the automated capture and re-use of recordkeeping metadata.  The expectation was that this initial investigation would expose the complex network of issues to be addressed in order to achieve metadata interoperability and automate the movement of recordkeeping metadata between systems, along with enabling researchers to develop skills and understandings of the existing technologies that support metadata translation and transformation.

The simple solution involved instantiating a scenario in which to show:-

  • Extraction of standard compliant metadata from the source system
  • Automated translation to the target metadata standard using existing technologies
  • Ingestion of the translated standard compliant metadata into a target system

The scenario used for testing was that of a policy development process common to almost all organisations.

Key Findings and Challenges

Recordkeeping metadata re-use is possible but feasibility and sustainability are key issues

The case was made to further develop the conceptualisation of a metadata broker as infrastructure for scalable and sustainable interoperability.

Systems integration is constrained by interoperability limitations of current business, records and archival management applications

To overcome these constraints there is a need to conceive of the metadata broker as a service within an integrated systems environment based on a service oriented architecture (SOA).

Existing metadata standards are not as interoperable as their developers have assumed

Recordkeeping metadata re-use is complex and dynamic. There is a need to further understandings of the types of interoperability associated with re-using recordkeeping metadata in other contexts and associated with the re-use of data/metadata as recordkeeping metadata. There is also a need to explore dynamic models for recordkeeping metadata.

Recordkeeping metadata re-use is constrained by workflow configured around retrospective metadata creation as per paper records management and archival processes

There is a need to re-engineer and re-positioning recordkeeping and archiving processes in line with a continuum view.

No concrete cost/benefit evaluation framework that can be used to assess the business case for metadata

There is a need to develop a cost/benefit model for recordkeeping metadata.

Key Documents

Second Iteration: Demonstrator of Automated Recordkeeping Metadata Capture and Re-use

Following the outcomes of the first iteration prototype, the aspiration for the second iteration involved:-

  • developing the CRKM Broker as a cluster of web services
  • revising scenario processes in line with continuum and service oriented thinking,
  • testing use of the CRKM Broker within a SOA framework, and
  • developing a case for recordkeeping metadata within a SOA framework.

The second iteration:-

  • created a working model of translation and registry services as components of a metadata broker using web services technologies,
  • demonstrated the use of the ebXML Registry specification to provide core registry and repository management functions, and
  • developed the CRKM Registry Information Model (CRKM RIM) to specify descriptive requirements for negotiating use of schemas and crosswalks in translation services.

Key Findings and Challenges

There are significant barriers to interoperability within our current metadata standards and in our current records management and archival control frameworks.

Recordkeeping metadata schemas are designed for human rather than machine processing and at abstract rather than representational level. Semantics are imprecise and ambiguous - particularly with the mix of entity and relationship description.

Recordkeeping control tools are designed for paper records management - for example for true interoperability, we want to have access to recordkeeping requirements in machine processable forms rather than as values derived disposal classes.  This will require rethinking and redesigning recordkeeping systems.

Translation beyond a web services environment into a fully realised service oriented architecture is outstripping implementation reality, with current technology constraints illustrating that truly service oriented implementations are really things of the future

Our community has an opportunity to evolve towards that future via web services used initially to wrap legacy systems to achieve data interoperability as we progressively move towards decomposing and re-engineering recordkeeping functionality as services and creating appropriate business process and rules infrastructure.

Key Documents

The following chapters of Joanne Evans's PhD thesis,Building Capacities for Sustainable Recordkeeping Metadata Capabilitiesdocument these activities:

  • Chapter 6 - Conceptualising a Metadata Broker
  • Chapter 7 - The Second Iteration of the CRKM Demonstrator
  • Appendix B - Registry Initiatives
  • Appendix C - Developing the Requirements for the Registry Component of the CRKM Broker