Clever Recordkeeping Metadata Project Research Method and Findings
The research methodology was designed within an action-research framework where a close alignment between the practical development of tools and active reflection on each stage of their development iteratively informs both the further development of the tools and also identifies challenges and issues to be addressed in an ongoing fashion.
The research involved the initial development of a proof-of-concept prototype to demonstrate that metadata re-use is possible and illustrate the business utility of recordkeeping metadata. From that initial proof of concept, the project intended to develop a more robust demonstrator available for wider dissemination.
First Iteration: Development of Proof of Concept Prototype
The first iteration of the CRKM Project investigated a simple solution for demonstrating the automated capture and re-use of recordkeeping metadata. The expectation was that this initial investigation would expose the complex network of issues to be addressed in order to achieve metadata interoperability and automate the movement of recordkeeping metadata between systems, along with enabling researchers to develop skills and understandings of the existing technologies that support metadata translation and transformation.
The simple solution involved instantiating a scenario in which to show:-
- Extraction of standard compliant metadata from the source system
- Automated translation to the target metadata standard using existing technologies
- Ingestion of the translated standard compliant metadata into a target system
The scenario used for testing was that of a policy development process common to almost all organisations.
Key Findings and Challenges
Recordkeeping metadata re-use is possible but feasibility and sustainability are key issues
The case was made to further develop the conceptualisation of a metadata broker as infrastructure for scalable and sustainable interoperability.
Systems integration is constrained by interoperability limitations of current business, records and archival management applications
To overcome these constraints there is a need to conceive of the metadata broker as a service within an integrated systems environment based on a service oriented architecture (SOA).
Existing metadata standards are not as interoperable as their developers have assumed
Recordkeeping metadata re-use is complex and dynamic. There is a need to further understandings of the types of interoperability associated with re-using recordkeeping metadata in other contexts and associated with the re-use of data/metadata as recordkeeping metadata. There is also a need to explore dynamic models for recordkeeping metadata.
Recordkeeping metadata re-use is constrained by workflow configured around retrospective metadata creation as per paper records management and archival processes
There is a need to re-engineer and re-positioning recordkeeping and archiving processes in line with a continuum view.
No concrete cost/benefit evaluation framework that can be used to assess the business case for metadata
There is a need to develop a cost/benefit model for recordkeeping metadata.
- Proof-of-Concept Demonstrator Prototype - First Iteration Report (DOC, 0.34 MB) - This document reports on the first iteration of a demonstrator of automated recordkeeping metadata capture and re-use.
- List of unsuccessful or problematic metadata value translations (DOC, 0.05 MB) - This document outlines the unsuccessful or problematic metadata value translations that occurred for the first iteration of the proof-of-concept prototype.
- Metadata Spreadsheets (XLS, 0.12 MB) - This spreadsheet was used to instantiate metadata instances for the scenario.
- Metadata instances and stylesheets
- RKMS Metadata - Planning File (XML, 0.01 MB)
- RKMS Metadata - Project Plan (XML, 0.01 MB)
- RKMS Metadata - Project Plan Approval Email (XML, 0.01 MB)
- RKMS Metadata - Policy File (XML, 0 MB)
- RKMS Metadata - Policy Document (XML, 0.01 MB)
- RKMS Metadata - Authorisation Email for Publication of Policy Document (XML, 0 MB)
- AGLS Metadata - Translation of Policy Document RKMS Metadata (XML, 0 MB)
- CRS Item Metadata - Translation of Policy Document RKMS Metadata (XML, 0 MB)
- CRS Item Metadata - Translation of Authorisation Email for the Policy Document RKMS Metadata (XML, 0 MB)
- CRS Series Metadata - Translation of RKMS Metadata (XML, 0 MB)
- XSLT - RKMS to AGLS (XSL, 0 MB)
- XSLT - RKMS to CRS Series (XSL, 0 MB)
Second Iteration: Demonstrator of Automated Recordkeeping Metadata Capture and Re-use
Following the outcomes of the first iteration prototype, the aspiration for the second iteration involved:-
- developing the CRKM Broker as a cluster of web services
- revising scenario processes in line with continuum and service oriented thinking,
- testing use of the CRKM Broker within a SOA framework, and
- developing a case for recordkeeping metadata within a SOA framework.
The second iteration:-
- created a working model of translation and registry services as components of a metadata broker using web services technologies,
- demonstrated the use of the ebXML Registry specification to provide core registry and repository management functions, and
- developed the CRKM Registry Information Model (CRKM RIM) to specify descriptive requirements for negotiating use of schemas and crosswalks in translation services.
Key Findings and Challenges
There are significant barriers to interoperability within our current metadata standards and in our current records management and archival control frameworks.
Recordkeeping metadata schemas are designed for human rather than machine processing and at abstract rather than representational level. Semantics are imprecise and ambiguous - particularly with the mix of entity and relationship description.
Recordkeeping control tools are designed for paper records management - for example for true interoperability, we want to have access to recordkeeping requirements in machine processable forms rather than as values derived disposal classes. This will require rethinking and redesigning recordkeeping systems.
Translation beyond a web services environment into a fully realised service oriented architecture is outstripping implementation reality, with current technology constraints illustrating that truly service oriented implementations are really things of the future
Our community has an opportunity to evolve towards that future via web services used initially to wrap legacy systems to achieve data interoperability as we progressively move towards decomposing and re-engineering recordkeeping functionality as services and creating appropriate business process and rules infrastructure.
The following chapters of Joanne Evans's PhD thesis,Building Capacities for Sustainable Recordkeeping Metadata Capabilitiesdocument these activities:
- Chapter 6 - Conceptualising a Metadata Broker
- Chapter 7 - The Second Iteration of the CRKM Demonstrator
- Appendix B - Registry Initiatives
- Appendix C - Developing the Requirements for the Registry Component of the CRKM Broker