Data Archive Interoperability Working Group Report Fall 2019

Achievements of the DAI WG include progress on OAIS standards, audit metrics, and architecting trustworthy digital repositories. The group faces challenges with resource allocation for updating Control Authority Bases. Despite good momentum, support from consistent participants is crucial for continued success.

• Data Archive
• Interoperability
• DAI WG
• Standards
• Digital Preservation

mar_sh Follow

Uploaded on Sep 16, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Data Archive Interoperability DAI WG Report to MOIMS Darmstadt Germany Fall 2019 David Giaretta (WG Chair) John Garrett (WG Deputy Chair) Steve Hughes (Presenter) 24 October 2019 MOIMS Fall 2019 - 1

2. DAI WG Executive Summary Achievements for this meeting cycle: CCSDS 650.0 (ISO 14721) OAIS v3 CCSDS 652.0 (ISO 16363) Audit Metrics About half way through addressing submitted comments CCSDS 652.1 (ISO 169191) Guidelines for Auditing Bodies IPELTU OAIS-IF Architecture Description document Certifications as Trustworthy Digital Repository based on our Standards US Government Publishing Office system Indira Gandhi Audiovisual Archive CERN Responded to Deputy Area Director comments Responded to comments from new ESA participants Ready for formal review Initiated new project and requested approval Continued expansion of annexes, e.g. for Space Mission programs, including input from ISO 19165-2 (Draft Waiting final approval) Significant Modelling activity Upcoming emphasis on Services covers all government publications received initial certification and moving towards its surveillance audit suspended pending scheduling of surveillance audit working towards initial certification Large number of earlier DAI WG standards were reconfirmed following the last meeting 24 October 2019 MOIMS Fall 2019 - 2

3. DAI WG Executive Summary Working Group Status: GOOD Good Momentum projects schedules updated, Massive OAIS effort ending, Good progress on Audit books looking to complete before next meeting, Architecture work accelerating Both Agency and outside participation in DAI is stable, but support from consistent participants is hard to maintain Several Agencies (and outside participants) are expressing keen interest in WG activities Limited in person attendance at this meeting, (competing conferences) with additional virtual attendance via Skype Support from remaining Agencies is needed Interaction with other WGs We did not hold any joint meetings during this meeting cycle 24 October 2019 MOIMS Fall 2019 - 3

4. DAI WG Executive Summary Problems and Issues: This CCSDS Meeting overlapped with other Space conference as well as other Digital Preservation conference DAI has not identified any resources to prototype any updates to Control Authority BBs Request made to DAI to update these BBs to use SANA registry 630.0-B-1 Standard Formatted Data Units Control Authority Procedures 631.0-G-2 Standard Formatted Data Units Control Authority Procedures Tutorial 632.0-B-1 Standard Formatted Data Units Control Authority Data Structures Any BB update to refer to SANA registries would require 2 new prototypes Need ask to CMC to identify if anyone will provide prototype resources (Resolution?) Current missions see no reason to expend resources to make this change and may continue using the current version even if BB is updated If no prototype resources are identified, we suggest they be reconfirmed 24 October 2019 MOIMS Fall 2019 - 4

5. DAI WG Executive Summary Resolutions agreed upon this meeting DAI-1: We wish to thank ESA for hosting this meeting DAI-2: Submit CCSDS 650.0-M-3 - Reference Model for an OAIS for final concurrent CCSDS and ISO reviews Limited current use of CAOs and ADIDs (for use w/in SFDU) However extensive amounts of SFDU exist in legacy archives CCSDS 630.0-B-1 - Standard Formatted Data Units Control Authority Procedures CCSDS 631.0-G-2 - Standard Formatted Data Units Control Authority Procedures Tutorial CCSDS 632.0-B-1 - Standard Formatted Data Units Control Authority Data Structure DAI-3: Reconfirm the following (pending from previous fall meeting Control Authority books) DAI-4: This CCSDS Meeting overlapped with an important Space Conference (the IAC in Washington DC) as well as other Digital Preservation conferences. We request that the Secretariat check for conflicts with those specific conferences each time when planning a CCSDS Plenary meeting. Further, we suggest that the Secretariat should send to the CESG-All distribution, the list of conferences currently checked when the Secretariat is advising a host for scheduling a meeting, and solicit additional input for conferences/conflicts that need to be avoided. WGs submitting additional conflict checks should provide a link where the Secretariat can check the schedules for each conflict check . 24 October 2019 MOIMS Fall 2019 - 5

6. DAI WG Executive Summary Further Resolutions anticipated in the next 6 months DAI-5: Publish CCSDS 652.0-M-2 - Audit and Certification of Trustworthy Digital Repositories for concurrent CCSDS and ISO reviews DAI-6: Start a new project for approval of the review process of CCSDS 652.1-M-3 - Guidelines for Bodies Providing Audit and Certification of Candidate Trustworthy Digital Repositories which will lead to concurrent CCSDS and ISO reviews 24 October 2019 MOIMS Fall 2019 - 6

7. DAI Executive Summary Planning: Area and WG name CCSDS Ref Nr Document Title Status / Comments Start and / or Target Publication Date MOIMS DAI 653.0 INFORMATION PREPARATION TO ENABLE LONG TERM USE OK, Progress being made after lagging Added proforma to annex expect new inputs in proforma format shortly Start date 03 April 2014 End date 31 August 2021 OK, Ready for formal review All suggested changes have been addressed Currently addressing DAD comments MOIMS DAI 650.0 Reference Model for an Open Archival Information System (OAIS) - 5 year review Start date 22 August 2016 End date 06 June 2022 MOIMS DAI 652.0 Audit and Certification of Trustworthy Digital Repositories. (ISO 16363) - 5 year review OK, On Schedule Resolved several suggested changes this cycle more SCs will be required to keep consistency with OAIS updates Start date 22 August 2016 End date 30 October 2022 MOIMS DAI 671.0 Data Archive Architecture Description Document OK, Good work this past year developing info model Moving on to interfaces Start date 31 July 2017 End date 25 September 2023 24 October 2019 MOIMS Fall 2019 - 7

8. DAI WG Upcoming New Work Items Resources Needed (total, Editor, Proto 1, Proto 2) [man month] Area and WG name CCSDS Ref Nr Document Title Target Start / Publication Date Comments/Rationale What if not started? TOTAL EDITOR PROTO1 PROTO2 Guidelines for Bodies Providing Audit and Certification of Candidate Trustworthy Digital Repositories MOIMS DAI 652.1 15 November 2019 30 October 2022 2019 1 wm 1 wm 5 year update Document specializes from ISO 17021 for archives Reviews to follow on OAIS update review and coincide with updated Archive Auditing MB NASA (US Industry volunteer) to provide Document Editor 2020 2021 1 wm 0.5 wm 1 wm 0.5 wm 2022 0.5 wm 0.5 wm 24 October 2019 MOIMS Fall 2019 - 8

9. DAI WG Issues for CESG / CMC Incorporating SANA Registry in 5 year update of Control Authority docs BBs so update to refer to SANA registries would require 2 new prototypes We are willing to support update effort if agencies fund prototype efforts Limited and legacy use therefore not appropriate to retire Limited and legacy use therefore we view it as not worth effort for extensive work for update Therefore we recommend reconfirmation as is Action on DAI from CESG: DAI WG to find out and list the Control Authority offices that are active and functioning. DAI WG to consider the fact that these CA registries were used as the source format for the RMP ( due date: by Fall Meeting). Response: A full analysis and answer to the CESG action on DAI has been prepared and emailed to Mario. Outstanding Issues from previous meetings Accessible CCSDS web site statistics and document access statistics 24 October 2019 MOIMS Fall 2019 - 9

10. Backup - DAI WG Documents Reconfirmed Since Last Meeting 610.0-G-5 Space Data Systems Operations with Standard Formatted Data Units: System and Implementation Aspects 620.0-B-2 Standard Formatted Data Units Structure and Construction Rules 621.0-G-1 Standard Formatted Data Units A Tutorial 622.0-B-1 Standard Formatted Data Units Referencing Environment 641.0-B-2 Parameter Value Language Specification (CCSD0006 and CCSD0008) 641.0-G-2 Parameter Value Language A Tutorial 643.0-B-1 ASCII Encoded English (CCSD0002) 647.1-B-1 Data Entity Dictionary Specification Language (DEDSL) Abstract Syntax (CCSD0011) 647.2-B-1 Data Entity Dictionary Specification Language (DEDSL) PVL Syntax (CCSD0012) 647.3-B-1 Data Entity Dictionary Specification Language (DEDSL) XML/DTD Syntax (CCSD0013) 651.2-G-1 Producer-Archive Interface Specification (PAIS) A Tutorial 24 October 2019 MOIMS Fall 2019 - 10

11. BACKUP: Integrated Schedule Panel 2 Years DAI Years DAI WG Integrated Schedule 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 Doc Number Book Type IN DEVELOPMENT Information Preparation to Enable Long Term Use (IPELTU) Reference Model for an Open Archival Information System (OAIS) (Update) 650x0m3 653.0 Magenta 4 8 Magenta v3 6 4 6 IN DEVELOPMENT Audit and Certification of Trustworthy Digital Repositories (Update) 652x0m2 Magenta v2 10 10 Requirements for Bodies Providing Audit and Certification of Candidate Trustworthy Digital Repositories (update) Data Archive Architecture Description Document 652x1m3 Magenta Magenta v3 10 10 671.0 v1 3 9 PLANNED Producer-Archive Interface Protocol Consumer Archive Interface Methodology Abstract Standard Consumer Archive Interface Specification Consumer Archive Interface Protocol Space Mission Data Archive Binding (API?) Audit and Certification of Archival Software Requirements for Bodies Providing Audit and Certification of Archival Software 654.1? 651.4? 672.0? 672.1? 672.2? 671.1? 654.0? Blue Magenta Blue Blue Blue Magenta Magenta 7 7 7 7 7 7 PLANNED 7 7 7 7 7 7 7 7 COMPLETED Space Data Systems Operations with Standard Formatted Data Units: System and Implementation Aspects 610x0g5 620x0b2 621x0g1 622x0b1 630x0b1 631x0g2 632x0b1 641x0b2 641x0g2 643x0b1 644x0b3 645x0g1 646x0g1 647x1b1 647x2b1 647x3b1 647x4o1 650x0m2 651x0m1 651x1b1 651x2g1 652x0m1 652x1m2 661x0b1 Green Blue Green Blue Blue Green Blue Blue Green Blue Blue Green Green Blue Blue Blue Orange Magenta Magenta Blue Green Magenta Magenta Blue v5 v1 2 6 6 6 6 2 v2 5 5 11 Standard Formatted Data Units Structure and Construction Rules Standard Formatted Data Units A Tutorial ISO ISO Standard Formatted Data Units Referencing Environment 5 ISO COMPLETED Standard Formatted Data Units Control Authority Procedures 6 6 11 5 5 5 ISO TIME NOW Standard Formatted Data Units Control Authority Procedures Tutorial. v1 Standard Formatted Data Units Control Authority Data Structures 11 ISO Parameter Value Language Specification (CCSD0006 and CCSD0008). v1 v1 5 5 ISO v2 6 6 6 6 6 ISO Parameter Value Language A Tutorial v2 ASCII Encoded English (CCSD0002) 11 ISO The Data Description Language EAST Specification (CCSD0010) v1 5 5 5 v2 ISO 11 v3 6 6 6 6 6 ISO ISO The Data Description Language EAST A Tutorial The Data Description Language EAST List of Conventions Data Entity Dictionary Specification Language (DEDSL) Abstract Syntax (CCSD0011) 6 6 6 6 6 ISO Data Entity Dictionary Specification Language (DEDSL) PVL Syntax (CCSD0012) ISO Data Entity Dictionary Specification Language (DEDSL) XML/DTD Syntax (CCSD0013) 6 ISO Data Entity Dictionary Specificaton Language (DEDSL) - XML/XSD Syntax 1 1 Reference Model for an Open Archival Information System (OAIS) Producer-Archive Interface Methodology Abstract Standard Producer-Archive Interface Specification (PAIS) Producer Archive Interface Specification Tutorial Audit and Certification of Trustworthy Digital Repositories Requirements for Bodies Providing Audit and Certification of Candidate Trustworthy Digital Repositories v1 v2 6 ISO v3 6 4 6 6 ISO 5 6 6 ISO 2 2 ISO ? ? ? ? 9 9 v2 9 11 ISO 11 v2 10 10 10 6 v1 3 v3 10 ISO XML Formatted Data Unit (XFDU) Structure and Construction Rules 9 9 9 ISO DEPRECATED CCSDS Panel 2 Methodology for Development of Recommendations XML Formatted Data Unit (XFDU) Tutorial 611x0y1 661x1g1 Yellow Green 6 W X 24 October 2019 MOIMS Fall 2019 - 11