Ocean Observatories Initiative Data Management Review
Operating and maintaining the Ocean Observatories Initiative involves executing processes, procedures, and work instructions to meet operational requirements and deliver data products. The anticipated O&M organization structure aims at sustaining end-to-end data delivery. Assessment and improvement of OOI data delivery focus on technical feasibility, scheduling approved work, and integrating external science recommendations into the annual work plan subject to budgetary approval by NSF.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Ocean Observatories Initiative Data Management Organization NSF Data Management Review April 28, 2014
Operating and Maintaining the OOI The OOI team will execute processes, procedures, work instructions to meet operational requirements and deliver data products. Activities include: Marine hardware and instrumentation System status monitoring, reporting and corrective actions Pre-deploy (test, integration, calibrate), Deploy (platform ops, command/control) Vehicle ops, alarms/alerts, sampling strategies, data QC Recovery, post-recovery (refurb, calibrate), data recovery Performance monitoring and reporting, Asset management, Incident Reporting Data delivery (system services and products) Cybersecurity, System monitoring, status display, software maintenance Performance measurement, network and application enhancement Hardware/Software Development, Maintenance, Quality assessment Maintenance and improvement of data products, algorithms, metadata Data QA/QC enhancements, user request and support NSF Data Management Review April 28, 2014 1
Anticipated O&M Organization Structure Sustaining end-to-end data delivery Ocean Leadership UNOLS OOSC Observatory Director Safety Assurance Marine Assurance Award Administrator WHOI UCSD OSU UW Global and Pioneer Cyberservices Endurance Regional /Cabled Assets Subaward SIO Global NSF Data Management Review April 28, 2014 2
Assessment and improvement of OOI data delivery Observatory User groups UNOLS OOSC OOI Operations sustaining data delivery Technical feasibility Technical refresh Scheduling of approved work External science recommendations data QC prioritization of sensor improvements The OOI team sustains the approved configuration Will be integrated into the Annual Work Plan Subject to budgetary approval by NSF NSF Data Management Review April 28, 2014 3
Operational structure: support external science and data delivery Ocean Observing Science Committee (OOSC) of UNOLS fills external advisory role. Observatory User Groups will possess expertise to inform the advisory committee. The cross-observatory science team (small in 2015) will be the internal resource for User Groups and external advisors. Size of science and data QC team evaluated annually. NSF Data Management Review April 28, 2014 4
Organizational and Functional Management Ocean Leadership Leverage individual IOs to manage functional areas Monitoring Engineering Deployment Refurbishment Data Mgmt OOI-wide Data Management Team CI CG EA RSN EPE NSF Data Management Review April 28, 2014 5
Organizational and Functional Management Ocean Leadership Leverage individual IOs to manage functional areas Monitoring Engineering Deployment Refurbishment OOI-wide Data Management Team CI CG EA RSN EPE NSF Data Management Review April 28, 2014 6
Data Management Ocean Leadership Observatory Director OOI-wide Data Management Team Data Management Lead (TBD) 1 FTE Will direct the activities of representatives from each Marine Organization 2015 3 FTE 2016 3 FTE 2017 TBD following internal and external assessment Concept of Operations establishing efficiencies in non-data areas, resulting in opportunities to increase staffing within data management and data QC NSF Data Management Review April 28, 2014 7
Objectives for data processes in O&M Sensor data will be validated through pre- and post-deployment calibrations, in-water validations when possible, and tracking of sensor engineering data during deployment. Full metadata description will be provided for each unit. After commissioning of elements of the OOI arrays, the data products will have quality control annotations ( flags ) based on community standard error-detection algorithms. The need for additional post-processing data QC will be assessed on an annual basis, within the constraints of the OOI operational budget. All the data QC steps will be documented and accessible to users. NSF Data Management Review April 28, 2014 8
Maintaining Data Integrity Two aspects to Data Integrity (as reviewed in previous presentations) Management Observatory configuration (asset tracking) Metadata Subset of asset tracking Calibration tables Quality QC, identifying defects of products Automated QC flags, manual review of data QA, identifying defects in process NSF Data Management Review April 28, 2014 9
Pre-Deployment Procedures 1. Incoming Inspection Completed for all Instruments and Platforms Verifies configuration and state as delivered 2. Calibration Records Records for each instrument or platform are archived in Vault 3. Quality Conformance Tests (QCT) Completed for all Instruments and Platforms Confirms basic functionality ( bench test ), detects failures or damage 4. Requirements Verification Completed for each instrument type or Class Validate first article against requirements and specifications 5. Platform Integration and Test Platform operation verified using platform controller End-to-End communication verified, instrument to shore station NSF Data Management Review April 28, 2014 10
Maintenance of data delivery in O&M Annual Review of: Data delivery % of sensors providing data (weekly compilation of status) Timeline of QC stats per sensor (monthly compilation) Identification (weekly) and annotation (monthly?) of data issues Data Product List Data Production Algorithms QA/QC Algorithms Data Calibration/Validation Procedures Sampling Strategy Plan Community feedback about data products, QC Propose improvements, implement through annual O&M work plan NSF Data Management Review April 28, 2014 11
Data processes in O&M Objectives for Data QC after acquisition Compilation of error statistics for each data product Qualitative examination of the data records by project personnel Aggregate performance metrics for each type of sensor and set of data products Quarterly report of statistics provided to Observatory Director, then to External Advisory Committee and Observatory User Groups consisting of an observatory performance evaluation and recommendations for improvements feedback will be solicited from the user community via UNOLS OOSC - to supplement the assessments from the OOI O&M team. NSF Data Management Review April 28, 2014 12
Data processes in O&M Objectives for Data QC after acquisition Annual data QC meetings will be held with NSF, the Observatory Director, the Cross-observatory Science Team and the OOSC Establish clear recommendations for data QC improvements Propose to implement within Work Plan (and budget) for the next period of performance. NSF Data Management Review April 28, 2014 13
NSF Data Management Review April 28, 2014 14
Ocean Observatories Initiative COL Plan for work-to-go for Data Management NSF Data Management Review April 28, 2014
Deployed Scope of OOI (over 750 instruments distributed over all moorings, benthic packages, seafloor nodes, gliders and AUVs) Global Arrays Coastal Arrays Subsystems Global Arrays Station Papa Components Instruments Service Frequency Subsystems Coastal Arrays Pioneer Components Instruments Service Frequency 3 Surface Moorings 2 Surface-Piercing Profilers Moorings 5 Profiler Moorings 3 AUVs 6 Gliders 3 Surface Moorings 2 Surface-Piercing Profilers Moorings 1 Hybrid Profiler Mooring 1 Benthic Experiment Package 1 Multi-Function Nodes 60 18 29 18 30 50 18 16 10 8 68 18 5 30 Twice a year 1 Subsurface Hybrid Profiler Mooring 2 Flanking Moorings 3 Gliders 1 Surface Mooring 1 Subsurface Hybrid Profiler Mooring 2 Flanking Moorings 3 Gliders 1 Surface Mooring 1 Subsurface Hybrid Profiler 2 Flanking Moorings 3 Gliders 1 Surface Mooring 1 Subsurface Hybrid Profiler 2 Flanking Moorings 3 Gliders 12 32 9 23 12 32 9 23 12 32 9 23 12 32 9 Yearly Irminger Sea Yearly Endurance (Oregon Line) Twice a year Southern Ocean Yearly Endurance (Washington Line) 3 Surface Moorings Twice a year 2 Surface-Piercing Profilers Moorings 1 Profiler Mooring 6 Gliders Argentine Basin Yearly Cabled Arrays Cyberinfrastructure Subsystems Regional Scale Nodes Hydrate Ridge Components Instruments Service Frequency Computing platforms, software applications, storage, and high speed network equipment Seafloor: Primary and Secondary Profiler Winched Profiler Wire crawler Midwater Platform@ 200m Bottom Instrument Package Seafloor: Primary and Secondary Profiler Winched Profiler Wire crawler Midwater Platform @ 200m Bottom Instrument Package 16 10 5 8 6 26 10 5 8 6 Yearly Cyber Points of Presence (CyberPoPs) Acquisition Points Distribution Points Integrated Observatory Network OOI Net Hardware / Software Axial Seamount Yearly Redundant computing environment Connected by 880km of seafloor cable, with 10KW power, internet connectivity between 7 primary nodes, multiple secondary nodes, and all distributed instrumentation Extensive details about each component can be found on the OOI website (http://oceanobservatories.org) NSF Data Management Review April 28, 2014 16
OOI Installation Schedule 2011 OOI Installation Schedule J A S O N D 2012 2013 2014 Q2 A M J 2015 Q1 J F M Q3 J A S Q4 O N D Q1 J F M Q2 A M J Q3 J A S Q4 O N D Q1 J F M Q2 A M J Q3 Q4 Q1 J F M Q2 A M J Q3 J A S Q4 O N D Cyberinfrastructure R2 R3 R1 C Education R4 R2 R3 Argentine Basin I C G Irminger Sea Southern Ocean55S Global Arrays C I G I C G Station Papa C I G Phase II I Phase III Phase I Endurance C C I Coastal Arrays C G A I I Pioneer I C C C G G Regional Arrays I C I I Submarine Cable Installed Primary Nodes Moorings & Seafloor Sensors NSF Data Management Review April 28, 2014 L e g e n d AUVs Deployed Software Release # Gliders Deployed A I C Commissioning 17 R# Data Flow Installation G 2014-02-21_ver_5-17 This figure is a simplified graphic representation of the schedule for installation of OOI infrastructure arrays and software releases. The placement of icons represent when, by calendar year and quarter, the basic types of infrastructure (e.g., submarine cable, moored array, glider, software release) are initially installed, when data becomes available, and when the infrastructure element/array is commissioned. Commissioning is the milestone event signifying the full operational readiness of the array, major array element, and software release. For details of what will be deployed during each installation event, see the OOI Installation Schedule Table. Note: This graphic does not, and is not intended to, portray a ship cruise schedule. In some cases, multiple installation events may be combined on a single installation cruise.
Lifecycle Maintenance Global Arrays on a 12-month maintenance cycle, fixed and mobile assets (recover, redeploy replacement) Coastal Arrays (most elements) on a 6-month maintenance cycle (recover, redeploy replacement) Cabled assets on a 12-month maintenance cycle (not all assets recovered annually) Coastal gliders will be rotated every 2-3 months initially These requirements lead to a steady rhythm during operations NSF Data Management Review April 28, 2014 18
OOINet Release Plan, 2014 February March April May June July August Driver & Algorithm Development MIP Steps 1 - 4 Regional Scale Node (29 drivers) 31 30 Dates subject to revision when performance metrics become available Station Papa (13 drivers) 25 Coastal Gliders (6 drivers) 25 Pioneer (9 drivers) 18 14 Integration, Deployment, and Release MIP Step 5 14 Release 2.3 2 Release 2.2 (3 Drvrs) 31 8 Release 2.2.1 (9 Drvrs) 31 25 Deployment Cruise Dates Station PAPA Endurance WFP 10 16 Pioneer WFP RSN 11 13 Coastal Gliders 1 OOINET Release 3 Scope to Complete Release 3.0 (Beta 2) STC (beta) UX Screens Marine Int. Release 3.0 (Final) Defect Corrections PRR Build Release 3.0 (Beta1) STC (alpha) UX Screens Marine Int. Release 3.0 (Beta 3) STC (beta) UX Screens Marine Int. PRR July PRR 16 27 25 31 17 21 NSF Data Management Review April 28, 2014 19
OOI Data flow uncabled cabled NSF Data Management Review April 28, 2014 20
NSF Data Management Review April 28, 2014 21
