Data Quality Protocols and Procedures for Instrument Calibration and Sampling Rate Management
Data Quality
Ed Chapman
OOI Chief Systems Engineer
Goal
Address Areas
for
Recommendations:
#2 “Data Policy, Data Quality Protocols and
Procedures” and #4 “Data sampling rate strategy
development and management ”
Specific topics:
“Shoreside & at-sea instrument and subsystem
quality/calibration procedures/protocols,
automated thresholds/flags, manual data QC,
exception management, and long term time-series
data sampling rate management.”
Shoreside & at-sea instrument and
subsystem quality/calibration
procedures/protocols
Pre-Deployment Procedures
1. Incoming Inspection
•
Completed for all Instruments and Platforms
•
Verifies configuration and state as delivered
2. Calibration Records
•
Vendor records for each instrument or platform
3. Quality Conformance Tests (QCT)
•
Completed for all Instruments and Platforms
•
Confirms basic functionality (“bench test”), detects failures or damage
4. Instrument Requirements Verification
•
Completed for each instrument model
•
Evaluate first article against Requirements and specifications
5. Platform Integration and Test
•
Platform operation verified using platform controller
•
End-to-End communication verified, instrument to shore station
•
Platform Requirements Verification
At-Sea Procedures: Pioneer-1
Platform monitoring
•
Full platform function available when within WiFi range
•
Communication with shore station when out of range
Shipboard underway sampling
•
Meteorology time series from Knorr bow mast
•
Thermosalinograph time series from Knorr system
•
Bathymetry from echoshounder and multi-beam
Shipboard CTD profiles
•
Post-deployment casts at each of 3 sites for Pioneer-1
•
Seabird 9-11 with DO, Fluoro, beam x-miss, turb, PAR
Physical Samples
•
Post-deployment casts at each of 3 sites for Pioneer-1
•
Salinity and Oxygen completed onboard
•
Nitrate/Nitrate, Chlorophyll and Carbon system done in shore labs
Automated QC Thresholds and Flags
L1b and L2b
Instrument
Driver and Agent
Data Product
Algorithm
Calibration Table
User
QC Flags
POLYVAL
Algorithm
Secondary Post-Deployment
calibration values
POLYVAL
Algorithm
Secondary Post-Recovery
calibration values
QC algorithms
(range, spike, stuck, gradient, trend, combined)
Lookup Tables
Permanent storage
Interpolation
Automated QC Checks
•
Seven QC Checks
–
Global Range Test
–
Local Range Test
–
Spike Test
–
Stuck Value Test
–
Trend Test
–
Gradient Test
–
Combined QC Flags
When?
•
QC Checks are run on a periodic basis
–
when data is ingested from the uncabled instruments
–
Continuously from the cabled instruments
•
QC Flags are
stored
.
Automated QC actions
–
PSs create Look Up Tables and values are uploaded
through the UI as csv files
Automated QC Updates
•
If new values are uploaded for any of the QC
Flags those values overwrite the original values.
•
OOINet reruns the QC check for
all
data products
and creates and stores new QC Flags
•
QC is “value added” so we don’t retain prior flags
Human in the loop QC
L1c and L2c
Instrument
Driver and Agent
Data Product
Algorithm
Calibration Table
User
POLYVAL
Algorithm
Secondary Post-Deployment
calibration values
POLYVAL
Algorithm
Secondary Post-Recovery
calibration values
QC algorithms
(range, spike, stuck, gradient, trend, combined)
Lookup Tables
Permanent storage
Interpolation
L1c
Human in the loop
Human in the Loop QC Actions
•
PS periodically downloads an L1 or L2 product
•
PS performs HITL QC locally on PS machine
•
PS uploads L1c or L2c values, and HITL metadata
(provenance, etc.) into OOINet
•
User who downloads L1 or L2 product to which
HITL QC has been applied will see L1c or L2c
variables in the downloaded time series
–
Only for the time range for which the HITL QC was
applied
Human in the Loop QC Updates
•
If new HITL values are uploaded for
a time period
that has already been uploaded
those values
overwrite the original values.
Relationship of QC level a, b, and c
Database
L1 Data
Product
Algorithm
L2 Data
Product
Algorithm
GUI
User
Primary
Calibration
Function
Secondary
Calibration
Functions
QC
Algorithms
QC
Algorithms
Human In
The Loop
Human In
The Loop
L0
L0
L0
L1a
L1b and QC flags
L1c
L2c
L2b
L2b
L1b
L1a
QC flags
exception management
Long term time-series data
sampling rate management
•
PSs and external scientists established
pivotal
and
default
sampling rates
•
Pivotal
is the minimum necessary sampling to answer
long term science questions
•
Default
is the standard sampling rate
•
Rates guided by power and energy budgets for each
platform type.
Pivotal and Default Sampling Rates
•
Annual external reviews to recommend
default
rate
changes
•
Annual iteration between OOI engineers and external
science advisors (via UNOLS committee) to assure that
recommended adjustments are feasible
Annual Review of
Default
Rate
•
Any recommended change in sampling rate by external proposers
will be evaluated with respect to the required maintenance of
'pivotal' rates and power and energy budgets (this becomes part
of the annual interaction between OOI and the UNOLS advisory
structure)
•
O&M team will sustain the
default
sampling rates.
•
Changes in sampling rates only occur in response to:
–
safety threats to life or property
–
technical issues that compromise performance
–
pre-approved responses to defined events (volcanic eruptions,
hurricanes…)
–
approved retasking of observatory elements (NSF proposal
process, UNOLS scheduling)
Adjustment to sampling rates
Questions?
Specific topics:
“Shoreside & at-sea instrument and subsystem
quality/calibration procedures/protocols, automated
thresholds/flags, manual data QC, exception management,
and long term time-series data sampling rate management.”
This document outlines the data quality protocols and procedures recommended for instrument calibration, automated quality control thresholds, and flagging, as well as sampling rate strategy development and management for oceanographic instruments both shoreside and at-sea. It covers pre-deployment procedures, at-sea monitoring protocols, and post-deployment calibration and sampling activities. The focus is on ensuring data accuracy, reliability, and long-term time-series data quality for research purposes.
Uploaded on Oct 07, 2024 | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Data Quality Ed Chapman OOI Chief Systems Engineer 10/7/2024 1 1
Goal Address Areas for Recommendations: #2 Data Policy, Data Quality Protocols and Procedures and #4 Data sampling rate strategy development and management Specific topics: Shoreside & at-sea instrument and subsystem quality/calibration procedures/protocols, automated thresholds/flags, manual data QC, exception management, and long term time-series data sampling rate management. 10/7/2024 2 2
Shoreside & at-sea instrument and subsystem quality/calibration procedures/protocols 10/7/2024 3 3
Pre-Deployment Procedures 1. Incoming Inspection Completed for all Instruments and Platforms Verifies configuration and state as delivered 2. Calibration Records Vendor records for each instrument or platform 3. Quality Conformance Tests (QCT) Completed for all Instruments and Platforms Confirms basic functionality ( bench test ), detects failures or damage 4. Instrument Requirements Verification Completed for each instrument model Evaluate first article against Requirements and specifications 5. Platform Integration and Test Platform operation verified using platform controller End-to-End communication verified, instrument to shore station Platform Requirements Verification 10/7/2024 4 4
At-Sea Procedures: Pioneer-1 Platform monitoring Full platform function available when within WiFi range Communication with shore station when out of range Shipboard underway sampling Meteorology time series from Knorr bow mast Thermosalinograph time series from Knorr system Bathymetry from echoshounder and multi-beam Shipboard CTD profiles Post-deployment casts at each of 3 sites for Pioneer-1 Seabird 9-11 with DO, Fluoro, beam x-miss, turb, PAR Physical Samples Post-deployment casts at each of 3 sites for Pioneer-1 Salinity and Oxygen completed onboard Nitrate/Nitrate, Chlorophyll and Carbon system done in shore labs 10/7/2024 5 5
Automated QC Thresholds and Flags L1b and L2b 10/7/2024 6 6
Permanent storage Instrument Driver and Agent Data Product Algorithm Calibration Table Secondary Post-Deployment calibration values POLYVAL Algorithm User Secondary Post-Recovery calibration values POLYVAL Algorithm QC Flags Interpolation QC algorithms Lookup Tables (range, spike, stuck, gradient, trend, combined)
Automated QC Checks Seven QC Checks Global Range Test Local Range Test Spike Test Stuck Value Test Trend Test Gradient Test Combined QC Flags 10/7/2024 8 8
When? QC Checks are run on a periodic basis when data is ingested from the uncabled instruments Continuously from the cabled instruments QC Flags are stored. 10/7/2024 9 9
Automated QC actions PSs create Look Up Tables and values are uploaded through the UI as csv files 10/7/2024 10 10
Automated QC Updates If new values are uploaded for any of the QC Flags those values overwrite the original values. OOINet reruns the QC check for all data products and creates and stores new QC Flags QC is value added so we don t retain prior flags 10/7/2024 11 11
Human in the loop QC L1c and L2c 10/7/2024 12 12
Permanent storage Instrument Driver and Agent Data Product Algorithm Calibration Table Secondary Post-Deployment calibration values POLYVAL Algorithm User Secondary Post-Recovery calibration values POLYVAL Algorithm Interpolation QC algorithms Lookup Tables (range, spike, stuck, gradient, trend, combined) L1c Human in the loop
Human in the Loop QC Actions PS periodically downloads an L1 or L2 product PS performs HITL QC locally on PS machine PS uploads L1c or L2c values, and HITL metadata (provenance, etc.) into OOINet User who downloads L1 or L2 product to which HITL QC has been applied will see L1c or L2c variables in the downloaded time series Only for the time range for which the HITL QC was applied 10/7/2024 14 14
Human in the Loop QC Updates If new HITL values are uploaded for a time period that has already been uploaded those values overwrite the original values. 10/7/2024 15 15
Relationship of QC level a, b, and c 10/7/2024 16 16
Database L0 L0 L1 Data Product Algorithm L2 Data Product Algorithm Primary Calibration Function L1a L2b Secondary Calibration Functions L1b QC QC Algorithms Algorithms Human In The Loop Human In The Loop L1a L1b and QC flags L1c L0 L2c QC flags L2b GUI User
exception management 10/7/2024 18 18
Long term time-series data sampling rate management 10/7/2024 19 19
Pivotal and Default Sampling Rates PSs and external scientists established pivotal and default sampling rates Pivotal is the minimum necessary sampling to answer long term science questions Default is the standard sampling rate Rates guided by power and energy budgets for each platform type. 10/7/2024 20 20
Annual Review of Default Rate Annual external reviews to recommend default rate changes Annual iteration between OOI engineers and external science advisors (via UNOLS committee) to assure that recommended adjustments are feasible 10/7/2024 21 21
Adjustment to sampling rates Any recommended change in sampling rate by external proposers will be evaluated with respect to the required maintenance of 'pivotal' rates and power and energy budgets (this becomes part of the annual interaction between OOI and the UNOLS advisory structure) O&M team will sustain the default sampling rates. Changes in sampling rates only occur in response to: safety threats to life or property technical issues that compromise performance pre-approved responses to defined events (volcanic eruptions, hurricanes ) approved retasking of observatory elements (NSF proposal process, UNOLS scheduling) 10/7/2024 22 22
Questions? Specific topics: Shoreside & at-sea instrument and subsystem quality/calibration procedures/protocols, automated thresholds/flags, manual data QC, exception management, and long term time-series data sampling rate management. 10/7/2024 23 23
