Stochastic Coastal Regional Uncertainty Modelling II (SCRUM2) Overview
SCRUM2 project aims to enhance CMEMS through regional/coastal ocean-biogeochemical uncertainty modelling, ensemble consistency verification, probabilistic forecasting, and data assimilation. The research team plans to contribute significant advancements in ensemble techniques and reliability assessment for probabilistic systems. The project aligns with CMEMS priorities for observation, assimilation, and product quality improvements, emphasizing dual focus on physics and biogeochemistry. SCRUM2 builds on previous work by expanding stochastic modelling to ecosystem-sensitive variables and introducing methodologies to ensure ensemble consistency with observational data.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Stochastic Coastal/Regional Uncertainty Modelling II (SCRUM2) Consistency, reliability, probabilistic forecasting & contribution to CMEMS Ensemble Data Assimilation V. Vervatis1, P. De Mey2, S. Sofianos1, N. Ayoub2, M. Kailas1 in close collaboration with colleagues and experts: Stefano Ciavatta (PML), C.E. Testut and Mercator Ocean team 1National and Kapodistrian University of Athens, Greece 2LEGOS, France CMEMS Service Evolution SCRUM2 Kickoff Meeting Teleconference, April 3, 2018
Outline Objectives CMEMS Relevance & Benefits Description of Work o SCRUM Heritage o SCRUM2 Task 1 Task 2 Task 3 Task 4 Research Team Project Timeline & Deliverables Management
Objectives Strengthening CMEMS in the areas of Regional/coastal ocean-biogeochemical uncertainty modelling Ensemble consistency verification Ensemble probabilistic forecasting Ensemble data assimilation
CMEMS Relevance & Benefits R&D priorities outlined in Batch 5 cross-cutting developments on observation, assimilation and product quality improvements Short/mid-term R&D addressed in SES section 4.11 Advanced assimilation for large-dimensional systems (i.e. ensemble techniques, reliability assessment of probabilistic systems) Dual focus on physics and biogeochemistry linked to SES section 4.5 Modelling and data assimilation for marine ecosystems and biogeochemistry Interdisciplinary topics addressed in SES section 4.6 Seamless interactions between CMEMS and coastal systems
Description of Work SCRUM Heritage Consolidate stochastic modelling using up-to-date perturbation and analysis techniques Expand stochastic modelling work to ecosystem-sensitive variables within a coupled ocean-biogeochemical forecasting system Introduce and test two methodologies aimed at checking the consistency of Ensembles with respect to TAC data and arrays Showcase the use of Ensemble-modelled uncertainties in a DA impact exercise Contribute guidance in support of upcoming decisions regarding the evolution of regional/coastal data assimilation schemes in CMEMS Develop appropriate R&D applied in a subgrid of IBI36
Description of Work SCRUM2 SCRUM2 expands the work undertaken in SCRUM, focussing on the following topics: T1: Sensitivity of simulations to uncertainties and generation of Ensembles with account for the age of errors and in response to atmospheric Ensembles (Y1-2) T2: Ensemble consistency, reliability and forecast probability diagnostics (Y1-2) Ensemble consistency analysis using the SCRUM tools Ensemble consistency analysis in presence of correlated observational errors Methods to assess the reliability of Ensembles in probabilistic assimilation systems T3: Practical improvements to the design of Ensemble data assimilation schemes (Y1-2) T4: Towards CMEMS outreach & uptake (Y2) Preparing future Ensemble DA for CMEMS
Description of Work Task 1 Sensitivity of simulations to uncertainties and generation of Ensembles in response to atmospheric Ensembles with account for the age of errors Background perturbation methodologies: o SPPT-AR1 (in SCRUM Wind, Tair, SLP, drag coefs., bio-SMS) o SPUF T/S & SPPT-AR1 bio-parameters (new to this project) (a) (b)
Description of Work Task 1 Production of Ensembles over repeated periods mirroring MFCs operational practices with the aim to augment Ensemble spread give access to aging errors within two weeks forecast lead time o ECMWF-EPS 50 members: multivariate atm. Ensembles large controlled ocean Ensemble spread without DA (triggering large scale response?) o Complementing with stochastic approaches expand SCRUM stochastic parametrizations, unresolved processes/scales o The length of Ensemble time-chunks will be adjusted between 2-weeks and a month Including a short spin-up period of a few days to bring the Ensemble up to empirical consistency with TAC data And a Usable Period (UP) of Ensemble forecasts , spanning the rest of the time chunk, to assess model uncertainties and probabilities over lead times within the UP o Compare different periods of the year with identical ages of errors
Description of Work Task 1 Inflation methods to produce less smooth Ensemble estimates (mainly for DA through NEMO-SDAP coupling) Possible use of prototype coupled ocean-atmosphere Ensembles from ECMWF s forecast model, as an alternate reference for Ensemble consistency checking, and possibly as a way to enrich our base of uncertainties (ECMWF contact person Kristian Mogensen)
Description of Work Task 1 Use of atmospheric Ensembles (new to this project): o Control run with the deterministic ECMWF-HRES (9km) o Perform time-chunks of ocean Ensemble forecasts with the 1stday of the ECMWF-EPS Ensemble reforecasts (18 km), i.e. the first guess closest to the ECMWF-EDA analysis o Using the whole period of the Ensemble reforecasts (not just the 1stday) (a) flow regime: 1stday of reforecast New ECMWF-SP for the years 2018-2020 based on: http://www.ecmwf.int/en/research/special-projects/spgrverv-2016 time (b) flow regime: whole reforecast time
Description of Work Task 1 ECMWF-EPS atmospheric Ensemble for two members rows one-two: 2m air temperature in the global and Bay of Biscay domain and their difference rows three-four: difference for other forcing variables
Description of Work Task 2 Ensemble consistency, reliability and forecast probability diagnostics In this Task, we are interested in the verification of Ensembles and of probabilistic forecasts, typically issued for an interval of a variable or other type of event. More precisely, we are interested in probabilistic forecasts based on Ensemble forecasting. Two categories of diagnostics are concerned: The statistical properties of Ensemble forecasts, and in particular their consistency with observations, e.g. using Rank Histograms: this has been the main focus of the previous SCRUM Service Evolution project; this new proposal expands on SCRUM in several ways (age of errors, correlated observational errors): Tasks 2.1 and 2.2 Probabilistic diagnostics based on Ensemble forecasts converted into probabilistic information. This is Task 2.3.
Description of Work Task 2 Example in SCRUM of Ensemble Consistency Hovm llers along array modes, f(ArM rank) OSTIA L4 ArM spectrum (t) white cell = inconsistent (ArM consistency criterion) Original resolution 1/5 decimation Each ArM rank is associated with a pattern = eigenmode of SRM
Description of Work Task 2 Subtasks: Task 2.1: Ensemble consistency analysis using the SCRUM tools Task 2.2: Ensemble consistency analysis in presence of correlated observational errors Task 2.3: Methods to assess the reliability of Ensembles in probabilistic assimilation systems.
Description of Work Task 2 Task 2.1: Ensemble consistency analysis using the SCRUM tools, over two periods: o Period A, i.e. for SCRUM Ensembles in 2012 (& SCC) o Period B, i.e. for Sentinel-3A data products starting in 2017 Observations CMEMS Product Identifiers Period A SLA SEALEVEL_GLO_SLA_(L3_REP, MAP_L4)_OBSERVATIONS_008_(018, 027) SST SST_(GLO, NWS)_(L3S, L4)_(REP, NRT)_OBSERVATIONS_010_(001, 003, 010, 023) CHL OCEANCOLOUR_GLO_CHL_(L3, L4)_REP_OBSERVATIONS_009_(065, 093) Size Class CHL (SCC) Transferable to OCTAC by the CMEMS-SE CALL1 TOSCA project (PML contact person Stefano Ciavatta) In-situ INSITU_IBI_(TS_REP, NRT)_OBSERVATIONS_013_(033, 040) Period B (Sentinel-3A) SLA SEALEVEL_(GLO, EUR)_PHY_(ASSIM_L3,L3,L4)_(REP, NRT)_OBSERVATIONS_008_(043, 045, 046) CHL OCEANCOLOUR_ATL_CHL_L4_NRT_OBSERVATIONS_009_037
Description of Work Task 2 Task 2.2: Ensemble consistency analysis in presence of correlated observational errors Errors of measurements used in DA are correlated because they include correlated components (e.g. representativity, pre-processing and observation operator errors) o introduce a correlated obs error ArM-CA criterion to check the diagonality of the ICM in the space of array modes o augment scrumcat with an algorithm to diagnose elements of R from the available ensemble information (derived from Desroziers approach)
Description of Work Task 2 Probability forecasts indicate how likely an event is to occur at some forecast range and given current conditions: event of dichotomous nature (e.g. occurrence or non-occurrence of an upwelling, bloom etc) event expressed as a quantity (an index for instance) that can fall into several categories (e.g. cold, normal or warm conditions with respect to a given reference) event defined as a continuous quantity (e.g. maximum surface current larger that a quantity) object-oriented events (e.g. storm surge)
Description of Work Task 2 Task 2.3: Methods to assess the reliability of Ensembles in probabilistic assimilation systems o Define probabilistic events related to specific stake quantities, e.g.-- Freshwater fronts over the shelf in winter, in order to identify hydrological landscapes and monitor habitats, ecosystems and sediment transports Mesoscale activity over the south-easternmost part of the basin controlling SST and Chl-a distribution Navidad events linked to the fall/winter variability of the IPC Ratio abundances of Plankton Functional Types relative to the community structure (S. Ciavatta CMEMS-SE TOSCA project), e.g. in our model Diatoms/ChlTotaland Nano/ChlTotal o Formalize event occurrence criteria o Calculate probabilistic forecasts of these occurrences from Ensembles at various ranges/ages of errors o Develop and demonstrate tools (scores) to verify pf s (as in Murphy, 1973; Candille and Talagrand, 2005) Illustrate with simulated (from Ensemble member) and real data o Test feasibility of statistical forecast calibration rules (as in Toth et al., 2003).
Description of Work Task 2 (continued) o Two important properties to check: Reliability in data space: this is the consistency between prior probabilities (in our case, from the Ensemble) and observed frequencies of the occurrence of events , as a function of range, i.e. lead time (the age of errors , which we can now attain with the new stochastic scheme used in this proposal) from a reference time (i.e. the beginning of the time-chunk UP) Resolution in data space: this is the ability to separate cases when an event occurs or not, as a function of range. o We intend to define and use (variants of) BS and RPS o We will provide graphical representations of the forecast scores, for the meaningful events defined above, as well as some of the classic representations (e.g. Hit rate, False alarm rate, Confusion matrix, Reliability diagram).
Description of Work Task 2 Brier Score (BS) partitioned into Reli, Unc and Reso: ?? = ? (? ? )2+ ????(1 ????) ? (? ????)2 1 ? ?=1 forecast probabilities ? The reference/climatology probability is given by ????= ? ? The term ????(1 ????) is the Unc = uncertainty which depends only on observations The reliability term is null for a reliable system and the resolution term varies from zero for a perfect system to one for a system that is not providing more information than the Unc . ? where ? = ? ? = ??the probability of n observed events ojwithin an interval of Continuous Ranked Probability Score (CRPS): 2 ? ??? ??? ???2?? + ? ? ???? = ? ?? Both scores can be expressed with their skill scores: ???? ???????= 1 ???? ?? ????? = 1 ??? = 1 ??? ?????
Description of Work Task 2 Tool development in SCRUM2 Task 2 The tools will be developed as an expansion of the scrumcat toolbox developed in the previous project Task 2 SCRUMCAT ConsistencyAnalysis Toolbox (includes NEMO & SDAP coupling) Stochastic ensembles input * CMEMS observations input * Basic Ensemble statistics * Rank Histograms * Array-spaceconsistency analysis * OLA file output * updated OLA file output * Ensemble and error inflation * Correlated Obs errors & ArM w/correlated Obs errors * R estimate (EDDA) * BS & CRPS * Task 1 Ensemble Production * SCRUM R&D * SCRUM2 R&D
Description of Work Task 3 Practical improvements to the design of Ensemble data assimilation schemes Linking and coordination with Ensemble work beyond SCRUM2: Facilitate linkages between data assimilation specialists of CMEMS MFCs aimed at developing Ensemble DA tools (e.g. check feasibility of developing a common Off-line analysis explorer OLAX between SDAP and SAM) SDAP MPI domain decomposition in Ensemble DA ( SDAP 2.0 ): o Introducing full MPI domain decomposition in SDAP (the platform already uses MPI among Ensemble members) o Aim: feasibility of high-resolution/high-dimension Ensemble DA o Run on modern platforms with 16+ cores per node and make it compatible with pnetcdf (parallel implementation of NetCDF) and nVidia GPU-based libraries (cublas culapack) SDAP-NEMO interfacing: develop Ensemble DA in NEMO (EnOI and other ensemble methods); EnOI sensitivity experiments will be conducted to assess the impact of different state vectors and/or observations (e.g. ocean physics and bgc).
Description of Work Task 4 Preparing future Ensemble DA for CMEMS Reach out to the various CMEMS components, facilitating uptake and impact Link with other groups (SANGOMA, GOV DA-TT, etc.) Link with the project s external experts.
Research Team Manpower engaged in SCRUM2 Contributor S. Sofianos V. Vervatis M. Kailas P. De Mey N. Ayoub LEGOS postdoc TOTAL Contribution 10% (coordination), permanent personnel = 3 months 100% on SCRUM2, paid on SCRUM2 = 24 months 40% on SCRUM2, Ph.D. = 10 months 30% on SCRUM2, permanent personnel = 8 months 10% on SCRUM2, permanent personnel = 3 months 100% on SCRUM2, paid on SCRUM2 = 18 months 66 months, 2.75 Equivalent Full-Time UoA scientists: S. Sofianos and V. Vervatis, leading the work in Task 1. In addition, M. Kailas will contribute 40% Ph.D. thesis time to this project. LEGOS scientists: P. De Mey will be the responsible scientist in Task 2 and the other involved scientist will be N. Ayoub. In addition, LEGOS will hire a postdoctoral researcher for 18 months (starting Month 7) to carry out much of the Task 2 work (some work in Task 2 will be done by UoA). In Tasks 3 and 4, the LEGOS team will contribute time to link CMEMS with the listed external projects, and to support CMEMS evolution with respect to regional/coastal data assimilation schemes.
Project Timeline Project active: April 2018 March 2020 1styear 2ndyear Generation of Ensembles in response to atmospheric Ensembles with account for the age of errors T1 T2 Ensemble consistency, reliability and forecast probability diagnostics Practical improvements to the design of Ensemble data assimilation schemes T3 Towards CMEMS outreach & uptake (+ 3rdyear) T4
Deliverables due date Partner Item Title Kickoff meeting April-2018 (teleconference) Task-1: development of NEMO to incorporate ECMWF atmospheric Ensembles, in combination with stochastic approaches, following an enhanced MPI strategy for double parallelization Task-1, Task-2.1: diagnostics of Ensemble spin-up and spread as a function of lead time, performing medium-range (S-) sensitivity experiments with atmospheric Ensembles Task-2.1, Task-2.2, Task-2.3: consistency analysis on the new time-chunked Ensembles in presence of correlated errors; probability-based diagnostics Task-1, Task-2, Task-3: consolidated versions of above diagnostics; results incorporating the response to atmospheric Ensembles; link SCRUM2 to ongoing CMEMS-SE and external projects Evaluation mid-term meeting mid-2019 (Mercator Ocean, Toulouse, France) Task-1: production Ensembles of seasonal-range (Ens-) incorporating the response to atmospheric Ensembles and stochastic modelling; additional, sensitivity experiments (S-) Task-2: interim set of consistency and probabilistic diagnostics Task-2: full set of consistency and probabilistic diagnostics including Sentinel data Tasks-1-2-3-4: finalized full set of Ensemble production and all diagnostics integrated in the scrumcat tool, calculated and analyzed Tasks-1-2-3-4:synthesis of project s overall results and future work 06/30/18 UoA D1 09/30/18 UoA D2 12/31/18 LEGOS, UoA D3 03/31/19 LEGOS, UoA D4 06/30/19 UoA D5 09/30/19 12/31/19 LEGOS, UoA LEGOS, UoA D6 D7 03/31/20 LEGOS, UoA D8 03/31/20 UoA, LEGOS D9 Publications in peered review journals and conferences with proceedings acknowledging CMEMS funding 03/31/20 UoA, LEGOS D10 Final meeting early-2020 (Mercator Ocean, Toulouse, France) Two additional meetings during the project: results will be presented to the people in charge of R&D in MFCs, TACs and CIS
Management We still do not have a signed contract The LEGOS post-doc has probably been identified, but it is hard to commit without a contract signature! We started to draft the subcontract between CNRS/DR14 and UoA the DR14 contact is C cile Barat LEGOS prefers to be paid directly by Mercator Ocean Next deadlines?
Stochastic Coastal/Regional Uncertainty Modelling II (SCRUM2) Consistency, reliability, probabilistic forecasting & contribution to CMEMS Ensemble Data Assimilation V. Vervatis1, P. De Mey2, S. Sofianos1, N. Ayoub2, M. Kailas1 in close collaboration with colleagues and experts: Stefano Ciavatta (PML), C.E. Testut and Mercator Ocean team 1National and Kapodistrian University of Athens, Greece 2LEGOS, France CMEMS Service Evolution SCRUM2 Kickoff Meeting Teleconference, April 3, 2018
