Introducing SOLEIL SciCAT Status Update
Explore the latest advancements in the SOLEIL SciCAT system, including upgrades towards SOLEIL II, deployment challenges, and the architecture base with WSO2 API Manager and Apache Kafka. Learn about SciCAT's functionalities for ingestion of proposals and datasets, dataset access, and AAI/IAM implementation with Keycloak based on DUO information. Discover how SciCAT streamlines inter-communication processes and mutualizes resources for a more efficient workflow.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
SOLEIL SciCAT Status @ISPyB / MXCuBe meeting 2024/11/20 Majid OUNSY, Patrick Madela, Gwenaelle Abeille
Context: SOLEIL Upgrade (Towards SOLEIL II) Present Future Legacy: Solutions developed based on evolving needs, different silos Deployment challenges: SciCAT, Sample Tracking, VISA, PUMA, DSW, Objective: Harmonized Application s inter-communication processes, Mutualise Resources PLUSS (Plateforme d Urbanisation du Synchrotron SOLEIL) A Platform for exchangingdata between heterogeneous applications Separationof Concerns: One Application Per Service Integration: Inter-communication (APIs, Event Based NO MORE SILOS
Architecture Base WSO2 API Manager (Open Source) o Synchronos Communcations o WEB API Catalog (REST, GraphQL,..) o Controls Usage (security, quotas,..) Apache KAFKA (Open Source) o Asynchronous o High throughput messaging o Many Existing Connectors o Secure
SciCAT:Ingestion of Proposals Sun Set: SOLEIL DUO o REST API to query project s information o Common Producer to publish DUO Events on a single KAFKA topic o Topic used by various applications: SciCAT and Sample Tracking Application SciCAT Consumer o Ingests Proposal s Metdata from DUO topic o Handles both Simple and BAG proposals
SciCAT:Ingestion of Datasets Ongoing Work Device Evaluation o Modifie Tango Control System Devices to Gather Metadata information directly Kafka producer and consumer o Use of KsqlDB to integrate metadata from DUO topics
SciCAT:Dataset Access Ongoing Work Two options evaluated o Mirroring System: Mirror experimental data storage for requested data o Archive creation: Create an archive of requested data Acess Method o Globus Public Endpoint
SciCAT:AAI IAM with Keycloak o Implementing Keycloak based IAM based on DUO information Evaluation of Autorisation solution o DUO s REST API: Use of Endpoints o Database Integration: Direct Access to a DUO s database view
SOLEIL flash report @ISPyB / MXCuBe meeting 2024/11/20 Majid OUNSY, Tatiana ISABET, Idrissou CHADO
SOLEIL status report Production status: Two versions: ISPyB 5.23 without Keycloak EXI2 with Keycloak Link to MXCube and SAXS measurements pyISPyB status: SOLEIL & MAXIV: Full installation of pyISPyB (frontend & backend) under joint efforts Some issues remain to be solved Aim to share experience on functional & technical requirements MX stuffs migration from ISPyB to pyISPyB is ongoing Forseen : SOLEIL expects to use pyISPyB s CryoEM function on its new CryOEM installation SOLEIL expects to add BioSAXS features into pyISPyB (in the long term) Pied de page 9
Questions ? Pied de page 10
