EGI Federated Cloud Infrastructure Overview

The EGI Federated Cloud is a collaborative platform for research and education, offering cloud federations with various levels of interoperability. It provides a hybrid cloud federation where multiple cloud providers pool their resources and data, following common policies for federated service management. Users can access federated services in a modular way while leveraging local tools and interfaces. The platform includes community collaborations, open standards, and mandatory compliance with EGI services such as Service Registry, AAI, Accounting, and Monitoring.

• EGI Federated Cloud
• Research
• Education
• Hybrid Cloud Federation
• Service Management

melb_nhi Follow

Uploaded on Sep 26, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. The EGI Federated Cloud e-Infrastructure Enol Fern ndez EGI Cloud Technologist www.egi.eu EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number 654142

2. EGI Federated Cloud EGI Federated Cloud is a collaboration of communities developing, innovating, operating and using cloud federations for research and education. The EGI Cloud is a open hybrid cloud federation Different levels of federation are possible offering various degrees of interoperability All cloud providers pooling resources and data in a multi tenant model Need to adhere to a set of common policies, procedures and processes for federated service management Can join in by adopting the federation model that provides the interoperability needs of the target groups Can choose among federated services offered by EGI in a modular way, while still relying on local tools and exposing own service interfaces to the federation, as applicable 9/26/2024 2

3. EGI Federated Cloud Community Platform Community Platform Collaboration Platform EGI endorsed images VM image catalogue Open Standards Cloud Realm OCCI/CDMI Federated VM Image management OpenStack Cloud Realm OpenStackAPIs Federated VM image management Cloud Realm EGI Core Infrastructure Platform AAI, Service Registry, Accounting, Monitoring, Federated Service Management Physical Infrastructure 9/26/2024 3

4. Cloud Realms Service Cloud Realm Integration EGI Federated Service Management mandatory EGI Service Registry mandatory EGI AAI compliance mandatory EGI Accounting mandatory, depends on EGI AAI compliance EGI Monitoring Monitoring is a passive activity, i.e. no active integration from the side of provider is necessary. Monitoring of IaaS interfaces (OCCI, OpenStack, CDMI) requires EGI AAI compliance EGI Information Discovery optional VMI replication optional VMI catalogue optional IaaS standard interface compliance optional, all providers within a realm must provide interoperable interfaces EGI helpdesk optional 9/26/2024 4

5. Federation Services Service Registry & Information Discovery GOCDB service catalogue with static information BDII provides real-time view of the actual capabilities of the federation OGF GlueSchema 2 AAI IGTF Federation with X509 certificates + VOMS extensions Credential translation for other IdP federations Accounting Collect, aggregate and display usage information across the whole federation. OGF Usage Record extended for Cloud Availability Monitoring Health monitoring of services A/R metrics for SLA/OLAs EGI ARGO 9/26/2024 5

6. VM Image Catalogue and Management VM Image Catalogue VM Image Management Open Library of Virtual Appliances Use on clouds or for personal download Re-use, share, associate contextualization EGI endorsed set of VM images prepared to run in a cloud in a secure manner Community curated sets of images Automatic and secure distribution of sets to cloud providers HEPiX lists for distribution 9/26/2024 6

7. Current EGI Federated Cloud Realms Open Standards Realm OpenStack Realm Service GOCDB EGI Service Registry X.509 proxy certificates with VOMS extensions EGI AAI compliance Cloud Usage Records, APEL repository and portal EGI Accounting EGI ARGO monitoring EGI Monitoring BDII EGI Information Discovery HEPiX image lists (vmcatcher) VMI replication AppDB Cloud Marketplace VMI catalogue OCCI & CDMI APIs IaaS interface OpenStack API GGUS with dedicated SUs EGI helpdesk 9/26/2024 7

8. EGI Federated Cloud Architecture IaaS Uniform interfaces and behaviour Capabilities Share & endorse VM images Cloud Management Framework Replicate VM images EGI Cloud Service Providers Federation Integration VM Image Catalogue Operation-Level Federator Services EGI AppDB EGI Core Platform 9/26/2024 8

9. Open Standards Realm IaaS OCCI CDMI Capabilities Share & endorse VM images OpenStack OpenNebula Synnefo Replicate VM images EGI Cloud Service Providers Federation Integration VM Image Catalogue Information discovery Service Registry AAI Monitoring Accounting EGI AppDB EGI Core Platform 9/26/2024 9

10. Federated Cloud infrastructure Production operations since May 2014 Currently: 21 providers 6.400 cores in total 343K Virtual Machines in last 12 months 9/26/2024 10

11. EGI Federated Cloud Task Force Collaboration board that gathers user communities, resource centers and technology providers together The Task Force: Defines the technical roadmap of the federation Identify, integrate and develop open source tools and services that enable cloud federations for research and education Provides training and support for building and using federated clouds Meetings every two weeks on Tuesday at 11.00 (Amsterdam time) 9/26/2024 11

12. OpenStack Integration 9/26/2024 12

13. OpenStack Integration (I) OpenStack components OpenStack resource provider WSGI add-ons Keystone EGI integration tools Get access token Keystone- VOMS VOMS x509 proxy WSGI application Apache+SSL+ mod_wsgi Publish accounting records EGI IaaS Operations Extract usage information cASO + SSM ceilometer accounting repository Publish endpoint information cloud-bdii- provider Site BDII Native OS clients Extract site information nova-api Top BDII OCCI clients ooi Monitor site Register/update images Subscribe to image lists EGI vmcatcher + hooks AppDB (vmcaster) glance monitoring 9/26/2024 13

14. OpenStack Integration (II) EGI AAI Keystone-VOMS plugin for Keystone V2 API Requires modification of the Keystone WSGI pipeline Manages federation users OCCI interface ooi uses public nova-api interface in a WSGI pipeline Can be installed along an existing nova-api endpoint or as a separate WSGI application EGI Accounting, Information Discovery, and VMI replication: Software components using the public OpenStack interfaces 9/26/2024 14

15. OpenStack FedCloud Appliance A single VM with all the components using the public OpenStack interfaces Accounting, Information discovery, VMI replication Packaged as Docker containers, available at docker hub https://hub.docker.com/u/egifedcloud/ Documentation: https://wiki.egi.eu/wiki/MAN10#Integration_with_EGI_F edCloud_Appliance Appliance at AppDB: https://appdb.egi.eu/store/vappliance/fedcloud.integra tion.appliance.openstack 9/26/2024 15

16. OpenStack Integration with Appliance OpenStack components OpenStack resource provider WSGI add-ons Keystone EGI integration tools Get access token Keystone- VOMS VOMS x509 proxy WSGI application Apache+SSL+ mod_wsgi FedCloud Appliance Publish accounting records EGI IaaS Operations Extract usage information cASO + SSM ceilometer accounting repository Publish endpoint information cloud-bdii- provider Site BDII Native OS clients Extract site information nova-api Top BDII OCCI clients ooi Monitor site Register/update images Subscribe to image lists EGI vmcatcher + hooks AppDB (vmcaster) glance monitoring 9/26/2024 16

17. References Federated Cloud at EGI wiki: https://wiki.egi.eu/wiki/EGI_Federated_Cloud Installation manual: https://wiki.egi.eu/wiki/MAN10 Resource Center certification: https://wiki.egi.eu/wiki/PROC09 AppDB https://appdb.egi.eu/ GOCDB http://goc.egi.eu/ Monitoring http://argo.egi.eu/ Accounting http://accounting.egi.eu/cloud.php 9/26/2024 17

18. Thank you for your attention. Questions? www.egi.eu This work by Parties of the EGI-Engage Consortium is licensed under a Creative Commons Attribution 4.0 International License Creative Commons Attribution 4.0 International License.