Automating Cleanup in OpenStack Using Mistral Workflows
Introduction to utilizing Mistral workflows in OpenStack to automate tenant cleanup processes based on instance expirations and policies. Learn about defining expiration policies, running actions at specific times, and leveraging Mistral's workflow engine for efficient resource management.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Using OpenStack to clean up after itself: Automatically run Mistral workflows in response to OpenStack notifications Nina Goradia, IRC: ninag Todd Johnson, IRC: toddjohn
Agenda Background Introduction to Mistral Tenant cleanup Demo Expirations Demo Next steps for Newton Questions
Background Legacy feature in our private Cloud solution: Instance Expirations OpenStack PWG use case: http://specs.openstack.org/openstack/openstack-user-stories/user- stories/draft/lifecycle_management.html (Extract: To address VM sprawl apply a life cycle model. When a tenant launches a VM, they can specify the type of use case (ex development) which will assign a lease length (ex 90 days))
Background Needed a way to do the following: Define expiration policies on a per project basis Run a set of actions at certain times Realized: That an expiration policy is really just meta data on the instance (or any OpenStack resource) with a set of default values that could be modified by the user within a set of constraints And the actions to run is really a workflow that is triggered to run at a certain time in the future as long as the instance is still active.
Background OpenStack to the rescue: Glance provides a meta data definition Mistral provides workflow support
Introduction to Mistral Workflow engine for OpenStack User creates workflows in YAML with YAQL expressions. The YAML DSL is documented in the OpenStack docs Documentation: http://docs.openstack.org/developer/mistral/
Mistral Workflows Contains 1-n tasks Runs: Actions: Standard set: send email, ssh, http Common OpenStack clients (keystone, glance, nova, cinder, heat, neutron) Custom (plugin point for any bit of python code) Workflows Publish data for other tasks Sequence follow-on tasks based on success, failure, completion Can run in parallel 2 Types: Direct and Reverse Direct workflows run tasks in order Reverse workflows run a task graph calculated on task dependencies Either be private to a project or public Can be run immediately or at a specific time (cron trigger)
Mistral Missing features: Mistral action pack Glance meta data definitions action pack Notification trigger
Tenant clean-up Delete orphaned resources when a tenant is deleted Keystone emits identity.project.deleted event when a project is deleted: Notification contains the project_id of the deleted project Have asked for the project name also for usability purposes Register the project_deleted_wf workflow to run on the identity.project.deleted event, using the proposed event trigger support
Tenant clean-up Mistral runs the project deleted workflow: Using the new mistral action pack, get a list of all the workflows with a name that starts with project_cleanup in the admin project. Run the project_cleanup workflows (passing in the deleted project_id) Delete nova servers Delete project private neutron port groups and networks Delete glance images Delete cinder volumes Gather the list of deleted resources and send the admin an email
Tenant clean-up Demo
Expirations Expire instances after a given amount of time. Send user email notifications When the instance is about to expire (warning) When instance expired For the purpose of the prototype expire will mean the instance is stopped. However, the mechanism should support any set of actions on the instance when it expires or prior to expiration. Policy based: Project level, configurable Naturally extensible to other OpenStack resources, such as volumes
Expirations Register the setup_expiration_wf workflow to run on the compute.instance.create.end event, using the proposed event trigger support Mistral runs the workflow when an instance is created: Checks the event payload for expiration meta data Gets expiration policy meta data from Glance meta data definitions Based on the above, schedules cron-triggers to: Send warning emails Expire the instance (includes sending instance expired email)
Expirations Register the delete_expiration_wf workflow to run on the compute.instance.delete.end event Mistral runs the workflow when an instance is deleted: Checks to see if there are cron-triggers set up for the instance being deleted, and if so, deletes the cron-triggers
Expirations Demo
Next steps for Newton Validate PoC, welcome additional input and contributions Notification trigger support in Mistral Blueprint: https://blueprints.launchpad.net/mistral/+spec/event-notification- trigger Spec: https://review.openstack.org/#/c/308664/ Mistral action pack for Glance Meta Data Definitions support Publish Mistral workflows in the OpenStack Application Catalog Blueprint: https://blueprints.launchpad.net/app-catalog/+spec/add-mistral- assets
IBM Sponsored Track 11:15 Kick-off Event Jesse Proudman OpenStack for Beginners Brad Topol Shamail Tahir Tyler Britten The open cloud: A platform of possibilities Local, Dedicated and Public use cases Jason McGee Azmir Mohamed Modelling, Deploying and Managing Applications in IBM Blue Box with Cloudsoft AMP Duncan Johnston-Watt (Cloudsoft) Hernan Alvarez Open without Limits -- LinuxONE, Ubuntu 16.04 and OpenStack Mark Shuttleworth (Canonical) Angel Diaz Jessica Murillo Kershaw Mehta One network to rule them all: Open, scalable, & integrated networking for Containers & VMs Kyle Mestery Phil Estes Networking Time-out with Kyle Mestery and Team Skip the coffee: iPad Air 2 raffle, Texan specialty beers, sodas, cool posters Accelerating Mobile App delivery with IBM UrbanCode Deploy, Heat, and OpenStack Tim Pouyer Tyson Lawrie Glen Hickman April 26th, Austin Convention Center, Level 4, MR 18 C/D 11:20 12:05 2:00 2:50 3:40 4:20 4:40
