Efficient ACME Workflow Automation for Browser-based Team Collaboration
Visualize the streamlined ACME workflow process led by Sterling Baldwin and smoothly executed by the dedicated team. Each step, from running models to analyzing output data, flows seamlessly, enhancing collaboration and efficiency in a browser environment.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Workflow Group: ACME Workbench Sterling Baldwin ACME Workflow in the Browser Team Lead: Sterling Baldwin Sam Fries, Matt Harris, Bibi Raju, Lukasz Lacinski, Jim McEnerney, Jeff Painter, Charles Doutriaux, Sahsa Ames
Project Goals Entire ACME workflow in the browser
Project Goals Entire ACME workflow in the browser Run the model Generate climatologies Run diagnostics Analyze and visualize output data Transfer files between facilities Archive to HPSS Publish to ESGF
Project Goals Each part of the process works independently, and flows into the next step
Project Goals Each part of the process works independently, and flows into the next step Run the model -> Slurm/Moab and the ACME_script.csh
Project Goals Each part of the process works independently, and flows into the next step Run the model -> Slurm/Moab and the ACME_script.csh Generate climatologies -> ncclimo
Project Goals Each part of the process works independently, and flows into the next step Run the model -> Slurm/Moab and the ACME_script.csh Generate climatologies -> ncclimo Run diagnostics -> metadiags
Project Goals Each part of the process works independently, and flows into the next step Run the model -> Slurm/Moab and the ACME_script.csh Generate climatologies -> ncclimo Run diagnostics -> metadiags Analyze and visualize output -> Python and UVCDAT
Project Goals Each part of the process works independently, and flows into the next step Run the model -> Slurm/Moab and the ACME_script.csh Generate climatologies -> ncclimo Run diagnostics -> metadiags Analyze and visualize output -> Python and UVCDAT Transfer files between facilities -> Globus
Project Goals Each part of the process works independently, and flows into the next step Run the model -> Slurm/Moab and the ACME_script.csh Generate climatologies -> ncclimo Run diagnostics -> metadiags Analyze and visualize output -> Python and UVCDAT Transfer files between facilities -> Globus Archive to HPSS -> HTAR
Project Goals Each part of the process works independently, and flows into the next step Run the model -> Slurm/Moab and the ACME_script.csh Generate climatologies -> ncclimo Run diagnostics -> metadiags Analyze and visualize output -> Python and UVCDAT Transfer files between facilities -> Globus Archive to HPSS -> HTAR Publish to ESGF -> esgf publication python module
Project Goals Each part of the process works independently, and flows into the next step Run the model -> Slurm/Moab and the ACME_script.csh Generate climatologies -> ncclimo Run diagnostics -> metadiags Analyze and visualize output -> Python and UVCDAT Transfer files between facilities -> Globus Archive to HPSS -> HTAR Publish to ESGF -> esgf publication python module That s a lot of tools!
Workbench vision Wrap each tool in a UI Show the options for the tool (no manpage lookup required) Only allow correct input Automate where possible Each tool in one place Zero install on users machine
Workbench progress Data Manager Use Globus to transfer files from Edison Import data from ESGF Publish to our private ESGF staging node Run Manager Create, configure, and execute diagnostic jobs Browse output Upload to Diagnostic Viewer Visualizations Barebones VCS visualization of netCDF files
Future work ACME model run Automatic post processing and data transfer, HPSS archive Job chaining job A triggers job B which triggers job C and D, or an error causes A to restart and email the error to the user. Access controlled sharing for all user data Share anything you want with just the people you want Full visualization power of UV-CDAT VCS in the browser
