CHTC System Migration Strategies Revealed at HTCondor Week 2017

Slide Note
Embed
Share

In these detailed presentations, Greg Thain discusses the process of transitioning the CHTC system from RHEL 6 to RHEL 7, highlighting the importance of migration without complications, the necessity of ecryptfs for an encrypted sandbox on CentOS, and the shifting OS preferences of CHTC users towards EL 7. The analysis of EL 7-only and EL 6-only user segments sheds light on the evolving user requirements. Moreover, the upcoming shift towards RHEL 7 for CHTC machines underlines the significance of adaptation in the compute environment.

reeb_lis
reeb_lis Follow

Uploaded on Dec 12, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript

  1. Moving CHTC from Moving CHTC from RHEL 6 to RHEL 7 RHEL 6 to RHEL 7 Greg Thain HTCondor Week 2017

  2. Migration without Migraines

  3. Actually from SL CentOS, not RHEL Need ecryptfs for encrypted sandbox (Actually CentOSplus ) executable = calculate.exe encrypt_execute_directory = true queue

  4. EL

  5. Todays CHTC Users OS %age of CHTC users 5% Require EL 7 Either EL 6 or 7 90% ???? Require EL 6 5% ????

  6. Who are EL 7 only users?

  7. Who are EL 6 only users? Standard universe jobs that have started And ????

  8. Tomorrows CHTC Users OS %age of CHTC users Was 5%, going Was 90%, going ? Was 5%, going Require EL 7 Either EL 6 or 7 Require EL 6

  9. Observation OS %age of CHTC users Was 5%, going Was 90%, going ? Was 5%, going Require EL 7 Either EL 6 or 7 Require EL 6

  10. Moving CHTC machines Moving CHTC machines - -> RHEL 7 > RHEL 7

  11. Moving CHTC machines Moving CHTC machines - -> CentOS7 > CentOS7

  12. Moving CHTC machines Moving CHTC machines - -> CentOS7 Easy not the focus of this talk > CentOS7

  13. Moving CHTC machines Moving CHTC machines - -> CentOS7 jobs jobs > CentOS7

  14. Moving CHTC jobs Moving CHTC jobs - -> CentOS7 users users > CentOS7

  15. I know, Ill deploy containers! Containers as an Infrastructure, or as jobs? CaaI or CaJ? Containers as job require user work to setup Tricky for GPU jobs GOAL: minimize user work!

  16. The Time is now! CHTC formed a team in Nov 2016 Condor + Ops + RCF + OSG Controlled transition with minimum user pain

  17. Plan on a slide Before: Every job gets EL 6 Phase 1: Roll out some # of EL 7 machines, but Default to EL6, users can opt into EL7 or BOTH Communicate to EVERY user Flag Day! Default to EL7, users can opt into EL6 or BOTH

  18. CHTC Pool CHTC execute machines jobs CHTC submit machines foreign execute machines (flocking, OSG glidein, etc.) Foreign submit machines

  19. CHTC Pool CHTC execute machines Want all configuration changes in our Schedds it s the point we control the most! jobs CHTC submit machines foreign execute machines (flocking, OSG glidein, etc.) Foreign submit machines

  20. Attributes: Useful and Otherwise Find the EL attribute? OpSys = "LINUX" OpSysAndVer = "CentOS7" OpSysLongName = "CentOS Linux release 7.3.1611 (Core)" OpSysMajorVer = 7 OpSysName = "CentOS" OpSysShortName = "CentOS" OpSysVer = 703 How to force a job to land here?

  21. Forcing a job to EL 7 OpSys = "LINUX" OpSysAndVer = "CentOS7" OpSysLongName = "CentOS Linux release 7.3.1611 (Core)" OpSysMajorVer = 7 OpSysName = "CentOS" OpSysShortName = "CentOS" OpSysVer = 703 Requirements = OpSysMajorVer == 7 Submit file Works with no startd changes existing glideins, etc.!

  22. Forcing a job to either OpSys = "LINUX" OpSysAndVer = "CentOS7" OpSysLongName = "CentOS Linux release 7.3.1611 (Core)" OpSysMajorVer = 7 OpSysName = "CentOS" OpSysShortName = "CentOS" OpSysVer = 703 Requirements = (OpSysMajorVer == 6) || (OpSysMajorVer == 7) Submit file

  23. Pop Quiz: What about default? OpSys = "LINUX" OpSysAndVer = "CentOS7" OpSysLongName = "CentOS Linux release 7.3.1611 (Core)" OpSysMajorVer = 7 OpSysName = "CentOS" OpSysShortName = "CentOS" OpSysVer = 703 Requirements = nothing said Submit file

  24. Hint: APPEND_REQUIREMENTS won t work OpSys = "LINUX" OpSysAndVer = "CentOS7" OpSysLongName = "CentOS Linux release 7.3.1611 (Core)" OpSysMajorVer = 7 OpSysName = "CentOS" OpSysShortName = "CentOS" OpSysVer = 703 APPEND_REQUIREMENTS is unconditional!

  25. Schedd xforms to the Rescue! # https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=OsMigrationHints JOB_TRANSFORM_NAMES = EL, EL_VER JOB_TRANSFORM_EL @= end REQUIREMENTS JobUniverse == 5 && Regexp("OpSysMajorVer",UnParse(Requirements),"i") =?= false && Regexp("\"WINDOWS\"", UnParse(Requirements)) =?= false SET Requirements (Target.OpSysMajorVer == 6) && $(MY.Requirements) @end JOB_TRANSFORM_EL_VER @= end [ eval_set_WantELVer = isError(int(Regexps("OpSysMajorVer[[:space:]]*=[?]?=[[:space:]]*([0-9])", UnParse(Requirements), "\\\\1", "i"))) ? 0 : int(Regexps("OpSysMajorVer[[:space:]]*=[?]?=[[:space:]]*([0-9])", UnParse(Requirements), "\\\\1", "i")); ] @end

  26. Regexp("OpSysMajorVer",UnParse(Requirements),"i")

  27. Engineering vs. Science Requirements = blah blah blah Requirements = (Target.OpSysMajorVer == 6) && blah blah blah

  28. Engineering vs. Science Requirements = OpSysMajorVer == 7 && blah Requirements = OpSysMajorVer == 7 && blah

  29. Current Status ~ 10 % of CHTC pool running CentOS 7 Some users invited to try CentOS 7 Every one successfully running EL 6 binaries on CentOS 7 with no changes! No transform surprises!

  30. Future work What about OSG? How to ask for EL6 vs EL7? Keep working with users

  31. Summary Summary Users need RHEL 7 sooner than you may think With some work, transition may be less painful for them than you may think

  32. Avoid User Migraines, not ours!

Related


No related presentations.

More Related Content