Services for Experienced and Starting HPC Tier 3 Users (SES-HPC) Overview

Discover the SES-HPC initiative at the Jan Steiner Zentrum für Informations- und Medientechnologie, University of Siegen. This project aims to support both seasoned and novice users in navigating High-Performance Computing environments. From motivation to infrastructure details and pillars of support, SES-HPC offers a comprehensive approach for users at all levels.

• HPC
• High Performance Computing
• SES-HPC
• University of Siegen
• Support

rotondi_a Follow

Uploaded on Nov 23, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Services for Experienced and Starting HPC Tier 3 Users (SES-HPC) Jan Steiner Zentrum f r Informations- und Medientechnologie Universit t Siegen

2. Outline Motivation Project Structure First Experiences Status 4.12.17 SES-HPC 2

3. Motivation: Tier 1 National/ International Permeability between Tier 3 and higher tiers Code development on Tier 3 productive runs on Tier 1 Less experienced users Tier 2 Cheaper resources National/Federal Start early Get people onto Tier 3 Facilitate movement to higher tiers Tier 3 Federal/Local 4.12.17 SES-HPC 3

4. Background: Existing Infrastructure User groups on the cluster HorUS: Current Cluster at Uni Siegen Mathematics 1% Electrical Engineering and computer science 1% Physics 6% Diverse users Chemistry and Biology 13% Planned future cluster Proposal underway Additional users, e.g. Big Data Mechanical Engineering 79% Good position to support users early 4.12.17 SES-HPC 4

5. Outline Motivation Project Structure First Experiences Status 4.12.17 SES-HPC 5

6. Five pillars of support Knowledge Transfer Teaching and Training Performance Anlysis Third-party Code Support Tier Change Support 4.12.17 SES-HPC 6

7. Five pillars of support Knowledge Transfer Teaching and Training Performance Analysis Third-party Code Support Tier Change Support Beginner and advanced devs Hold classes Advise on external courses Gauge demand for new courses 4.12.17 SES-HPC 7

8. Five pillars of support Knowledge Transfer Teaching and Training Performance Analysis Third-party Code Support Tier Change Support Beginner and advanced devs Experienced code developers Hold classes Performance reviews Advise on external courses Performance measurement tools Gauge demand for new courses 4.12.17 SES-HPC 8

9. Five pillars of support Knowledge Transfer Teaching and Training Performance Analysis Third-party Code Support Tier Change Support Users of comm- ercial/open-source codes Beginner and advanced devs Experienced code developers Hold classes Performance reviews Support in finding optimal settings Advise on external courses Performance measurement tools Find most suitable hardware Gauge demand for new courses 4.12.17 SES-HPC 9

10. Five pillars of support Knowledge Transfer Teaching and Training Performance Analysis Third-party Code Support Tier Change Support Users of comm- ercial/open-source codes Dev teams who want to apply for higher tier hardw. Beginner and advanced devs Experienced code developers Hold classes Performance reviews Support in finding optimal settings Find most suitable hardware Advise on external courses Performance measurement tools Find most suitable hardware Test and evaluation of software Gauge demand for new courses 4.12.17 SES-HPC 10

11. Five pillars of support Knowledge Transfer Teaching and Training Performance Analysis Third-party Code Support Tier Change Support Users of comm- ercial/open-source codes Dev teams who want to apply for higher tier hardw. Beginner and advanced devs Experienced code developers All HPC users Establish and maintain wiki Hold classes Performance reviews Support in finding optimal settings Find most suitable hardware Advise on external courses Organize networking workshops Performance measurement tools Find most suitable hardware Test and evaluation of software Gauge demand for new courses 4.12.17 SES-HPC 11

12. Outline Motivation Project Structure First Experiences Status 4.12.17 SES-HPC 12

13. Example consulting session Client: PhD student R script, runs 24 cases in sequence Script runs 4 weeks, job would often die before that Paper deadline in 4 weeks This is not to make fun of him 4.12.17 SES-HPC 13

14. Example consulting session Act 1: Troubleshooting What did you set the walltime to? What s a walltime? Default then, which queue was it in? What s a queue? OK never mind, lets have a look at your script. 4.12.17 SES-HPC 14

15. Example consulting session Act 2: The Script Script: same calculation done 24 times just within this script If-blocks with conditions for case 1, 2, , 24 OK, make a shell script with a loop, and set the condition via command line argument for the R script. Then call your script with & What s a shell script? 4.12.17 SES-HPC 15

16. Example consulting session Act 3: Listen, the Mensa is About To Close Copy your script 24 times, write condition=1,2,etc. at the top and name them script_1.r, script_2.r, In the shell script, you write: ./script_1.r & ./script_2.r & ... Thank you so much! It runs 24 times faster now! 4.12.17 SES-HPC 16

17. Lessons learned We let this guy on the cluster Nobody told him anything Cluster website At least google SLURM He ran his jobs for months I met him by sheer coincidence We actually helped him a lot 4.12.17 SES-HPC 17

18. Lessons learned Who is the bigger fool? The fool? The fool who fails to take him by his hand? Source: New Line Cinema 4.12.17 SES-HPC 18

19. Lessons learned Not just user s job to inform themselves Don t know what they don t know Not good: Keep away from cluster Help but also educate (sustainability) 4.12.17 SES-HPC 19

20. Outline Motivation Project Structure First Experiences Status 4.12.17 SES-HPC 20

21. Status Interviews with all institutes that use cluster R-script guy is not completely unique Cluster website review Getting started section Additional feedback: Mech.-Eng. Students Little prior knowledge of Linux Seminars Cluster Introduction , Linux Introduction 4.12.17 SES-HPC 21

22. Wanted: second position Knowledge Transfer Teaching and Training Performance Analysis Third-party Code Support Tier Change Support Users of comm- ercial/open-source codes Dev teams who want to apply for higher tier hardw. Beginner and advanced devs Experienced code developers All HPC users Establish and maintain wiki Hold classes Performance reviews Support in finding optimal settings Find most suitable hardware Advise on external courses Organize networking workshops Performance measurement tools Find most suitable hardware Test and evaluation of software Gauge demand for new courses 4.12.17 SES-HPC 22

23. THANK YOU FOR YOUR KIND ATTENTION. 4.12.17 SES-HPC 23