Status of Belle.II Computing Meeting at KEK Japan
undefined
Status of Belle II
Data Handling Group
Kihyeon Cho
High Energy Physics Team
KISTI (Korea Institute of Science and Technology Information)
April 19, 2011
Belle II Computing Meeting, KEK, Japan
1
Contents
•
Schedule
•
Activities of AMGA team
•
Snap shot vs. Query
•
Standalone data registration
2
Schedule
•
Korea Belle/Belle II Meeting
•
Date: 2011. 4. 30 (Saturday)
•
Place: Hanyang University, Seoul, Korea
•
Report on Belle II DH Group
•
Junghyun Kim and Kihyeon Cho
•
Belle II computing workshop (?)
3
AMGA Team
•
Sunil Ahn, Taesang Huh, Soonwook Hwang
•
AGMA itself
•
AMGA v 2.1.2 released
•
Federation feature
•
GUI client toolkit
•
Included in the EMI-1 release
•
AMGA for Belle II
•
To plug AMGA system (KISTI) into Grid farm => done
4
AMGA test at Grid Farm (since January)
Belle VO
Belle II VO
@ AMGA Team
@ HEP Team
@ Melbourne
•
Sunil has written documentation of how to submit grid jobs.
•
We had tested AMGA Sever at HEP team(150.183.246.196).
•
AMGA Server at Melbourne is in Grid.
•
To test AMGA server at KEK => delayed
5
1. Snap shot
2. Query
•
Query vs. Snap shot => Snap shot (1.5 TB/user)
•
Junghyun is making new idea since snap shot increases meta-data
size due to duplication
Snap shot vs. Query
6
Query
…
3. Snapshot-like Method => Working on
Delete
Delete
OK?
OK?
Delete
Keep metadata
(Change Permission)
yes
yes
No
No
A
A
B
B
C
C
Query
Metadata
•
Junghyun Kim, Sunil Ahn and Taesang Huh are working on it.
•
Snap-shot style => snap-shot like style for user created data
•
To store queries which are not duplicated
•
To keep the metadata for old data
7
Martin’s mail (4/14)
•
Data Now that many Belle skim datasets have been created and distributed
around the world, I think it is very important we implement a dataset
registration tool for our distributed computing solution.
This is defined in redmine feature 196:
http://ekpbelle2.physik.uni-karlsruhe.de/redmine/issues/196
The feature would place a dataset on a grid enabled storage element,
register it with the LFC and place the appropriate metadata associated
with the skim in the AMGA meta-database.
With this tool we can begin to use our distributed data analysis
system to analyse Belle data. This project is particularly vital given
the situation with B-computer at KEK. We have access to large amounts
of computing power on the grid but without this feature we can't
really use it.
I thought that you might be interested working together in developing
this feature since it involves data handling, AMGA and the python
interface to AMGA. It would also give you a chance to become familiar
with gbasf2.
8
#196 Standalone dataset
registration tool kit
•
Junghyun and Taesang Huh
•
C++ module
for Belle II
1
st
step. Gbasf2 @ KISTI
•
To check Logfile message and meta data
2
nd
step C++ Module
•
Open metadata system
•
New metadata system with registration
•
Close metadata system
•
A couple of month
9
Thank you.
10
Belle.II Computing Meeting held on April 19, 2011, at KEK, Japan, focusing on the data handling group's progress. The meeting included discussions on activities, schedule, AMGA team updates, testing at the Grid Farm, snapshot vs. query methods, and dataset registration tool implementation. Key participants included Kihyeon Cho, Junghyun Kim, Sunil Ahn, Taesang Huh, and Soonwook Hwang from KISTI and other institutions.
Uploaded on Feb 20, 2025 | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
April 19, 2011 Belle II Computing Meeting, KEK, Japan Status of Belle II Data Handling Group Kihyeon Cho High Energy Physics Team KISTI (Korea Institute of Science and Technology Information) 1
Contents Schedule Activities of AMGA team Snap shot vs. Query Standalone data registration Kihyeon Cho 2
Schedule Korea Belle/Belle II Meeting Date: 2011. 4. 30 (Saturday) Place: Hanyang University, Seoul, Korea Report on Belle II DH Group Junghyun Kim and Kihyeon Cho Belle II computing workshop (?) Kihyeon Cho 3
AMGA Team Sunil Ahn, Taesang Huh, Soonwook Hwang AGMA itself AMGA v 2.1.2 released Federation feature GUI client toolkit Included in the EMI-1 release AMGA for Belle II To plug AMGA system (KISTI) into Grid farm => done Kihyeon Cho 4
AMGA test at Grid Farm (since January) Belle VO Belle II VO Grid Farm UI @ AMGA Team @ HEP Team @ Melbourne AMGA Server Farm KISTI Grid farm (sdfarm.kr) KEK Grid farm (kek2-cd05.cc.kek.jp) Sunil has written documentation of how to submit grid jobs. We had tested AMGA Sever at HEP team(150.183.246.196). AMGA Server at Melbourne is in Grid. To test AMGA server at KEK => delayed Belle VO OK (By Sunil) Belle II VO N/A OK (By Tom) OK (By Sunil ) Kihyeon Cho 5
Snap shot vs. Query 1. Snap shot A Metadata Duplicated Metadata Duplicated Metadata B C 2. Query A Metadata B C Query Query Query vs. Snap shot => Snap shot (1.5 TB/user) Junghyun is making new idea since snap shot increases meta-data size due to duplication 6
3. Snapshot-like Method => Working on Delete yes Delete OK? A Metadata No Keep metadata (Change Permission) B C Query Query Junghyun Kim, Sunil Ahn and Taesang Huh are working on it. Snap-shot style => snap-shot like style for user created data To store queries which are not duplicated To keep the metadata for old data 7
Martins mail (4/14) Data Now that many Belle skim datasets have been created and distributed around the world, I think it is very important we implement a dataset registration tool for our distributed computing solution. This is defined in redmine feature 196: http://ekpbelle2.physik.uni-karlsruhe.de/redmine/issues/196 The feature would place a dataset on a grid enabled storage element, register it with the LFC and place the appropriate metadata associated with the skim in the AMGA meta-database. With this tool we can begin to use our distributed data analysis system to analyse Belle data. This project is particularly vital given the situation with B-computer at KEK. We have access to large amounts of computing power on the grid but without this feature we can't really use it. I thought that you might be interested working together in developing this feature since it involves data handling, AMGA and the python interface to AMGA. It would also give you a chance to become familiar with gbasf2. Kihyeon Cho 8
#196 Standalone dataset registration tool kit Junghyun and Taesang Huh C++ module for Belle II 1ststep. Gbasf2 @ KISTI To check Logfile message and meta data 2ndstep C++ Module Open metadata system New metadata system with registration Close metadata system A couple of month Kihyeon Cho 9
Thank you. 10
