Enhancing Provenance Research for Reproducibility and Performance

 
P
r
o
v
e
n
a
n
c
e
 
R
e
s
e
a
r
c
h
 
BIBI RAJU, TODD ELSETHAGEN, ERIC STEPHAN
 
1
 
Pacific Northwest National Laboratory, Richland, WA
 
P
r
o
v
e
n
a
n
c
e
 
G
o
a
l
s
 
Provenance Overarching Goals
Provide thorough results explanation
Reuse, repeat, or reproduce
workflows
Enable performance optimization
 
Provenance Work in FY15
Developed a provenance capture
ontology
Provenance infrastructure build out
Scalable provenance capture
mechanism
Developed a Client API that aids in
production of provenance
 
 
2
 
P
r
o
v
e
n
a
n
c
e
 
f
o
r
 
R
e
p
r
o
d
u
c
i
b
i
l
i
t
y
 
a
n
d
P
e
r
f
o
r
m
a
n
c
e
 
P
r
o
v
e
n
a
n
c
e
 
Provenance for Reproducibility
Work towards achieving numerical, experimental and execution reproducibility.
Performance Provenance
Augment/replace existing strategies with the Open Provenance Model-based
WorkFlow Performance Provenance (OPM-WFPP):
Capture empirical performance information from workflows and systems
Links provenance information and performance metrics
 
3
Access
Protocol
 
S
t
a
t
u
s
 
Roadmap for FY16
Incorporate time-series system environment metrics store (In Progress)
Add provenance capture mechanism that can handle the high-velocity
provenance information – scalability (In Progress)
Develop different language bindings for ProvEn Client API
Design services supporting the harvesting of provenance from native
source types
Performance metrics reporting user interface
 
4
 
A
C
M
E
 
 
I
P
P
D
 
 
P
a
n
o
r
a
m
a
 
C
o
l
l
a
b
o
r
a
t
i
o
n
 
Joint route forward, where the Panorama Pegasus workflow is used to
implement an ACME workflow and capture provenance.
ACME workflow with ASCR Integrated end-to-end Performance
Prediction and Diagnosis for Extreme Scientific Workflows (IPPD)
provenance model adapted for ACME
IPPD developed provenance store instance for ACME.
 
5
 
T
h
a
n
k
 
Y
o
u
!
b
i
b
i
.
r
a
j
u
@
p
n
n
l
.
g
o
v
 
6
Slide Note
Embed
Share

Explore the development of provenance research at the Pacific Northwest National Laboratory, focusing on goals, methodologies, and plans for incorporating time-series metrics to increase scalability and performance. Collaboration with ACME IPPD Panorama is highlighted, showcasing advancements in capturing and utilizing provenance data for scientific workflows.


Uploaded on Sep 22, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. Provenance Research BIBI RAJU, TODD ELSETHAGEN, ERIC STEPHAN Pacific Northwest National Laboratory, Richland, WA 1

  2. Provenance Goals Provenance Overarching Goals Provide thorough results explanation Reuse, repeat, or reproduce workflows Enable performance optimization Provenance Work in FY15 Developed a provenance capture ontology Provenance infrastructure build out Scalable provenance capture mechanism Developed a Client API that aids in production of provenance 2

  3. Provenance for Reproducibility and Performance Provenance Provenance for Reproducibility Work towards achieving numerical, experimental and execution reproducibility. Performance Provenance Augment/replace existing strategies with the Open Provenance Model-based WorkFlow Performance Provenance (OPM-WFPP): Capture empirical performance information from workflows and systems Links provenance information and performance metrics Input Workflow WFM WFM WFM Output Appl. Appl. Input Output Access Protocol Access Protocol System Software Protocol OS Protocol Protocol OS Network Storage Network Inter connect Core Core Storage Systems 3

  4. Status Roadmap for FY16 Incorporate time-series system environment metrics store (In Progress) Add provenance capture mechanism that can handle the high-velocity provenance information scalability (In Progress) Develop different language bindings for ProvEn Client API Design services supporting the harvesting of provenance from native source types Performance metrics reporting user interface 4

  5. ACME IPPD Panorama Collaboration Joint route forward, where the Panorama Pegasus workflow is used to implement an ACME workflow and capture provenance. ACME workflow with ASCR Integrated end-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows (IPPD) provenance model adapted for ACME IPPD developed provenance store instance for ACME. 5

  6. Thank You! bibi.raju@pnnl.gov 6

Related


More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#