Distributed Data Flow Language for Multi-Party Protocols

Slide Note

"Exploring a Distributed Data Flow Language designed for Multi-Party Protocols, the research by Krzysztof Ostrowski, Ken Birman, and Danny Dolev from Cornell University and Hebrew University showcases innovative solutions for efficient data processing across distributed systems. The work highlights the importance of seamless communication and coordination in complex network environments."

hollister_a Follow

Uploaded on Oct 10, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript

Distributed Data Flow Language for Multi-Party Protocols Krzysztof Ostrowski , Ken Birman , Danny Dolev Cornell University, Hebrew University {krzys|ken}@cs.cornell.edu, dolev@cs.huji.ac.il

Introduction

http://liveobjects.cs.cornell.edu

http://liveobjects.cs.cornell.edu node1 node2 user distributed peer-to-peer protocols node3

http://liveobjects.cs.cornell.edu node1 node2 user distributed peer-to-peer protocols multicast commit locking node3 replication

http://liveobjects.cs.cornell.edu user shared content distributed peer-to-peer protocols multicast commit locking replication

http://liveobjects.cs.cornell.edu user replicated content distributed peer-to-peer protocols multicast commit locking replication

http://liveobjects.cs.cornell.edu synchronization, coordination user replicated content distributed peer-to-peer protocols multicast commit locking replication

http://liveobjects.cs.cornell.edu node1 node2 user A2 A1 app. component A3 node3

http://liveobjects.cs.cornell.edu instance of the protocol stack Distributed Protocol Instance A2 P2 A1 app. P1 component P3 A3 instance of the protocol stack instance of the protocol stack

http://liveobjects.cs.cornell.edu instance of the protocol stack Distributed Protocol Instance A2 P2 A1 app. P1 component P3 events A3 instance of the protocol stack instance of the protocol stack

http://liveobjects.cs.cornell.edu instance of the protocol stack Distributed Protocol Instance A2 P2 m m network messages m m A1 app. P1 component P3 m events A3 instance of the protocol stack instance of the protocol stack

http://liveobjects.cs.cornell.edu proxy of the object Live Distributed Object (LO) basic building block P2 m m network messages m m P1 P3 m proxy of the object proxy of the object

http://liveobjects.cs.cornell.edu proxy of the object Live Distributed Object (LO) basic building block P2 m m network messages m m P1 P3 m proxy of the object proxy of the object

http://liveobjects.cs.cornell.edu proxy of the object Live Distributed Object (LO) basic building block P2 m m network messages m m P1 P3 m proxy of the object proxy of the object

http://liveobjects.cs.cornell.edu P2 m m P2 m P1 m P3 m m m m P2 P1 m P3 m m m m P2 P1 P3 m m m m m P2 P2 P1 m m m P3 m m m m m need lots of different objects (protocols) P1 P3 m m P1 P3 m m P2 P2 m m m m P1 m P3 m m m P1 P3 m m

http://liveobjects.cs.cornell.edu user-defined objects P2 m m P2 m P1 m P3 m m m m P2 P1 m P3 m m m m P2 P1 P3 m m m m m P2 P2 P1 m m m P3 m m m m m need lots of different objects (protocols) P1 P3 m m P1 P3 m m P2 P2 m m m m P1 m P3 m m m P1 P3 m m

How to Implement New Objects? custom user-defined object P2 m m m P1 P3 m m

How to Implement New Objects? Java / C# / C++ custom user-defined object protocol composition Frameworks (Ensemble, BAST, Appia) P2 m m m P1 P3 m m MACE, P2, etc.

How to Implement New Objects? Java / C# / C++ custom user-defined object protocol composition Frameworks (Ensemble, BAST, Appia) P2 m m m P1 P3 m m MACE, P2, etc.

How to Implement New Objects? Java / C# / C++ custom user-defined object protocol composition Frameworks (Ensemble, BAST, Appia) P2 m m m P1 P3 m m MACE, P2, etc.

How to Implement New Objects? higher-level logic (making decisions) Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) intermingled, tightly-coupled lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping) MACE, P2, etc.

How to Implement New Objects? higher-level logic (making decisions) intermingled, tightly-coupled lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping)

How to Implement New Objects? higher-level logic (making decisions) intermingled, tightly-coupled Protocol 1 Protocol 2 node ACK/NAK recovery lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping) Node node node Protocol 3 Region A A C AC node node node ABC AB BC C B B

How to Implement New Objects? harder to write/debug Java / C# / C++ error- prone protocol composition Frameworks (Ensemble, BAST, Appia) Protocol 1 Protocol 2 node ACK/NAK recovery Node node node Protocol 3 Region A A C AC node node node ABC AB BC C B less B sophisticated less MACE, P2, etc. flexibility

How to Implement New Objects? harder to write/debug Java / C# / C++ error- prone protocol composition Frameworks (Ensemble, BAST, Appia) Protocol 1 Protocol 2 node ACK/NAK recovery Node node node Protocol 3 Region A A C AC node node node ABC AB BC C B less B sophisticated less MACE, P2, etc. flexibility

How to Implement New Objects? harder to write/debug Java / C# / C++ error- prone protocol composition Frameworks (Ensemble, BAST, Appia) Protocol 1 Protocol 2 node ACK/NAK recovery Node node node Protocol 3 Region A A C AC node node node ABC AB BC C B less B sophisticated less MACE, P2, etc. flexibility

How to Implement New Objects? harder to write/debug Java / C# / C++ error- prone protocol composition Frameworks (Ensemble, BAST, Appia) Protocol 1 Protocol 2 node ACK/NAK recovery Node node node Protocol 3 Region A A C AC node node node ABC AB BC C B less B sophisticated less MACE, P2, etc. flexibility

How to Implement New Objects? separation of concerns higher-level logic (making decisions) Protocol 1 Protocol 2 node ACK/NAK recovery lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping) Node node node Protocol 3 Region A A C AC node node node ABC AB BC C B B

How to Implement New Objects? separation of concerns programmer Protocol 1 Protocol 2 node ACK/NAK recovery Node compiler and runtime node node Protocol 3 Region A A C AC node node node ABC AB BC C B B

How to Implement New Objects? separation of concerns programmer Protocol 1 Protocol 2 node ACK/NAK recovery Node compiler and runtime node node Protocol 3 Region A A C AC node node node ABC AB BC C B B

Our Approach: Flows Objects Aggregation Batching Recursion

physical machine (node)

physical machine (node) node x1

software physical machine (node) node x1

layer 1 functional layer software node x1

layer 1 functional layers layer 2 software node x1

layer 1 functional layers interface exposed by layer2 layer 2 node x1

consumed by layer1 layer 1 functional layers interface exposed by layer2 layer 2 node x1

consumed by layer1 method call (event) layer 1 e1 interface exposed by layer2 layer 2 node x1

method call (event) layer 1 e1 occurred at time t0 layer 2 node x1

layer 1 e1 occurred at time t0 at node x1 layer 2 node x1

time layer 1 e1 later occurred at time t0 at node x1 layer 2 time is increasing bottom-up earlier t0 e1 node x1 x1 location (node)

time layer 1 e1 later occurred at time t0 at node x1 layer 2 time is increasing bottom-up earlier t0 e1 node x1 x1 location (node)

time layer 1 e1 later occurred at time t0 at node x1 layer 2 time is increasing bottom-up earlier t0 e1 node x1 x1 location (node)

time layer 1 e1 later layer 2 time is increasing bottom-up earlier t0 e1 node x1 x1 location (node)

time layer 1 layer 1 e2 later layer 2 layer 2 time is increasing bottom-up earlier e2 t1 t0 e1 node x1 node x2 x1 x2 x3 location (node)

time layer 1 layer 1 layer 1 e3 later layer 2 layer 2 layer 2 time is increasing bottom-up e3 t2 earlier e2 t1 t0 e1 node x1 node x2 node x3 x1 x2 x3 location (node)

time layer 1 layer 1 layer 1 e5 e4 later layer 2 layer 2 layer 2 time is increasing bottom-up e4 e5 t3 e3 t2 earlier e2 t1 t0 e1 node x1 node x2 node x3 x1 x2 x3 location (node)