Cognitive Publish/Subscribe for Heterogeneous Clouds

This study explores a cognitive pub/sub system designed for heterogeneous clouds, focusing on decentralized architecture to adapt to network dynamics. It discusses scalable pub/sub services, tradeoffs in node-degree, dissemination structures, gossip-based pub/sub overlays, and cognitive structuring for clustering peers with similar interests in cloud environments.

• Cloud Computing
• Pub/Sub System
• Decentralized Architecture
• Cognitive Overlay
• Heterogeneous Environments

mur_b Follow

Uploaded on Feb 24, 2025 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

Presentation Transcript

1. Cognitive Publish/Subscribe for Heterogeneous Clouds ar nas Girdzijauskas, Swedish Institute of Computer Science (SICS) sarunas@sics.se Joint work with: Fatemeh Rahimian (SICS)

2. Future Clouds? Based on decentralized architecture Abundance of networked collection of connected devices forming micro-clouds Decentralized Publish/Subscribe service Content distribution IP TV Streaming Online gaming Collaborative editing Etc.. Adapting to the topology and network dynamics of microclouds Adapting to different usage patterns ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

3. Pub/Sub Systems: Our Focus Scalable pub/sub service Very large number of nodes Very large number of topics Heterogeneous environments Arbitrary geographical distribution Arbitrary subscription and dissemination patterns Central solutions will not scale ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

4. Pub/Sub Systems: Our Focus (2) Tradeoffs: Node degree Number of uninterested (relay) nodes involved Dissemination delay Dissemination cost Cognitive pub/sub: Fixed node degree Account for the underlying topology (bandwidth & cost) Minimize the number of relay nodes by exploiting user subscription correlation & event publication rates ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

5. Conceptual Architecture Pub/sub Dissemination structures Cognitive Overlay Physical Network ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

6. Gossip based pub/sub Gossip (epidemic) overlays A lot of research (e.g., Cyclon, T-man) Lightweight, scalable and robust mechanism Cyclic/Periodic, pair-wise interaction between peers (bounded amount of information) ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

7. Towards Cognitive Structure Gossiping enables us to find and cluster peers with similar interests connected by cheap and fast links A node starts with a local fixed size view in a random network Performs a bidirectional exchange of the view with a random node 2 views Keeps the only the preferred (ranking function) nodes in the view 1 view Repeat ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

8. Building Cognitive Structure Making clusters by utilizing ranking function which prefers neighbors with similar interests Peer interest similarity metric Node subscriptions s1, s2 T sim(s1, s2) =|s1 s2|/|s1 s2| Gossiping Weighted by link cost (bandwidth and $) Weighted by Topic publication rates $ Number of neighbors is limited! Decided locally on each peer ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

9. Problem: How to publish? Clustering peers of similar interests into bandwidth and cost effective clusters Clusters might (will) be disjoint Event publishing requires connected components for each topic ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

10. Inter-Cluster Connectivity Navigable Small-World Network Structure is added: Navigable Small-World topology Purely by using gossiping 14 12 24 7 32 13 10 34 9 6 20 5 29 16 18 2 31 21 8 15 22 28 3 4 1 23 27 19 35 17 26 30 11 ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

11. Building Navigable Structure Every peer decides on random ID Updating ranking function for choice of neighbors: Ring Link(s) Long-Range link (Small- World style) for polylogarithmic routing performance 2 5 8 1 4 3 7 6 9 10 ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

12. Inter-Cluster Connectivity Structure is added (Navigable Small-World made by gossiping) Ring Links Long-Range (finger) link(s) Clustering (friend) links Clusters are connected by greedy routes Rendezvous node for each topic All links are used! All topics become connected For publishing flood the topic , or Choose a rendezvous node to publish Navigable Small-World Network 14 12 24 7 32 13 10 34 9 6 20 5 29 16 18 2 31 21 8 15 22 28 3 4 1 23 27 19 35 17 26 30 11 ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

13. Ongoing work Synthetic data sets for user subscription correlation Twitter data set Skype churn data Our experiments show: Up to 10 fold reduction of relay traffic as compared to existing approaches (e.g., Scribe, Bayeux) ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

14. Gossip based pub/sub (recap) Large scale pub/sub for heterogeneous environments Dissemination structures are self-organizing Forming clusters of similar nodes Converging into least expensive dissemination paths on the underlying physical network Continuously adapting to the environment conditions Fast convergence, robustness to churn and failures. ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010

15. Thank you! Questions? ar nas Girdzijauskas, Cloud Futures, Redmond, April 2010