Enhancing Memcached Traffic Load Balancing using SDN

Explore the joint research work on Load Balancing Memcached Traffic utilizing Software-Defined Networking (SDN) by Idan Moyal and team from Herzliya, Israel. The study delves into the efficient management of Memcached servers, addressing the Hot Keys Problem and optimizing key assignment to servers based on key popularity. Memcached, a prevalent distributed caching system, plays a crucial role in improving performance in memory-based key-value storage systems.

• Load Balancing
• Memcached
• SDN
• Caching System
• Key Assignment

raega Follow

Uploaded on Sep 15, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Load Balancing Memcached Traffic Using SDN Idan Moyal Interdisciplinary Center Herzliya, Israel Joint work with: Anat Bremler-Barr IDC Herzliya David Hay Hebrew University Liron Schiff Tel-Aviv University Supported by the European Research Council under the European Union's Seventh Framework Programme

2. Memcached Memcached is a widely-used in-memory key value based distributed caching system

3. Memcached Memcached is a widely-used in-memory key value based distributed caching system Movie Reviews Web Application Architecture DBMS Application Server Get movie review Application Server Load Balancer Movie review Application Server

4. Memcached Memcached is a widely-used in-memory key value based distributed caching system Memcached server is picked by applying a hash function on the requested key Movie Reviews Web Application Architecture DBMS CACHE HIT Application Server Memcached Server Memcached Client Memcached Server Get movie review Application Server Memcached Server Load Balancer Memcached Client Movie review Memcached Server Application Server Memcached Server Memcached Client

5. Memcached Memcached is a widely-used in-memory key value based distributed caching system Memcached server is picked by applying a hash function on the requested key Movie Reviews Web Application Architecture DBMS CACHE MISS Application Server Memcached Server Memcached Client Memcached Server Get movie review Application Server Memcached Server Load Balancer Memcached Client Movie review Memcached Server Application Server Memcached Server Memcached Client

6. The Hot Keys Problem Memcached ignores key popularity when assigning keys to servers DBMS Application Server Memcached Server Memcached Client Memcached Server Server Memcached Application Server Memcached Server Load Balancer Memcached Client Memcached Server Application Server Memcached Server High Response Time Memcached Client memcached get guardians-of- the-galaxy-review

7. Key Popularity Follows Zipf Distribution Most popular key

8. Our Solution: MBalancer Our Solution: Duplicate hot keys across many servers Objectives: Balance the load between servers (e.g., minimize average load/max load) Seamless to both memcached client and memcached server Let the network handle the load balancing

9. Our Solution: MBalancer DBMS Application Server Memcached Server Memcached Client Memcached Server Server Memcached Application Server Memcached Server Load Balancer Memcached Client Memcached Server Application Server Memcached Server Memcached Client Good Response Time memcached get guardians-of- memcached get guardians-of- the-galaxy-review the-galaxy-review

10. Our Solution: MBalancer DBMS Same request, different destinations? Application Server Memcached Server Memcached Client Memcached Memcached Server Server Application Server Memcached Server Load Balancer Memcached Client Memcached Server Application Server Memcached Server Memcached Client Good Response Time memcached get guardians-of- the-galaxy-review the-galaxy-review memcached get guardians-of-

11. Our Solution: MBalancer Memcached statistics Using Software Defined Networking (SDN), which separates control plane and data plane: MBalancer Application SDN Controller Data plane match action rules Control plane insert/delete rules Control Plane Data Plane Application Server Memcached Server Memcached Client Memcached Server Server Memcached Application Server Memcached Server Load Balancer Memcached Client Memcached Server SDN Switch Application Server Memcached Server Memcached Client Good Response Time

12. Our Solution: MBalancer Memcached statistics MBalancer Application SDN Controller Control Plane Data Plane Payload matching capabilities of exact location. Common in many SDN Application Server Memcached Server Memcached Client Memcached Server Server Memcached Application Server Load Balancer switches. Memcached Server Memcached Client Memcached Server SDN Switch Application Server Memcached Server Memcached Client Good Response Time memcached get guardians-of- the-galaxy-review

13. Why duplication works? If we look at very few highly-loaded keys: Most of the system s load is concentrated there Very modest memory overhead to duplicate these keys values Only a few switching rules suffices to support load balancing

14. Why duplication works? Imbalance factor = average load / max load The closer the imbalance factor is to 1, the higher the throughput is. Assume only 10 hot keys are duplicated 40-60% improvement Worsen with the size of the system

15. MBalancer Tasks 1. Hot keys identification 2. Load-balance request in the switch. 3. Hot Keys <Key,Value> updating and duplication to additional/all Memcached servers

16. Task 1: Hot Keys Identification Identification using off the-shelf techniques (not part of MBalancer) Tumblr s, Etsy s Can also use sFlow, OpenFlow counters

17. Task 2:Load Balancing in the Switch Using OpenFlow Payload matching Flow Table Entry per hot key get request Entry per hot key set will be used later Group 2. Type: Select Servers containing the hot keys (e.g. all servers) Group Type Select (Since OpenFlow 1.3): Packets are processed by a single bucket in the group. Bucket selection provides equal load sharing (e.g., simlple round robin)

18. Load Balancing in the Switch Using OpenFlow Hot key request Per hot key: 1 get rule, 1 set rule Plus # of groups multiple by group rule size Group rule size is bounded by the number of servers Total number of rules: Payload matching Flow Table Group 2. Type: Select Hot key request redirected (works only for UDP requests) Load Balancing

19. Task 3: Key Updating and Duplication (OpenFlow Implementation) Flow Table Keys are installed and updated with set operations Group 2. Type: Select Hot Keys updates are captured and sent to the controller

20. Key Updating and Duplication MBalancer application contains a memcached client Re-duplicate hot keys on update The switch is configured to match set hot keys packets and send also to the controller MBalancer Application Memcached Client User performs a request which requires updating a hot key SDN Memcached set best-seller Control Plane Controller Memcached get best-seller Data Plane Application Server Memcached Server Memcached Client Memcached Server Application Server Memcached Server Load Balancer Memcached Client SDN Switch Memcached Server Memcached set best-seller Application Server Memcached Server Memcached Client

21. Experimental Results Setting: NoviKit 250 SDN switch (1Gbit/s network). RYU SDN controller. 64 memcached clients, 2 memcached servers. 1000 keys, 100KB values. H=10, Hot Keys are copied to all servers

22. Experimental results Throughput Latency Zipf Zipf 60/40 70/30 80/20 90/10 60/40 70/30 80/20 90/10

23. Comparing with Proxy-Based Solutions Additional servers dedicated for hot keys Hot Keys arrive at the proxy and sent to one of the servers behind it MBalancer does not require additional servers and architecture changes Used by Facebook The number of extra servers required to achieve the same imbalance factor of MBalancer

24. MBalancer: Limitations MBalancer performs load balancing for Memcached UDP get requests. UDP is usually preferred when the goal is better performance (Facebook s solution) SDN switch with payload matching capabilities is required. Memcached key length is limited (NoviKit250 limitation is 46 bytes).

25. Conclusions MBalancer solves an higher level application issue, the hot keys problem in Memcached, using SDN. Solution does not require a change in the application s architecture. Low overhead in the SDN switch and memory footprint. An innovative way to perform L7 load balancing Many extensions in the paper, including P4 implementation.

26. Thank You! Questions?

27. Lorelyn Medina 123RF.com