Understanding OpenFlow Switch Limitations
OpenFlow architecture presents limitations in flow table entries, matching fields, and asynchronous network events between switches and controllers. The background covers current applications, switch design, and network events related to OpenFlow technology.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Background: Current Applications Traffic Engineering application (performance) Fine grained rules and short time scales Coarse grained rules and long time scales Middlebox provision (perf + security) Fine grained rule and long time scales Network services Load balancer: fine-grained/short-time Firewall:fine-grained/long-time Cloud Services Fine grained/long-time scales
Background: Switch Design 13Mbs Network Controller Switch CPU+Mem 35Mbs Hash Table 250GB TCAM
OpenFlow Background: Flow Table Entries OpenFlow rules match on 14 fields Usually stored in TCAM (TCAM is much smaller) Generally 1K-10K entries. Normal switches 100K-1000K entries Only match on 1-2 fields
Background: Switch Design 13Mbs Network Controller Switch CPU+Mem 35Mbs Hash Table 250GB TCAM
OpenFlow Background: Network Events Packet_In (flow-table expect, pkt matches no rule) Asynch from switch to controller Flow_mod (insert flow table entries) Asynch from controller to switch Flow_timeout (flow was removed due to timeout) Asynch from switch to controller Get Flow statistics (information about current flows( Synchronous between switch & controller Controller sends request, switch replies
Background: Switch Design 3. Controller run code to process event 2. CPU create packet-in event, and sends to controller 13Mbs Network Controller Switch CPU+Mem 35Mbs 1. Check Flow table, If no match then Inform CPU From: Theo To: Bruce
Background: Switch Design 4. Controller creates flow event and sends a flow_mod event 2. CPU processes flow_mod and insert into TCAM 13Mbs Network Controller Switch CPU+Mem 35Mbs From: theo, to: bruce, send on port 1 Timeout: 10 secs, count: 0 From: Theo To: Bruce
Background: Switch Design 4. Controller creates flow event and sends a flow_mod event 2. CPU processes flow_mod and insert into TCAM 13Mbs Network Controller Switch CPU+Mem 35Mbs From: theo, to: bruce, send on port 1 Timeout: 10 secs, count: 1 From: Theo To: Bruce
Background: Switch Design 13Mbs Network Controller Switch CPU+Mem 1. Check Flow table 2. Found matching rule 3. Forward packet 4. Update the count 35Mbs From: theo, to: bruce, send on port 1 Timeout: 10 secs, count: 1 From: Theo To: Bruce
Background: Switch Design 13Mbs Network Controller Switch CPU+Mem 1. Check Flow table 2. No matching rule now we must talk to the controller 35Mbs From: theo, to: bruce, send on port 1 Timeout: 10 secs, count: 1 From: Theo To: John
Background: Switch Design 13Mbs Network Controller Switch CPU+Mem 1. Check Flow table 2. Found matching rule 3. Forward packet 4. Update the count 35Mbs From: theo, to: ***, send on port 1 Timeout: 10 secs, count: 1 From: Theo To: John
Background: Switch Design 13Mbs Network Controller Switch CPU+Mem 1. Check Flow table 2. Found matching rule 3. Forward packet 4. Update the count 35Mbs From: theo, to: ***, send on port 1 Timeout: 10 secs, count: 1 From: Theo To: Cathy
Background: Switch Design Problem with Wild-card Too general Can t find details of individual flows Hard to do anything fine- grained Switch CPU+Mem 35Mbs From: theo, to: ***, send on port 1 Timeout: 10 secs, count: 1
Background: Switch Design Doing fine-grained things Think hedera Find all elephant flows Put elephant flows on diff path How to do this? Controller sent get-stat request Switch respond will all stats Controller goes through each request Install special paths Switch CPU+Mem 35Mbs From: theo, to: bruce, send on port 1 Timeout: 1secs, count: 1K From: theo, to: john, send on port 1 Timeout: 10 secs, count: 1
Background: Switch Design Doing fine-grained things Think hedera Find all elephant flows Put elephant flows on diff path How to do this? Controller sent get-stat request Switch respond will all stats Controller goes through each request Install special paths Switch CPU+Mem 35Mbs From: theo, to: bruce, send on port 3 Timeout: 1secs, count: 1K From: theo, to: john, send on port 1 Timeout: 10 secs, count: 1
Problems with Switches TCAM is very small can only support a small number of rules Only 1k per switch, endhost generate lots more flows Controller install entry for each flow increases latency Takes about 10ms to install new rules So flow must wait!!!!!! Can install at a rate of 13Mbs but traffic arrives at 250Gbp Controller getting stats for all flows takes a lot resources For about 1K, you need about MB If you request every 5 seconds then you total:
Background: Switch Design 13Mbs Network Controller Switch CPU+Mem 35Mbs Hash Table 250GB TCAM
Problems with Switches TCAM is very small can only support a small number of rules Only 1k per switch, endhost generate lots more flows Controller install entry for each flow increases latency Takes about 10ms to install new rules So flow must wait!!!!!! Can install at a rate of 13Mbs but traffic arrives at 250Gbp Controller getting stats for all flows takes a lot resources For about 1K, you need about MB If you request every 5 seconds then you total:
Getting Around TCAM Limitation Cloud centric solutions Use Placement tricks Data Center centric solutions Use overlay: use placement tricks General technique: Difane Use Detour routing
DiFane Creates a hierarchy of switches Authoritative switches Lots of memory Collectively stores all the rules Local switches Small amount of memory Stores a few rules For unknown rules route traffic to an authoritative switch
Packet Redirection and Rule Caching Authority Switch Ingress Switch Egress Switch From: theo To; cathy First packet From: theo To: bruce Following packets Hit cached rules and forward Everything else 23
Three Sets of Rules in TCAM Type Priority Field 1 Field 2 Action Timeout 210 In ingress switches reactively installed by authority switches 00** 111* Forward to Switch B 10 sec Cache Rules 209 1110 11** Drop 10 sec 110 00** 001* Forward Trigger cache manager Infinity In authority switches Authority Rules 109 proactively installed by controller 0001 0*** Drop, Trigger cache manager 15 In every switch proactively installed by controller 0*** 000* Redirect to auth. switch Partition Rules 14 24
Stage 1 The controller proactively generates the rules and distributes them to authority switches. 25
Partition and Distribute the Flow Rules Flow space accept Distribute partition information Controller AuthoritySwitch B Authority Switch A Authority Switch C reject Authority Switch B Egress Switch Authority Switch A Ingress Switch Authority Switch C 26
Stage 2 The authority switches keeppackets always in the data plane and reactively cache rules. 27
Packet Redirection and Rule Caching Authority Switch Ingress Switch Egress Switch First packet Following packets Hit cached rules and forward A slightly longer path in the data plane is faster than going through the control plane 28
Virtual Switch Virtual switch has more Mem than hardware switch So you can install a lot more rules in virtual switches Create an overlay between virtual switches Install fine-grained in virtual switches Install normal OSPF rules in HW Can implement everything in virtual switch Has overlay draw-backs.
Bin-Pack in data Centers Insight: traffic is between certain servers If server placed together then their rules are only inserted in one switch
Getting Around CPU Limitations Prevent controller from being in flow creation loop Create clone rules Prevent controller from being in decision loops Create forwarding groups
Clone Rules Insert a special wild card rule When a packet arrives switch makes a micro-flow rule itself Micro-flow inherits all properties of the wildcard rule Switch CPU+Mem 35Mbs From: theo, to: ***, send on port 1 Timeout: 10 secs, count: 1 From: Theo To: Bruce
Clone Rules Insert a special wild card rule When a packet arrives switch makes a micro-flow rule itself Micro-flow inherits all properties of the wildcard rule Switch CPU+Mem 35Mbs From: theo, to: ***, send on port 1 Timeout: 10 secs, count: 1 From: theo, to: Bruce, send on port 1 Timeout: 10 secs, count: 1 From: Theo To: Bruce
Forwarding Groups What happens when there s a failure? Port 1 goes down? Switch must inform the controller Switch CPU+Mem 35Mbs From: theo, to: ***, send on port 1 Timeout: 10 secs, count: 1 From: theo, to: Bruce, send on port 1 Timeout: 10 secs, count: 1 Instead, have backup ports Each rule also states backup
Forwarding Groups What happens when there s a failure? Port 1 goes down? Switch must inform the controller Switch CPU+Mem 35Mbs From: theo, to: ***, send on port 1, backup: 2 Timeout: 10 secs, count: 1 From: theo, to: Bruce, send on port 1, backup2 Timeout: 10 secs, count: 1 Instead, have backup ports Each rule also states backup
How do I do load balancing? Something like ECMP? Or server load-balancing? Switch CPU+Mem Currently, Controller installs rules for each flow do load balancing when installing Controller can do get stats, and load balance later 35Mbs From: theo, to: ***, send on port 1 Timeout: 10 secs, count: 1 From: theo, to: Bruce, send on port 1 Timeout: 10 secs, count: 1
Forwarding Groups Instead, have port-groups Each rule specifies a group of ports to send on Switch CPU+Mem When micro-rule is create Switch can assign ports to micro-rules in a round robin matter Or based on probability 35Mbs From: theo, to: ***, send on port 1,2,4 Timeout: 10 secs, count: 1 From: theo, to: Bruce, send on port 1 Timeout: 10 secs, count: 1
Forwarding Groups Instead, have port-groups Each rule specifies a group of ports to send on Switch CPU+Mem When micro-rule is create Switch can assign ports to micro-rules in a round robin matter Or based on probability 35Mbs From: theo, to: ***, send on port 1(10%), 2(90%) Timeout: 10 secs, count: 1 From: theo, to: Bruce, send on port 2 Timeout: 10 secs, count: 1
Getting Around CPU Limitations Prevent controller from polling switches Introduce triggers: Each rule has a trigger and sends stats to the controller when the threshold is reached E.g. if over 20 pkts match flow, Benefits of triggers: Reduces the number entries being returned Limits the amount of network traffic
Summary Switches have several limitations TCAM space Switch CPU Interesting ways to reduce limitations Place more responsibility in the switch Introduce triggers Have switch create micro-flow rules from general rules
