Understanding RoCEv2: Updates from the IBTA

rocev2 update from the ibta l.w
1 / 11
Embed
Share

Explore the latest updates on RoCEv2 protocol advancements presented at the IBTA. Learn about the evolution of RoCE from v1, packet formats, timelines, data center implementations, and the need for IP routable RoCE. Dive into details about RoCEv2 specifications and its significance in RDMA applications.

  • RoCEv2
  • IBTA
  • RDMA
  • Data Centers
  • IP Routable

Uploaded on | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

E N D

Presentation Transcript


  1. RoCEv2 Update from the IBTA #OFADevWorkshop RoCEv2 Update from the IBTA

  2. Introduction - The RoCE (v1) Protocol Stack Introduction RoCE (v1) RDMA Application / ULP OFA Verbs Interface OFA* Stack IB IB Transport Protocol Transport Protocol Network Layer Network Layer InfiniBand Link Layer Ethernet Link Layer RoCE InfiniBand RoCE Specification InfiniBand Management Ethernet Management RoCEv2 Update from the IBTA 2

  3. The RoCE(v1) Packet Format InfiniBand LRH (L2 Hdr) BTH+ (L4 Hdr) L3 Hdr IB Payload ICRC VCRC No Changes Eth L2 Header BTH+ (L4 Hdr) RoCE L3 Hdr IB Payload ICRC FCS RoCEv2 Update from the IBTA 3

  4. RoCE(v1) Timeline May 2009 - IBTA forms RoCE WG July 2009 RoCE Prototype Available Aug 2009 Position paper on RoCE at HOTI April 2010 - Ratified IBTA Standard October 2010 RoCE in upstream Linux RoCEv2 Update from the IBTA 4

  5. RoCE(v1) and L2-Datacenters RoCE is a L2 Protocol ( flat ) L2 Eth Topology L2 Extensions are Supported TRILL Provider Networks (PB, PBB, etc) Virtualization Overlays RDMA within an Ethernet L2 Domain FCoE is similar Plain L2 or TRILL or PBB or Virtualization Overlays L2 L2 L2 L2 Scale up with Single Ethernet L2 Domain (from endnode perspective) RoCE within a Single Rack RoCEv2 Update from the IBTA 5

  6. The Need for (IP) Routable RoCE A Common Class of L3 Datacenter Nodes within a Rack share a Ethernet L2 Domain TOR Device L2 Switch for intra-rack communication L3 (IP) Router for inter-rack communication other topologies also apply L2 L2 L2 Customer Demand RDMA Across Racks i.e. Across IP Subnets Focus on Data Center Networks RoCE is Already close to being IP Routable RoCE Address in API is IP RoCE Packet Format / Wire Protocol Includes L3 Header L2 Domain L2 Domain L2 Domain RoCEv2 Update from the IBTA 6

  7. RoCEv2 A Straightforward Extension RDMA Application / ULP Software OFA Verbs Interface OFA* Stack IB IB IB Transport Protocol Transport Protocol Transport Protocol Typically HW UDP RoCEv2 Specification IB Network Layer IB Network Layer IP InfiniBand Link Layer Ethernet Link Layer Ethernet Link Layer RoCE InfiniBand RoCE RoCE v2 Specification InfiniBand Management Ethernet / IP Management Ethernet / IP Management RoCEv2 Update from the IBTA 7

  8. RoCEv2 - IP Routable Packet Format EtherType indicates that packet is RoCE (i.e. next header is IB GRH) Eth L2 Header IB BTH+ (L4 Hdr) EtherType RoCE IB GRH IB Payload ICRC FCS RoCEv2 Eth L2 Header IP IB BTH+ (L4 Hdr) EtherType Proto # Port # UDP Header IB Payload ICRC FCS Header EtherType indicates that packet is IP (i.e. next header is IP) UDP dport number Indicates that next header is IB.BTH ip.protocol_number indicates that packet is UDP RoCEv2 Update from the IBTA 8

  9. RoCEv2 Highlights Contained Change Clean L3 Replacement Strict Layering Preserved Generated and Consumed Below the Channel Interface (i.e. the API ) Transparent to Applications No Application Software Changes Current RoCE API is already IP L3 based Transparent to Underlying Network Infrastructure Mainstream processing on L2 Ethernet Switches / L3 IP Routers Additional Benefits of the RoCEv2 Approach (some examples) Traditional Network Management Tools Apply ACLs (Metering, Accounting, Firewalling) IGMP Snooping for optimized Multicast Network Monitoring Tools RoCEv2 Update from the IBTA 9

  10. Status Update IBTA SC Requested Technical Work on RoCEv2 (Nov 2013) IBTA Invited to give Presentation at November 2013 IETF Meeting IBTA Starting Definition of IP Routable RoCE Well Received by the IETF Community IBTA IBXoE Working Group Re-Activated to Specify RoCEv2 Started Work in December 2013 Initial RoCEv2 Specification Draft Undergoing IBXoE WG Review Next Steps RoCEv2 Update from the IBTA 10

  11. Thank You #OFADevWorkshop

Related


More Related Content