Abstracting Queues in the UApi for Improved Network Performance

undefined
 
Making queues a first class citizen :
Abstracting queues in the UApi
 
Magnus Karlsson & Björn Töpel, Intel
 
Problem Statement
 
2
bind(af_xdp_fd, netdev, queue_id)
Ethtool using queue_id
UAPI
 
Drivers decide what queue ids to use for what. No standard
Only possible to use pre-configured queues in the driver. No dynamic
creation
XDP and AF_XDP Tx queues invisible to user space
How to get rid of HW queue ids in user space and still be able to create an
AF_XDP socket and direct traffic to it?
Netdev
queues
XDP Tx
queues
AF_XDP Tx &
Rx queues
NIC specific
queues
 
Basic Requirements
 
Rx:
 Create a dedicated Rx queue and steer traffic to it
Bridge the gap with a getsockopt to get queue ids for use in ethtool
Tx:
 Multiple users (XDP Tx and AF_XDP Tx) but no traffic steering
Traffic shaping and QoS can be present though
Requirements:
1.
UAPI: create an AF_XDP socket without specifying a HW queue id
2.
Kernel: common module/functionality for allocating, freeing and
naming/enumeration of queues
3.
Drivers: No change to drivers that do not use this feature
4.
UAPI: configuring traffic-steering and QoS to and from this socket
By NIC HW steering or XDP programs
Will focus on 1 – 3 only in this presentation
 
3
Register, alloc & free
 
High-Level Proposal: Option 1 – Hide Queues
 
Driver registers queues with QM (non-netdev vs all?)
Components that need queues can alloc, free and lookup in QM
QM uses new ndo to perform allocs and frees in the driver
 
4
fd = bind(af_xdp_fd, netdev)
Ethtool using fd (or
opaque queue_id)
UAPI
Queue
module (QM)
XDP Tx
queues
AF_XDP Tx &
Rx queues
Xsk
Ethtool
Driver
Alloc, free & lookup fd -> HW qid
XDP
Register, alloc & free
 
High-Level Proposal: Option 2 – Expose Queues
 
Expose queues and required info to user space to make a decision
New queues can be created through ethtool or devlink, if needed
Feed the desired queue id into bind
Flow objects could be added to control steering
 
5
fd = bind(af_xdp_fd, netdev, queue_id)
Devlink to expose and
manipulate queues
UAPI
Queue
module (QM)
XDP Tx
queues
AF_XDP Tx &
Rx queues
Xsk
Devlink
Driver
Alloc, free & lookup fd -> HW qid
XDP
 
Reflections / Questions
 
Is the queue manager uneccesary and a subdev should be used?
Switchdev / tc offload to steer traffic
Extend __rx and __tx structs to implement a queue manager?
Socket, abstracted queue id or extended queue id in user space?
Should netdev use the queue manager too?
XDP Tx and AF_XDP Tx two separate paths. Unify?
 
AF_XDP ToDo list on xdp-project repo
Need infra in user space to list queues
 
6
 
Feedback from the Meeting
 
Option 2 preferred
Provides advanced configuration and control
Simplified user interface can be provided with libbpf
Start with the queue object, then the flow object
Use devlink to expose and manipulate queues and flows
Keep the current interface to bind()
Queue id can be treated as an opaque
 
7
undefined
Slide Note
Embed
Share

Abstracting queues in the UApi is crucial for enabling more efficient traffic steering and management in network devices. The proposal aims to enhance the handling of queues by hiding or exposing them to user space, allowing for better scalability and flexibility in network configurations.

  • Network Performance
  • Traffic Steering
  • Queue Abstraction
  • UApi Enhancement
  • Network Configuration

Uploaded on Sep 07, 2024 | 4 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. Making queues a first class citizen : Abstracting queues in the UApi Magnus Karlsson & Bj rn T pel, Intel Intel Public

  2. Problem Statement Ethtool using queue_id bind(af_xdp_fd, netdev, queue_id) UAPI NIC specific queues Netdev queues XDP Tx queues AF_XDP Tx & Rx queues Drivers decide what queue ids to use for what. No standard Only possible to use pre-configured queues in the driver. No dynamic creation XDP and AF_XDP Tx queues invisible to user space How to get rid of HW queue ids in user space and still be able to create an AF_XDP socket and direct traffic to it? Network Platforms Group 2 Intel Public

  3. Basic Requirements Rx: Create a dedicated Rx queue and steer traffic to it Bridge the gap with a getsockopt to get queue ids for use in ethtool Tx: Multiple users (XDP Tx and AF_XDP Tx) but no traffic steering Traffic shaping and QoS can be present though Requirements: 1. UAPI: create an AF_XDP socket without specifying a HW queue id 2. Kernel: common module/functionality for allocating, freeing and naming/enumeration of queues 3. Drivers: No change to drivers that do not use this feature 4. UAPI: configuring traffic-steering and QoS to and from this socket By NIC HW steering or XDP programs Will focus on 1 3 only in this presentation Network Platforms Group 3 Intel Public

  4. High-Level Proposal: Option 1 Hide Queues Ethtool using fd (or opaque queue_id) fd = bind(af_xdp_fd, netdev) UAPI Xsk Ethtool XDP Queue module (QM) Alloc, free & lookup fd -> HW qid Register, alloc & free Driver AF_XDP Tx & Rx queues XDP Tx queues Driver registers queues with QM (non-netdev vs all?) Components that need queues can alloc, free and lookup in QM QM uses new ndo to perform allocs and frees in the driver Network Platforms Group 4 Intel Public

  5. High-Level Proposal: Option 2 Expose Queues Devlink to expose and manipulate queues fd = bind(af_xdp_fd, netdev, queue_id) UAPI Xsk Devlink XDP Queue module (QM) Alloc, free & lookup fd -> HW qid Register, alloc & free Driver AF_XDP Tx & Rx queues XDP Tx queues Expose queues and required info to user space to make a decision New queues can be created through ethtool or devlink, if needed Feed the desired queue id into bind Flow objects could be added to control steering Network Platforms Group 5 Intel Public

  6. Reflections / Questions Is the queue manager uneccesary and a subdev should be used? Switchdev / tc offload to steer traffic Extend __rx and __tx structs to implement a queue manager? Socket, abstracted queue id or extended queue id in user space? Should netdev use the queue manager too? XDP Tx and AF_XDP Tx two separate paths. Unify? AF_XDP ToDo list on xdp-project repo Need infra in user space to list queues Network Platforms Group 6 Intel Public

  7. Feedback from the Meeting Option 2 preferred Provides advanced configuration and control Simplified user interface can be provided with libbpf Start with the queue object, then the flow object Use devlink to expose and manipulate queues and flows Keep the current interface to bind() Queue id can be treated as an opaque Network Platforms Group 7 Intel Public

  8. Intel Public

More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#