Prevention & Detection of DP/CP Memory Corruption in DPDK

Most custom packet processing hardware provides mechanisms to separate dataplane and controlplane memory, but similar protection is needed for DPDK applications running on General Purpose Processors. Shared libraries, multi-threaded models, and memory mapping complexities create challenges in preventing and detecting memory corruption. Strategies such as control-plane memory protection, dataplane memory access control, and DPDK mem-zone dump can help address these issues.

• Prevention
• Detection
• Memory Corruption
• DPDK
• Data Processing

charlesto Follow

Uploaded on Feb 18, 2025 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

Presentation Transcript

1. DPDK: Prevention & Detection of DP/CP memory corruption 9 MARCH 2018 AMOL PATEL, SIVAPRASAD TUMALLA

2. Background Most of the custom packet-processing hardware i.e. Network Processors provides mechanisms to, Separate Dataplane and Controlplane memory Protect and detect the Dataplane memory corruption/configuration issues We are Proposing something similar for the DPDK applications running on the GPP (General Purpose Processor)

3. Background Shared Libraries The DPDK applications are running on the multi-threaded model DPDK primary process memory map P r i m a r y P r o c e s s M e m o r y Hugepage mmap Main process spawns worker-threads for the Dataplane processing Stack-n This is a flat memory architecture wherein Control-thread and DP worker- thread s memory is part of the same address-space, it would be difficult to isolate the issue Dataplane/Controlplane data structure corruption Thread - n Huge Page Segment 1 Stack-2 Mem-zone area 1 Thread - 2 Stack-1 Thread - 1 in case of Huge Page Segment 2 Heap Mem-zone area 2 Huge Page Segment 3 text We need ways to prevent/detect the Controlplane/Dataplane corruption. memory

4. Ways to Prevent and Detect DP/CP Memory corruption Prevention Control-plane memory (stack) protection Dataplane memory protection Controlplane and Dataplane memory access control Detection Dataplane Memory Corruption Detection DPDK Mem-zone Dump

5. Prevention Controlplane stack protection Dataplane memory protection Controlplane and Dataplane memory access control

6. Controlplane memory Protection : Existing memory model Worker-threads are the Linux threads Shared Libraries P r i m a r y P r o c e s s M e m o r y Hugepage mmap Stack memory for the worker- threads is allocated in primary- threads Address space Stack-n Thread - n Huge Page Segment 1 Thread level MMU protection does NOT exist Stack-2 Mem-zone area 1 Thread - 2 Stack-1 Thread - 1 All the worker-threads of the primary-process has access to stack memory of all the other threads, one worker thread can corrupt the other worker thread s stack Huge Page Segment 2 Heap Mem-zone area 2 Huge Page Segment 3 text

7. Worker thread Stack Protection: Proposed Memory Model Mechanism: Shared Libraries Stack of the worker-thread to be allocated on mem-zone Hugepage mmap P r i m a r y P r o c e s s Assign the allocated mem-zone memory to the worker-thread stack address, Stack-n Worker Thread-n int thread_attr_setstack(pthread_attr_t *attr void *stackaddr, size_t stacksize); Huge Page Segment 1 Stack-2 Mem-zone area 1 WorkerThread - 2 Stack-1 Advantages: Worker Thread- 1 Stricter control on any possible out-of-bound memory access for the stack variables Huge Page Segment 2 Heap Localizing the stack corruption effect to the thread Mem-zone area 2 Huge Page Segment 3 text Ability to dump the thread s stack contents by dumping the mem-zone

8. DP Memory Protection: Existing DPDK Dataplane Memory Layout Mem-zone 1 DP table Huge Page Segment 1 Mechanism: Mem-zone 2 Actual tables are allocated using the rte_lmp_create/rte_hash_create which internally invokes the rte_dpdk callocs() to allocate memory from the Hugepage. Huge Page Segment 2 Mem-zone 2 RTE_CONFIG Shared Libraries Disadvantages Huge Page Segment 3 Memory allocated for the different Dataplane tables might be scattered in different memory-segments/huge pages Stack Thread - n Stack Thread - 2 Stack Thread - 1 Huge Page Segment 4 No direct way to check for out-of-bound write for the Dataplane tables malloc, calloc, zmalloc regions DP tables DP caches DP counters Huge Page Segment 4 There is no way to dump the contents of all the Dataplane tables and counters in one go Other tables Heap - n Huge Page Segment 5 text

9. DP Memory Protection: Proposed DPDK DP Memory Allocation using memory zones DP table1 DP-Mem- zone Guard-band Huge Page Segment 1 Mechanism One mem-zone reserved for all the dataplane tables, data-structures, all the DP tables are placed on this this mem-zone by a user-defined wrappers Advantages DP table2 DP-Mem-zone Huge Page Segment 2 DP-Mem-zone Guard-band DP-Mem- zone DP counters Shared Libraries Dataplane memory separation Huge Page Segment 3 DPDK memory zone dump APIs supported to dump all the DP tables. Stack Thread - n Ease the troubleshooting the memory and configuration issues Stack Thread - 2 Stack Thread - 1 Huge Page Segment 4 Guard-band allows checking of any out-of-bound access Disadvantages rte_malloc, rte_calloc Huge Page Segment 4 Heap - n Additional wrappers for allocating the Dataplane tables (LPM, Hash, counters) on the mem-zones Huge Page Segment 5 text

10. CP / DP Memory Access Control Worker thread to access the dataplane-table memzones in Read- only mode (mmap call supports the RO access, DPDK would have to add this support). Dataplane code not to change any dataplane entries fastpath counters except the Control plane tracking the #entries for the DP tables and returning error if it s reached MAX (this way we can avoid any over-stepping to the next DP table) In the worker threads, all the dataplane pointers accessed with primitive to avoid any accidental corruption MUST constant be the Using restrict qualifier for all Read- Only function argument

11. Detection Detection (Debugging / troubleshooting) Dataplane Memory Corruption Detection DPDK Mem-zone Dump

12. Dataplane Memory Corruption Detection Single DP memzone for the dataplane objects (forwarding tables, counters, structure etc.), each DP reserves few extra-rows for every DP table (Max + N) which serves as the guard-band Monitoring the guard-bands b/w the individual DP tables to keep check on any out of bound access Use DPDK utilities to dump the memzones to dump the DP tables, counters for debugging Application test-framework to check any out-of-bound access for DP tables by monitoring the memzone

13. 4 Stage Memzone Dumping Process DPDKProcInfo m : Dump Memzone physical and Virtual address cat /proc/<pid>/maps : Dump process virtual memory map Huge_page mapping Indentify the Memzone Hugepage mapping by comparing the virtual-address hexdump /mnt/huge/mntrte_N : Dump the Memzone content

14. Detection DPDK Memzone Dump : dpdkprocinfo m

15. cat /proc/<pid>/maps

16. Troubleshooting DPDK Memzone Dump : hexdump /mnt/huge/rtemap_

17. Questions?