DeltaINT: General In-band Network Telemetry with Low Bandwidth Overhead
D
e
l
t
a
I
N
T
:
T
o
w
a
r
d
G
e
n
e
r
a
l
I
n
-
b
a
n
d
N
e
t
w
o
r
k
T
e
l
e
m
e
t
r
y
w
i
t
h
E
x
t
r
e
m
e
l
y
L
o
w
B
a
n
d
w
i
d
t
h
O
v
e
r
h
e
a
d
S
i
y
u
a
n
S
h
e
n
g
1
,
3
,
Q
u
n
H
u
a
n
g
2
,
a
n
d
P
a
t
r
i
c
k
P
.
C
.
L
e
e
3
,
1
University of Chinese Academy and Sciences
2
Peking University
3
The Chinese University of Hong Kong
1
I
n
-
b
a
n
d
N
e
t
w
o
r
k
T
e
l
e
m
e
t
r
y
(
I
N
T
)
Source pushes control information and device-internal states
Transit pushes states according to control information
Sink extracts INT information and reports an event
2
INT framework
L
i
m
i
t
a
t
i
o
n
s
o
f
I
N
T
Significant bandwidth overhead
•
Linearly grow with the length of forwarding path
•
Reduce effective bandwidth for network applications
•
Increase likelihood of IP-level fragmentation
Example
•
5-node fat-tree topology in data center
•
Trace device ID, ingress port, and egress port, of 4B each
•
12B per-node states and 8B INT control information
•
68B in total
at least 4.53% of 1,500B MTU in Ethernet
3
E
x
i
s
t
i
n
g
S
t
u
d
i
e
s
Sampling-based methods
•
Embed INT information to only a subset of sampled packets
•
R
e
d
u
c
e
b
a
n
d
w
i
d
t
h
o
v
e
r
h
e
a
d
y
e
t
w
i
t
h
s
l
o
w
c
o
n
v
e
r
g
e
n
c
e
•
Cannot retrieve INT information unless collecting sufficient packets
Other methods
•
Designed for specific telemetry tasks
A
l
l
e
x
i
s
t
i
n
g
m
e
t
h
o
d
s
s
u
f
f
e
r
f
r
o
m
l
o
w
g
e
n
e
r
a
l
i
t
y
•
Cannot support all families of common applications
4
O
u
r
C
o
n
t
r
i
b
u
t
i
o
n
s
D
e
l
t
a
I
N
T
,
a
g
e
n
e
r
a
l
I
N
T
f
r
a
m
e
w
o
r
k
•
Extremely low bandwidth overhead
•
High generality and convergence
Theoretical analysis on bandwidth mitigation guarantees
Software simulation for various applications
•
For example, reducing up to 93% bandwidth cost in gray failure detection
P4-based hardware implementation
Open-source DeltaINT prototype
5
F
o
u
r
F
a
m
i
l
i
e
s
o
f
A
p
p
l
i
c
a
t
i
o
n
s
Per-packet-per-node monitoring
•
Collect per-node states for each packet (e.g., gray failure detection)
Per-packet aggregation
•
Aggregate per-node states for each packet (e.g., congestion control)
Static per-flow aggregation
•
Collect static per-node states for each flow (e.g., path tracing)
Dynamic per-flow aggregation
•
Aggregate per-node states for each flow (e.g., latency measurement)
6
O
u
r
S
o
l
u
t
i
o
n
Key observation
•
D
e
l
t
a
,
t
h
e
c
h
a
n
g
e
b
e
t
w
e
e
n
c
u
r
r
e
n
t
s
t
a
t
e
a
n
d
e
m
b
e
d
d
e
d
s
t
a
t
e
•
D
e
l
t
a
i
s
o
f
t
e
n
n
e
g
l
i
g
i
b
l
e
a
t
m
o
s
t
t
i
m
e
i
n
t
y
p
i
c
a
l
a
p
p
l
i
c
a
t
i
o
n
s
•
For example, relatively stable hop latency and static device IDs
Motivating example
7
P
e
r
-
n
o
d
e
A
r
c
h
i
t
e
c
t
u
r
e
i
n
D
e
l
t
a
I
N
T
Per-node architecture
•
Calculate the delta between current states and embedded states
•
Only if the delta exceeds a threshold, we insert current states into a
packet and update the embedded state
How to maintain embedded states efficiently in data plane?
8
S
k
e
t
c
h
i
n
g
i
n
D
e
l
t
a
I
N
T
Sketch-based technique
•
Store approximate information with limited memory and computations
•
Track embedded states in the data plane with limited resources
Per-node sketch data structure
•
Each bucket stores a flowkey and the embedded states
•
Each entry of a packet includes a bitmap and the states being embedded
9
P
r
i
m
i
t
i
v
e
s
i
n
D
e
l
t
a
I
N
T
Four primitives to form DeltaINT workflow
•
StateLoad
•
Hash flowkey and load embedded states from the first bucket matching flowkey
•
DeltaCalc
•
Calculate the delta and compare with the predefined threshold
•
StateUpdate
•
Update flowkey and relevant embedded states in the hashed buckets
•
MetadataInsert
•
Insert a bitmap and the states with non-negligible deltas into the packet
Fit DeltaINT into applications with slight changes to primitives
10
U
p
d
a
t
e
E
x
a
m
p
l
e
11
E
v
a
l
u
a
t
i
o
n
Methodology
•
For software simulation, we use both bmv2 and NS3
•
For hardware implementation, we compile P4 in Barefoot Tofino switch
•
For sketch in the data plane, we keep 1MB memory and 1 hash function
Experiments
•
Gray failure detection
•
Congestion control
•
Path tracing
•
Latency measurement
•
Hardware resource usage
12
G
r
a
y
F
a
i
l
u
r
e
D
e
t
e
c
t
i
o
n
Tracked states
•
8-bit device ID, 8-bit ingress port, 8-bit egress port, and 32-bit latency
Bandwidth usage
•
D
e
l
t
a
I
N
T
(
8
.
1
b
i
t
s
)
m
i
t
i
g
a
t
e
s
9
3
%
b
a
n
d
w
i
d
t
h
u
s
a
g
e
o
f
I
N
T
-
P
a
t
h
(
1
1
2
b
i
t
s
)
•
Reason: DeltaINT only embeds critical states with non-negligible deltas
13
(a) Different epoch lengths
(b) Different thresholds
C
o
n
g
e
s
t
i
o
n
C
o
n
t
r
o
l
14
(a) Web search workload
(b) Hadoop workload
P
a
t
h
T
r
a
c
i
n
g
15
(a) Kentucky Datalink
(b) Fat Tree
P
a
t
h
T
r
a
c
i
n
g
Convergence
•
Average number of required packets:
DeltaINT (1) vs.
PINT (120)
•
Tail (99
th
percentile) number of required packets:
DeltaINT (1) vs.
PINT (350)
•
Reason
•
DeltaINT only embeds per-node device ID in the first packet of each flow
•
PINT needs sufficient sampled packets to retrieve per-flow device IDs
16
L
a
t
e
n
c
y
M
e
a
s
u
r
e
m
e
n
t
Tracked state: 8-bit latency
Bandwidth usage
•
Web search workload: DeltaINT (2.6 bits) is better than PINT (10.3 bits)
•
Hadoop workload:
DeltaINT (2.4 bits) is better than PINT (9.9 bits)
•
Reason: DeltaINT only embeds critical latency with non-negligible delta
17
(a) Web search workload
(b) Hadoop workload
H
a
r
d
w
a
r
e
R
e
s
o
u
r
c
e
U
s
a
g
e
Hardware resource usage
•
Percentages in brackets are fractions of total resource usage
•
DeltaINT incurs slightly more SRAM, stages, and stateful ALUs
•
DeltaINT needs to track embedded states in the data plane
•
INT incurs more PHV sizes and actions
•
INT has larger bandwidth overhead and hence more information to process and transmit
18
C
o
n
c
l
u
s
i
o
n
DeltaINT, a novel INT framework to achieve extremely low
bandwidth overhead
•
Generality
•
Convergence
Evaluation on various applications
•
DeltaINT incurs less bandwidth usage than state-of-the-art methods
Source code:
•
h
t
t
p
:
/
/
a
d
s
l
a
b
.
c
s
e
.
c
u
h
k
.
e
d
u
.
h
k
/
s
o
f
t
w
a
r
e
/
d
e
l
t
a
i
n
t
19
T
h
a
n
k
Y
o
u
!
Q
&
A
20
(prepare laser pointer)
Hi, I am Siyuan Sheng from The Chinese University of Hong Kong.
I am going to present the work “DeltaINT: Toward General In-band
Network Telemetry with Extremely Low Bandwidth Overhead”.
This is a joint work with Qun Huang and Patrick P. C. Lee.
This paper discusses DeltaINT, a novel framework for in-band network telemetry aimed at reducing bandwidth overhead while ensuring high generality and convergence. It addresses the limitations of existing methods by providing theoretical analysis on bandwidth mitigation guarantees and offering software simulation for various applications. DeltaINT shows promising results in reducing bandwidth costs in scenarios like gray failure detection, with potential hardware implementation using P4 technology. The paper introduces four families of applications for per-packet monitoring and aggregation, catering to different network telemetry tasks.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
DeltaINT: Toward General In-band Network Telemetry with Extremely Low Bandwidth Overhead Siyuan Sheng1,3, Qun Huang2, and Patrick P. C. Lee3, 1University of Chinese Academy and Sciences 2Peking University 3The Chinese University of Hong Kong 1
In-band Network Telemetry (INT) Source pushes control information and device-internal states Transit pushes states according to control information Sink extracts INT information and reports an event INT framework 2
Limitations of INT Significant bandwidth overhead Linearly grow with the length of forwarding path Reduce effective bandwidth for network applications Increase likelihood of IP-level fragmentation Example 5-node fat-tree topology in data center Trace device ID, ingress port, and egress port, of 4B each 12B per-node states and 8B INT control information 68B in total at least 4.53% of 1,500B MTU in Ethernet 3
Existing Studies Sampling-based methods Embed INT information to only a subset of sampled packets Reduce bandwidth overhead yet with slow convergence Cannot retrieve INT information unless collecting sufficient packets Other methods Designed for specific telemetry tasks All existing methods suffer from lowgenerality Cannot support all families of common applications 4
Our Contributions DeltaINT, a general INT framework Extremely low bandwidth overhead High generality and convergence Theoretical analysis on bandwidth mitigation guarantees Software simulation for various applications For example, reducing up to 93% bandwidth cost in gray failure detection P4-based hardware implementation Open-source DeltaINT prototype 5
Four Families of Applications Per-packet-per-node monitoring Collect per-node states for each packet (e.g., gray failure detection) Per-packet aggregation Aggregate per-node states for each packet (e.g., congestion control) Static per-flow aggregation Collect static per-node states for each flow (e.g., path tracing) Dynamic per-flow aggregation Aggregate per-node states for each flow (e.g., latency measurement) 6
Our Solution Key observation Delta, the change between current state and embedded state Delta is often negligible at most time in typical applications For example, relatively stable hop latency and static device IDs Motivating example 7
Per-node Architecture in DeltaINT Per-node architecture Calculate the delta between current states and embedded states Only if the delta exceeds a threshold, we insert current states into a packet and update the embedded state How to maintain embedded states efficiently in data plane? 8
Sketching in DeltaINT Sketch-based technique Store approximate information with limited memory and computations Track embedded states in the data plane with limited resources Per-node sketch data structure Each bucket stores a flowkey and the embedded states Each entry of a packet includes a bitmap and the states being embedded 9
Primitives in DeltaINT Four primitives to form DeltaINT workflow StateLoad Hash flowkey and load embedded states from the first bucket matching flowkey DeltaCalc Calculate the delta and compare with the predefined threshold StateUpdate Update flowkey and relevant embedded states in the hashed buckets MetadataInsert Insert a bitmap and the states with non-negligible deltas into the packet Fit DeltaINT into applications with slight changes to primitives 10
Update Example Receive the first packet of ?1 Receive the first packet of ?2 Receive the second packet of ?1 Receive the second packet of ?2 11
Evaluation Methodology For software simulation, we use both bmv2 and NS3 For hardware implementation, we compile P4 in Barefoot Tofino switch For sketch in the data plane, we keep 1MB memory and 1 hash function Experiments Gray failure detection Congestion control Path tracing Latency measurement Hardware resource usage 12
Gray Failure Detection Tracked states 8-bit device ID, 8-bit ingress port, 8-bit egress port, and 32-bit latency Bandwidth usage DeltaINT (8.1 bits) mitigates 93% bandwidth usage of INT-Path (112 bits) Reason: DeltaINT only embeds critical states with non-negligible deltas (b) Different thresholds (a) Different epoch lengths 13
Congestion Control Tracked state: 8-bit link utilization Bandwidth usage DeltaINT ( 1 bit) is better than PINT (8 bits) Reason: DeltaINT only needs a 1-bit bitmap for negligible delta such that controller can be aware of the stable link utilization (b) Hadoop workload (a) Web search workload 14
Path Tracing Tracked state: 8-bit device ID Bandwidth usage DeltaINT ( 1 bit) is better than PINT (8 bits) Reason: DeltaINT only needs a 1-bit bitmap for non-first packets of each flow due to static device ID with negligible delta (a) Kentucky Datalink (b) Fat Tree 15
Path Tracing Convergence Average number of required packets: DeltaINT (1) vs. PINT (120) Tail (99th percentile) number of required packets: DeltaINT (1) vs. PINT (350) Reason DeltaINT only embeds per-node device ID in the first packet of each flow PINT needs sufficient sampled packets to retrieve per-flow device IDs 16
Latency Measurement Tracked state: 8-bit latency Bandwidth usage Web search workload: DeltaINT (2.6 bits) is better than PINT (10.3 bits) Hadoop workload: DeltaINT (2.4 bits) is better than PINT (9.9 bits) Reason: DeltaINT only embeds critical latency with non-negligible delta (a) Web search workload (b) Hadoop workload 17
Hardware Resource Usage Hardware resource usage Percentages in brackets are fractions of total resource usage DeltaINT incurs slightly more SRAM, stages, and stateful ALUs DeltaINT needs to track embedded states in the data plane INT incurs more PHV sizes and actions INT has larger bandwidth overhead and hence more information to process and transmit 18
Conclusion DeltaINT, a novel INT framework to achieve extremely low bandwidth overhead Generality Convergence Evaluation on various applications DeltaINT incurs less bandwidth usage than state-of-the-art methods Source code: http://adslab.cse.cuhk.edu.hk/software/deltaint 19
Thank You! Q & A 20
