Evaluating Website Fingerprinting Attacks on Tor
O
n
l
i
n
e
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
:
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
Giovanni Cherubin,
Alan Turing Institute
Microsoft Research
Rob Jansen
, U.S. Naval Research Laboratory
Carmela Troncoso, EPFL SPRING Lab
R
o
b
J
a
n
s
e
n
,
P
h
.
D
.
Computer Security Research Scientist
Center for High Assurance Computer Systems
U.S. Naval Research Laboratory
3
1
s
t
U
S
E
N
I
X
S
e
c
u
r
i
t
y
S
y
m
p
o
s
i
u
m
B
o
s
t
o
n
,
M
A
,
U
S
A
August 10
th
, 2022
D
i
s
t
i
n
g
u
i
s
h
e
d
P
a
p
e
r
A
w
a
r
d
,
I
n
t
e
r
n
e
t
D
e
f
e
n
s
e
2
n
d
P
r
i
z
e
!
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
2
U.S. Naval Research Laboratory
H
o
w
T
o
r
W
o
r
k
s
Anonymous Communication and Tor
•
Separates identification from routing
•
Provides unlinkable communication
•
Promotes user safety and privacy online
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
3
U.S. Naval Research Laboratory
D
e
a
n
o
n
y
m
i
z
i
n
g
T
o
r
U
s
e
r
s
Website fingerprinting attack
•
Predict website visited by user
•
Requires access to
entry side only
predict website
?
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
4
U.S. Naval Research Laboratory
D
e
a
n
o
n
y
m
i
z
i
n
g
T
o
r
U
s
e
r
s
predict website
?
Problem:
•
Need
labels
to train ML classifiers
for website prediction
•
Genuine labels are
encrypted
Website fingerprinting attack
•
Predict website visited by user
•
Requires access to
entry side only
Encrypted by Tor
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
5
U.S. Naval Research Laboratory
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
T
h
r
e
a
t
M
o
d
e
l
Step 1: gather data & labels
•
Use automated browser
(selenium) to crawl websites
xyz.com
xyz.com
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
6
U.S. Naval Research Laboratory
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
T
h
r
e
a
t
M
o
d
e
l
Step 1: gather data & labels
•
Use automated browser
(selenium) to crawl websites
Step 2: train ML classifier
•
Use collected data & labels
xyz.com
xyz.com
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
7
U.S. Naval Research Laboratory
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
T
h
r
e
a
t
M
o
d
e
l
Step 1: gather data & labels
•
Use automated browser
(selenium) to crawl websites
Step 2: train ML classifier
•
Use collected data & labels
Step 3: deploy against users
•
Use ML model to predict website
cats.com
?
xyz.com
xyz.com
cats.com
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
8
U.S. Naval Research Laboratory
C
r
i
t
i
c
i
s
m
s
o
f
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
T
h
r
e
a
t
M
o
d
e
l
Step 1: gather data & labels
•
Use automated browser
(selenium) to crawl websites
Step 2: train ML classifier
•
Use collected data & labels
Step 3: deploy against users
•
Use ML model to predict website
?
xyz.com
Synthetic
too simple & unrealistic!
•
browser
: version, config, location
•
behavior
: URL choice, tabs, order
•
world
: static, small, closed
cats.com
xyz.com
cats.com
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
9
U.S. Naval Research Laboratory
W
h
a
t
i
s
t
h
e
t
h
r
e
a
t
o
f
W
F
a
t
t
a
c
k
s
i
n
t
h
e
r
e
a
l
w
o
r
l
d
?
Synthetic model
•
Overly
simple
and
unrealistic
•
High ML accuracy
in simple model
Stop using!!
Genuine model
•
Consider
genuine
data & labels from a
Tor exit relay
Our new approach
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
10
10
U.S. Naval Research Laboratory
K
e
y
I
n
s
i
g
h
t
:
E
x
i
t
s
O
b
s
e
r
v
e
G
e
n
u
i
n
e
D
a
t
a
&
L
a
b
e
l
s
Step 1: gather data & labels
•
Run a Tor exit relay and use to
to collect genuine Tor traffic
cats.com
Genuine labels: resolved domains
Genuine data: circuit traffic patterns
DNS
cats.com?
198.71.232.3
198.71.232.3
Exit can observe:
1.
New circuit
2.
DNS lookup
3.
Website load
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
11
11
U.S. Naval Research Laboratory
K
e
y
I
n
s
i
g
h
t
:
E
x
i
t
s
O
b
s
e
r
v
e
G
e
n
u
i
n
e
D
a
t
a
&
L
a
b
e
l
s
Step 1: gather data & labels
•
Run a Tor exit relay and use to
to collect genuine Tor traffic
cats.com
Genuine labels: resolved domains
Genuine data: circuit traffic patterns
DNS
cats.com?
198.71.232.3
198.71.232.3
Step 2: train ML classifier
•
Use collected data & labels
Exit can observe:
1.
New circuit
2.
DNS lookup
3.
Website load
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
12
12
U.S. Naval Research Laboratory
K
e
y
I
n
s
i
g
h
t
:
E
x
i
t
s
O
b
s
e
r
v
e
G
e
n
u
i
n
e
D
a
t
a
&
L
a
b
e
l
s
Step 1: gather data & labels
•
Run a Tor exit relay and use to
to collect genuine Tor traffic
cats.com
Genuine labels: resolved domains
Genuine data: circuit traffic patterns
DNS
cats.com?
198.71.232.3
198.71.232.3
Step 2: train ML classifier
•
Use collected data & labels
?
cats.com
Step 3: deploy against users
•
Use ML model to predict website
Exit can observe:
1.
New circuit
2.
DNS lookup
3.
Website load
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
13
13
U.S. Naval Research Laboratory
K
e
y
I
n
s
i
g
h
t
:
E
x
i
t
s
O
b
s
e
r
v
e
G
e
n
u
i
n
e
D
a
t
a
&
L
a
b
e
l
s
cats.com
Genuine labels: resolved domains
Genuine data: circuit traffic patterns
DNS
cats.com?
198.71.232.3
198.71.232.3
?
cats.com
Exit can observe:
1.
New circuit
2.
DNS lookup
3.
Website load
Benefits
•
Captures real world
diversity of browsers,
behavior, world size,
choice of pages
•
Can stop trying to fix
the synthetic model
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
14
14
U.S. Naval Research Laboratory
K
e
y
I
n
s
i
g
h
t
:
E
x
i
t
s
O
b
s
e
r
v
e
G
e
n
u
i
n
e
D
a
t
a
&
L
a
b
e
l
s
cats.com
Genuine labels: resolved domains
Genuine data: circuit traffic patterns
Exit can observe:
1.
New circuit
2.
DNS lookup
3.
Website load
DNS
cats.com?
198.71.232.3
198.71.232.3
?
cats.com
Caveats
•
Train at exit, deploy at
entry
noise
•
Domain, not page label
•
Need safe eval methods
Benefits
•
Captures real world
diversity of browsers,
behavior, world size,
choice of pages
•
Can stop trying to fix
the synthetic model
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
15
15
U.S. Naval Research Laboratory
S
a
f
e
E
v
a
l
u
a
t
i
o
n
u
s
i
n
g
O
n
l
i
n
e
L
e
a
r
n
i
n
g
Our safe evaluation plan:
•
Hash domain labels using keyed HMAC
−
Never learn true labels
cats.com
DNS
cats.com?
198.71.232.3
data: (-1,+1,…)
label: HMAC(cats.com)
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
16
16
U.S. Naval Research Laboratory
S
a
f
e
E
v
a
l
u
a
t
i
o
n
u
s
i
n
g
O
n
l
i
n
e
L
e
a
r
n
i
n
g
Our safe evaluation plan:
•
Hash domain labels using keyed HMAC
−
Never learn true labels
•
Use online learning
−
Adapted Triplet Fingerprinting [CCS’19]
−
Compute means in real time, discard data
−
Individual data items never stored
cats.com
DNS
cats.com?
198.71.232.3
data: (-1,+1,…)
label: HMAC(cats.com)
HMAC(cats.com)
correct?
yes or no
triplet feature extractor
?
1. predict label
2. update k-nn mev
k-nn model
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
17
17
U.S. Naval Research Laboratory
S
a
f
e
E
v
a
l
u
a
t
i
o
n
u
s
i
n
g
O
n
l
i
n
e
L
e
a
r
n
i
n
g
Our safe evaluation plan:
•
Hash domain labels using keyed HMAC
−
Never learn true labels
•
Use online learning
−
Adapted Triplet Fingerprinting [CCS’19]
−
Compute means in real time, discard data
−
Individual data items never stored
•
Other safety precautions
−
Never deanonymizes Tor users
−
Destroyed models, HMAC key after eval
•
Tor Safety Board reviewed plan
−
See paper for details!
cats.com
DNS
cats.com?
198.71.232.3
data: (-1,+1,…)
label: HMAC(cats.com)
HMAC(cats.com)
correct?
yes or no
triplet feature extractor
?
1. predict label
2. update k-nn mev
k-nn model
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
18
18
U.S. Naval Research Laboratory
E
v
a
l
u
a
t
i
o
n
Train and evaluate at exit relay
•
No noise from transferring to entry
•
Upper bound on attack accuracy
Details
•
1 week evaluation
−
3.9M data sequences, 671k unique sites
•
Multi-class classification
−
predict a monitored site, or ‘unmonitored’
•
Performance metric
−
instant accuracy (i.e., moving average)
−
# correct / # total predictions (10k window)
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
19
19
U.S. Naval Research Laboratory
E
v
a
l
u
a
t
i
o
n
Train and evaluate at exit relay
•
No noise from transferring to entry
•
Upper bound on attack accuracy
Details
•
1 week evaluation
−
3.9M data sequences, 671k unique sites
•
Multi-class classification
−
predict a monitored site, or ‘unmonitored’
•
Performance metric
−
instant accuracy (i.e., moving average)
−
# correct / # total predictions (10k window)
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
20
20
U.S. Naval Research Laboratory
E
v
a
l
u
a
t
i
o
n
Train and evaluate at exit relay
•
No noise from transferring to entry
•
Upper bound on attack accuracy
Details
•
1 week evaluation
−
3.9M data sequences, 671k unique sites
•
Multi-class classification
−
predict a monitored site, or ‘unmonitored’
•
Performance metric
−
instant accuracy (i.e., moving average)
−
# correct / # total predictions (10k window)
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
21
21
U.S. Naval Research Laboratory
E
v
a
l
u
a
t
i
o
n
Train and evaluate at exit relay
•
No noise from transferring to entry
•
Upper bound on attack accuracy
Details
•
1 week evaluation
−
3.9M data sequences, 671k unique sites
•
Multi-class classification
−
predict a monitored site, or ‘unmonitored’
•
Performance metric
−
instant accuracy (i.e., moving average)
−
# correct / # total predictions (10k window)
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
22
22
U.S. Naval Research Laboratory
G
e
n
u
i
n
e
v
s
.
S
y
n
t
h
e
t
i
c
D
a
t
a
Offline phase
•
Crawl ‘synthetic’ list of domains
−
Synthetic
: use crawl to train a classifier offline
Online phase
•
Train two classifiers online
−
Hybrid
: update copy of synthetic classifier with
genuine data
−
Real
: train new classifier on genuine data only
•
1 week evaluation
−
1.2M data sequences
−
observed 183 of 1,074 ‘synthetic’ domains
•
Binary classification
−
monitored set contains 5 sites
−
predict either ‘monitored’ or ‘unmonitored’
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
23
23
U.S. Naval Research Laboratory
G
e
n
u
i
n
e
v
s
.
S
y
n
t
h
e
t
i
c
D
a
t
a
Offline phase
•
Crawl ‘synthetic’ list of domains
−
Synthetic
: use crawl to train a classifier offline
Online phase
•
Train two classifiers online
−
Hybrid
: update copy of synthetic classifier with
genuine data
−
Real
: train new classifier on genuine data only
•
1 week evaluation
−
1.2M data sequences
−
observed 183 of 1,074 ‘synthetic’ domains
•
Binary classification
−
monitored set contains 5 sites
−
predict either ‘monitored’ or ‘unmonitored’
synthetic classifier performs poorly
against genuine data
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
24
24
U.S. Naval Research Laboratory
G
e
n
u
i
n
e
v
s
.
S
y
n
t
h
e
t
i
c
D
a
t
a
Offline phase
•
Crawl ‘synthetic’ list of domains
−
Synthetic
: use crawl to train a classifier offline
Online phase
•
Train two classifiers online
−
Hybrid
: update copy of synthetic classifier with
genuine data
−
Real
: train new classifier on genuine data only
•
1 week evaluation
−
1.2M data sequences
−
observed 183 of 1,074 ‘synthetic’ domains
•
Binary classification
−
monitored set contains 5 sites
−
predict either ‘monitored’ or ‘unmonitored’
synthetic classifier performs poorly
against genuine data
synthetic data does not improve
model over genuine data
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
25
25
U.S. Naval Research Laboratory
T
r
a
i
n
i
n
g
a
n
d
T
e
s
t
i
n
g
o
n
O
p
p
o
s
i
t
e
E
n
d
s
loss in accuracy is low for feasible
(i.e. small) monitored sets
Fully synthetic evaluation
•
Crawled 1k URLs 10x each
•
Pinned entry and exit on each circuit
•
Collected data sequences in both
positions on each circuit
•
Closed-world batch classification
−
50%-50% train-test split
E
v
a
l
u
a
t
i
n
g
W
e
b
s
i
t
e
F
i
n
g
e
r
p
r
i
n
t
i
n
g
A
t
t
a
c
k
s
o
n
T
o
r
i
n
t
h
e
R
e
a
l
W
o
r
l
d
|
26
26
U.S. Naval Research Laboratory
M
a
i
n
T
a
k
e
a
w
a
y
s
Insights
•
WF can be feasible with genuine data and
small monitored sets, online learning can
mitigate concept drift
•
Synthetic data is not useful when the
adversary deploys in the real world
•
Simple defenses may be more effective
than we thought
−
Adversary has to simulate defense
on top of undefended exit data
Contact
−
rob.g.jansen@nrl.navy.mil
−
robgjansen.com
−
@robgjansen
Future Research Areas
•
Improve accuracy when training on
genuine data
•
Reduce distortion when transferring
models from exit to entry
•
Defenses that make it harder to learn
from genuine data, increase distortion
R
e
a
d
t
h
e
p
a
p
e
r
!
This research evaluates website fingerprinting attacks on the Tor network in the real world. It discusses the methodology of deanonymizing Tor users through predicting visited websites, emphasizing the need for labels to train machine learning classifiers. The study presents a threat model involving data gathering, label generation, and ML classifier training to carry out the attack successfully.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Distinguished Paper Award, Internet Defense 2nd Prize! Online Website Fingerprinting: Evaluating Website Fingerprinting Attacks on Tor in the Real World Giovanni Cherubin, Alan Turing Institute Microsoft Research Rob Jansen, U.S. Naval Research Laboratory Carmela Troncoso, EPFL SPRING Lab Rob Jansen, Ph.D. Computer Security Research Scientist Center for High Assurance Computer Systems U.S. Naval Research Laboratory 31st USENIX Security Symposium Boston, MA, USA August 10th, 2022
How Tor Works Anonymous Communication and Tor Separates identification from routing Provides unlinkable communication Promotes user safety and privacy online U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 2
Deanonymizing Tor Users Website fingerprinting attack Predict website visited by user Requires access to entry side only ? predict website U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 3
Deanonymizing Tor Users Problem: Need labels to train ML classifiers for website prediction Genuine labels are encrypted Website fingerprinting attack Predict website visited by user Requires access to entry side only ? predict website Encrypted by Tor U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 4
Website Fingerprinting Threat Model Step 1: gather data & labels Use automated browser (selenium) to crawl websites xyz.com xyz.com U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 5
Website Fingerprinting Threat Model Step 1: gather data & labels Use automated browser (selenium) to crawl websites xyz.com xyz.com Step 2: train ML classifier Use collected data & labels U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 6
Website Fingerprinting Threat Model Step 1: gather data & labels Use automated browser (selenium) to crawl websites xyz.com xyz.com Step 2: train ML classifier Use collected data & labels cats.com ? cats.com Step 3: deploy against users Use ML model to predict website U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 7
Criticisms of Website Fingerprinting Threat Model Synthetic too simple & unrealistic! browser: version, config, location behavior: URL choice, tabs, order world: static, small, closed Step 1: gather data & labels Use automated browser (selenium) to crawl websites xyz.com xyz.com Step 2: train ML classifier Use collected data & labels cats.com ? cats.com Step 3: deploy against users Use ML model to predict website U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 8
What is the threat of WF attacks in the real world? Synthetic model Overly simple and unrealistic High ML accuracy in simple model Genuine model Consider genuine data & labels from a Tor exit relay Our new approach Stop using!! U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 9
Key Insight: Exits Observe Genuine Data & Labels Step 1: gather data & labels Run a Tor exit relay and use to to collect genuine Tor traffic Exit can observe: 1. New circuit 2. DNS lookup 3. Website load Genuine labels: resolved domains Genuine data: circuit traffic patterns DNS cats.com 198.71.232.3 U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 10
Key Insight: Exits Observe Genuine Data & Labels Step 1: gather data & labels Run a Tor exit relay and use to to collect genuine Tor traffic Exit can observe: 1. New circuit 2. DNS lookup 3. Website load Genuine labels: resolved domains Genuine data: circuit traffic patterns Step 2: train ML classifier Use collected data & labels DNS cats.com 198.71.232.3 U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 11
Key Insight: Exits Observe Genuine Data & Labels Step 1: gather data & labels Run a Tor exit relay and use to to collect genuine Tor traffic Exit can observe: 1. New circuit 2. DNS lookup 3. Website load Genuine labels: resolved domains Genuine data: circuit traffic patterns Step 2: train ML classifier Use collected data & labels DNS cats.com ? cats.com 198.71.232.3 Step 3: deploy against users Use ML model to predict website U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 12
Key Insight: Exits Observe Genuine Data & Labels Benefits Captures real world diversity of browsers, behavior, world size, choice of pages Can stop trying to fix the synthetic model Exit can observe: 1. New circuit 2. DNS lookup 3. Website load Genuine labels: resolved domains Genuine data: circuit traffic patterns DNS cats.com ? cats.com 198.71.232.3 U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 13
Key Insight: Exits Observe Genuine Data & Labels Benefits Captures real world diversity of browsers, behavior, world size, choice of pages Can stop trying to fix the synthetic model Exit can observe: 1. New circuit 2. DNS lookup 3. Website load Genuine labels: resolved domains Genuine data: circuit traffic patterns DNS cats.com ? Caveats Train at exit, deploy at entry noise Domain, not page label Need safe eval methods cats.com 198.71.232.3 U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 14
Safe Evaluation using Online Learning Our safe evaluation plan: Hash domain labels using keyed HMAC Never learn true labels data: (-1,+1, ) label: HMAC(cats.com) DNS cats.com 198.71.232.3 U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 15
Safe Evaluation using Online Learning k-nn model Our safe evaluation plan: Hash domain labels using keyed HMAC Never learn true labels HMAC(cats.com) correct? yes or no 1. predict label ? 2. update k-nn mev Use online learning Adapted Triplet Fingerprinting [CCS 19] Compute means in real time, discard data Individual data items never stored data: (-1,+1, ) label: HMAC(cats.com) DNS triplet feature extractor cats.com 198.71.232.3 U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 16
Safe Evaluation using Online Learning k-nn model Our safe evaluation plan: Hash domain labels using keyed HMAC Never learn true labels HMAC(cats.com) correct? yes or no 1. predict label ? 2. update k-nn mev Use online learning Adapted Triplet Fingerprinting [CCS 19] Compute means in real time, discard data Individual data items never stored data: (-1,+1, ) label: HMAC(cats.com) DNS triplet feature extractor Other safety precautions Never deanonymizes Tor users Destroyed models, HMAC key after eval cats.com Tor Safety Board reviewed plan See paper for details! 198.71.232.3 U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 17
Evaluation Train and evaluate at exit relay No noise from transferring to entry Upper bound on attack accuracy Details 1 week evaluation 3.9M data sequences, 671k unique sites Multi-class classification predict a monitored site, or unmonitored Performance metric instant accuracy (i.e., moving average) # correct / # total predictions (10k window) U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 18
Evaluation Train and evaluate at exit relay No noise from transferring to entry Upper bound on attack accuracy Details 1 week evaluation 3.9M data sequences, 671k unique sites Multi-class classification predict a monitored site, or unmonitored Performance metric instant accuracy (i.e., moving average) # correct / # total predictions (10k window) U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 19
Evaluation accuracy above 95% when monitoring 5 sites Train and evaluate at exit relay No noise from transferring to entry Upper bound on attack accuracy Details 1 week evaluation 3.9M data sequences, 671k unique sites Multi-class classification predict a monitored site, or unmonitored Performance metric instant accuracy (i.e., moving average) # correct / # total predictions (10k window) U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 20
Evaluation accuracy above 95% when monitoring 5 sites Train and evaluate at exit relay No noise from transferring to entry Upper bound on attack accuracy Details 1 week evaluation 3.9M data sequences, 671k unique sites Multi-class classification predict a monitored site, or unmonitored Performance metric instant accuracy (i.e., moving average) # correct / # total predictions (10k window) accuracy quickly falls below 80% when monitoring 25 sites U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 21
Genuine vs. Synthetic Data Offline phase Crawl synthetic list of domains Synthetic: use crawl to train a classifier offline Online phase Train two classifiers online Hybrid: update copy of synthetic classifier with genuine data Real: train new classifier on genuine data only 1 week evaluation 1.2M data sequences observed 183 of 1,074 synthetic domains Binary classification monitored set contains 5 sites predict either monitored or unmonitored U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 22
Genuine vs. Synthetic Data Offline phase Crawl synthetic list of domains Synthetic: use crawl to train a classifier offline Online phase Train two classifiers online Hybrid: update copy of synthetic classifier with genuine data Real: train new classifier on genuine data only 1 week evaluation 1.2M data sequences observed 183 of 1,074 synthetic domains Binary classification monitored set contains 5 sites predict either monitored or unmonitored synthetic classifier performs poorly against genuine data U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 23
Genuine vs. Synthetic Data synthetic data does not improve model over genuine data Offline phase Crawl synthetic list of domains Synthetic: use crawl to train a classifier offline Online phase Train two classifiers online Hybrid: update copy of synthetic classifier with genuine data Real: train new classifier on genuine data only 1 week evaluation 1.2M data sequences observed 183 of 1,074 synthetic domains Binary classification monitored set contains 5 sites predict either monitored or unmonitored synthetic classifier performs poorly against genuine data U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 24
Training and Testing on Opposite Ends Fully synthetic evaluation Crawled 1k URLs 10x each Monitored set size: Train and test on exit Train on exit, test on entry Loss in accuracy: 5 50 750 52.2% 34.1% 18.1% 91.2% 76.2% 86.4% 65.1% 4.8% Pinned entry and exit on each circuit 11.1% Collected data sequences in both positions on each circuit Closed-world batch classification 50%-50% train-test split loss in accuracy is low for feasible (i.e. small) monitored sets U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 25
Main Takeaways Insights Future Research Areas Improve accuracy when training on genuine data WF can be feasible with genuine data and small monitored sets, online learning can mitigate concept drift Reduce distortion when transferring models from exit to entry Synthetic data is not useful when the adversary deploys in the real world Defenses that make it harder to learn from genuine data, increase distortion Simple defenses may be more effective than we thought Adversary has to simulate defense on top of undefended exit data Read the paper! Contact rob.g.jansen@nrl.navy.mil robgjansen.com @robgjansen U.S. Naval Research Laboratory Evaluating Website Fingerprinting Attacks on Tor in the Real World | 26
