Memory Snapshotting by Vinod Ganapathy
H
o
w
t
o
s
n
a
p
s
h
o
t
m
e
m
o
r
y
V
i
n
o
d
G
a
n
a
p
a
t
h
y
vg@iisc.ac.in
s
e
c
u
r
e
l
y
T
h
e
l
a
y
e
r
-
b
e
l
o
w
p
r
i
n
c
i
p
l
e
2
U
s
e
r
a
p
p
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
Syscalls
Process
List
Kernel
Code
IDT
…
U
s
e
r
a
p
p
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
ls
,
ps
, &
bash
utilities
libc
gcc
…
…
T
h
e
l
o
w
e
r
y
o
u
g
o
,
t
h
e
m
o
r
e
c
o
n
t
r
o
l
y
o
u
h
a
v
e
Least
control
Most
control
Vinod Ganapathy - EECS Symposium
3
U
s
e
r
a
p
p
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
Example: Malware detection
Vinod Ganapathy - EECS Symposium
4
U
s
e
r
a
p
p
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
M
a
l
w
a
r
e
d
e
t
e
c
t
o
r
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
Example: Malware detection
Vinod Ganapathy - EECS Symposium
5
U
s
e
r
a
p
p
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
M
a
l
w
a
r
e
d
e
t
e
c
t
o
r
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
Example: Malware detection
cat
ps
ls
…
T
C
B
T
r
u
s
t
e
d
L
a
y
e
r
Vinod Ganapathy - EECS Symposium
6
U
s
e
r
a
p
p
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
M
a
l
w
a
r
e
d
e
t
e
c
t
o
r
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
Libraries may be compromised!
cat
ps
ls
Vinod Ganapathy - EECS Symposium
7
U
s
e
r
a
p
p
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
M
a
l
w
a
r
e
d
e
t
e
c
t
o
r
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
Solution: Query the OS
S
y
s
t
e
m
c
a
l
l
A
P
I
1
2
3
Query with syscall
OS reads file
Returns true
file content
3
T
C
B
Vinod Ganapathy - EECS Symposium
1
2
8
U
s
e
r
a
p
p
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
M
a
l
w
a
r
e
d
e
t
e
c
t
o
r
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
What if the OS is malicious?
S
y
s
t
e
m
c
a
l
l
A
P
I
Vinod Ganapathy - EECS Symposium
How does an OS get infected?
•
E
x
p
l
o
i
t
s
o
f
k
e
r
n
e
l
v
u
l
n
e
r
a
b
i
l
i
t
i
e
s
:
–
Injecting malicious code by exploiting a memory
error in the kernel
•
P
r
i
v
i
l
e
g
e
e
s
c
a
l
a
t
i
o
n
a
t
t
a
c
k
s
:
–
Exploit a
root
process and use resulting
administrative privileges to update the kernel
•
S
o
c
i
a
l
e
n
g
i
n
e
e
r
i
n
g
a
t
t
a
c
k
s
:
–
Trick user into installing fake kernel updates
•
Defeated via signature verification of kernel updates
•
Trivial to perform prior to the Windows Vista OS
Vinod Ganapathy - EECS Symposium
9
How prevalent are OS infections?
•
2
0
1
0
M
i
c
r
o
s
o
f
t
r
e
p
o
r
t
:
7
%
o
f
a
l
l
i
n
f
e
c
t
i
o
n
s
f
r
o
m
c
l
i
e
n
t
m
a
c
h
i
n
e
s
d
u
e
t
o
r
o
o
t
k
i
t
s
[
1
]
•
2
0
1
6
H
u
m
m
i
n
g
B
a
d
A
n
d
r
o
i
d
r
o
o
t
k
i
t
:
[
2
]
–
Up to 85 million Android devices infected?
–
Earns malware authors $300,000 each week
through fraudulent mobile advertisements
•
Used in many high-profile incidents:
–
Torpig and Storm botnets
–
Sony BMG (2005), Greek wiretapping (2004/5)
10
Vinod Ganapathy - EECS Symposium
11
U
s
e
r
a
p
p
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
M
a
l
w
a
r
e
d
e
t
e
c
t
o
r
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
How can we detect OS infection?
S
y
s
t
e
m
c
a
l
l
A
P
I
A
s
k
f
o
r
h
e
l
p
f
r
o
m
t
h
e
l
a
y
e
r
s
b
e
l
o
w
T
C
B
Vinod Ganapathy - EECS Symposium
12
U
s
e
r
a
p
p
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
M
a
l
w
a
r
e
d
e
t
e
c
t
o
r
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
How low can we go?
[
B
l
u
e
p
i
l
l
,
S
u
b
v
e
r
t
]
T
C
B
Vinod Ganapathy - EECS Symposium
13
U
s
e
r
a
p
p
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
M
a
l
w
a
r
e
d
e
t
e
c
t
o
r
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
How low can we go?
[
S
t
u
x
n
e
t
,
T
r
o
j
a
n
e
d
I
C
s
]
?
?
?
T
C
B
Vinod Ganapathy - EECS Symposium
Today’s talk
•
M
e
m
o
r
y
f
o
r
e
n
s
i
c
s
:
U
s
i
n
g
m
e
m
o
r
y
s
n
a
p
s
h
o
t
s
t
o
d
e
t
e
c
t
s
y
s
t
e
m
i
n
f
e
c
t
i
o
n
.
Vinod Ganapathy - EECS Symposium
14
Volatility Foundation
Modus operandi
Vinod Ganapathy - EECS Symposium
15
U
s
e
r
a
p
p
P
h
y
s
i
c
a
l
M
e
m
o
r
y
H
a
r
d
w
a
r
e
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
Syscall
Process
List
Kernel
Code
U
s
e
r
a
p
p
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
…
T
a
r
g
e
t
m
a
c
h
i
n
e
P
o
t
e
n
t
i
a
l
l
y
r
o
o
t
k
i
t
-
i
n
f
e
c
t
e
d
T
C
B
Modus operandi
Vinod Ganapathy - EECS Symposium
16
U
s
e
r
a
p
p
P
h
y
s
i
c
a
l
M
e
m
o
r
y
H
a
r
d
w
a
r
e
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
Syscall
Process
List
Kernel
Code
U
s
e
r
a
p
p
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
…
T
a
r
g
e
t
m
a
c
h
i
n
e
P
o
t
e
n
t
i
a
l
l
y
r
o
o
t
k
i
t
-
i
n
f
e
c
t
e
d
A
n
a
l
y
s
i
s
m
a
c
h
i
n
e
T
r
u
s
t
e
d
T
C
B
Modus operandi
Vinod Ganapathy - EECS Symposium
17
U
s
e
r
a
p
p
P
h
y
s
i
c
a
l
M
e
m
o
r
y
H
a
r
d
w
a
r
e
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
Syscall
Process
List
Kernel
Code
U
s
e
r
a
p
p
U
t
i
l
i
t
i
e
s
&
L
i
b
r
a
r
i
e
s
…
T
a
r
g
e
t
m
a
c
h
i
n
e
P
o
t
e
n
t
i
a
l
l
y
r
o
o
t
k
i
t
-
i
n
f
e
c
t
e
d
A
n
a
l
y
s
i
s
m
a
c
h
i
n
e
T
r
u
s
t
e
d
T
C
B
S
n
a
p
s
h
o
t
o
f
m
e
m
o
r
y
p
a
g
e
s
…
Research questions
•
How do we analyze memory snapshots?
–
Large body of work (~15 years) on this topic.
•
How can we snapshot memory pages without
involving the target’s OS?
–
F
o
c
u
s
o
f
t
h
i
s
t
a
l
k
!
Vinod Ganapathy - EECS Symposium
18
Vinod Ganapathy - EECS Symposium
19
Snapshot acquisition mechanism
1
2
3
Vinod Ganapathy - EECS Symposium
20
Tamper resistance
T
a
r
g
e
t
s
h
o
u
l
d
n
o
t
i
n
t
e
r
f
e
r
e
w
i
t
h
s
n
a
p
s
h
o
t
a
c
q
u
i
s
i
t
i
o
n
Vinod Ganapathy - EECS Symposium
21
Tamper resistance
T
a
r
g
e
t
s
h
o
u
l
d
n
o
t
i
n
t
e
r
f
e
r
e
w
i
t
h
s
n
a
p
s
h
o
t
a
c
q
u
i
s
i
t
i
o
n
P
h
y
s
i
c
a
l
M
e
m
o
r
y
V
i
r
t
u
a
l
H
a
r
d
w
a
r
e
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
H
y
p
e
r
v
i
s
o
r
•
Hypervisor can fetch memory
from virtual machine without
OS involvement
Vinod Ganapathy - EECS Symposium
22
Tamper resistance
T
a
r
g
e
t
s
h
o
u
l
d
n
o
t
i
n
t
e
r
f
e
r
e
w
i
t
h
s
n
a
p
s
h
o
t
a
c
q
u
i
s
i
t
i
o
n
P
h
y
s
i
c
a
l
M
e
m
o
r
y
H
a
r
d
w
a
r
e
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
•
Co-processor uses DMA
•
OS on target involved in
DMA setup
•
Malicious OS can hide
portions of memory with
malicious content
Vinod Ganapathy - EECS Symposium
23
Performance isolation
D
o
n
o
t
h
a
l
t
t
h
e
t
a
r
g
e
t
d
u
r
i
n
g
s
n
a
p
s
h
o
t
a
c
q
u
i
s
i
t
i
o
n
•
Necessary for situations where frequent snapshot
acquisition is necessary
•
Hypervisor-based acquisition requires pausing
the virtual machine
•
Co-processor can operate in concert with target
Snapshot consistency
24
Consistent snapshot
Inconsistent snapshot
Vinod Ganapathy - EECS Symposium
25
Snapshot consistency
S
n
a
p
s
h
o
t
s
h
o
u
l
d
f
a
i
t
h
f
u
l
l
y
r
e
p
r
e
s
e
n
t
t
a
r
g
e
t
’
s
s
t
a
t
e
a
t
a
g
i
v
e
n
i
n
s
t
a
n
t
i
n
t
i
m
e
P
h
y
s
i
c
a
l
M
e
m
o
r
y
H
a
r
d
w
a
r
e
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
F
1
F
2
…
T
F
1
F
2
…
T
+
δ
N
U
L
L
C
O
N
S
I
S
T
E
N
T
C
O
N
S
I
S
T
E
N
T
Vinod Ganapathy - EECS Symposium
26
Snapshot consistency
S
n
a
p
s
h
o
t
s
h
o
u
l
d
f
a
i
t
h
f
u
l
l
y
r
e
p
r
e
s
e
n
t
t
a
r
g
e
t
’
s
s
t
a
t
e
a
t
a
g
i
v
e
n
i
n
s
t
a
n
t
i
n
t
i
m
e
P
h
y
s
i
c
a
l
M
e
m
o
r
y
H
a
r
d
w
a
r
e
O
p
e
r
a
t
i
n
g
S
y
s
t
e
m
F
1
F
2
…
T
T
+
δ
Co-processor cannot pause target.
Snapshot may contain pages
obtained at different instants in time
I
N
C
O
N
S
I
S
T
E
N
T
Vinod Ganapathy - EECS Symposium
27
Introducing SnipSnap
•
New hardware-based design for memory
snapshotting.
•
Proposed deployment? Via die-stacked
manufacturing.
Die-stacked chip
Vinod Ganapathy - EECS Symposium
28
Picture courtesy of AMD
C
P
U
a
n
d
M
e
m
o
r
y
c
o
n
t
r
o
l
l
e
r
O
n
-
c
h
i
p
m
e
m
o
r
y
(
h
i
g
h
-
s
p
e
e
d
)
Die-stacked chip
Vinod Ganapathy - EECS Symposium
29
Picture courtesy of AMD
C
P
U
a
n
d
M
e
m
o
r
y
c
o
n
t
r
o
l
l
e
r
O
n
-
c
h
i
p
m
e
m
o
r
y
(
h
i
g
h
-
s
p
e
e
d
)
M
e
m
o
r
y
b
u
s
T
r
a
d
i
t
i
o
n
a
l
(
o
f
f
-
c
h
i
p
)
D
R
A
M
m
e
m
o
r
y
Our use of die-stacking
•
On-chip DRAM treated as a page-granularity
cache of off-chip DRAM memory
–
Every address accessed by the CPU will result in
the page frame being fetched to on-chip DRAM
Vinod Ganapathy - EECS Symposium
30
C
a
c
h
e
o
f
o
f
f
-
c
h
i
p
D
R
A
M
m
e
m
o
r
y
O
f
f
-
c
h
i
p
D
R
A
M
O
n
-
c
h
i
p
D
R
A
M
M
e
m
o
r
y
c
o
n
t
r
o
l
l
e
r
C
r
y
p
t
o
l
o
g
i
c
C
P
U
M
e
m
o
r
y
b
u
s
Triggering snapshot acquisition
Vinod Ganapathy - EECS Symposium
31
O
n
-
c
h
i
p
D
R
A
M
M
e
m
o
r
y
c
o
n
t
r
o
l
l
e
r
C
r
y
p
t
o
l
o
g
i
c
C
P
U
T
r
i
g
g
e
r
=
D
e
v
i
c
e
t
h
a
t
c
o
m
m
u
n
i
c
a
t
e
s
t
o
t
h
e
C
P
U
t
o
e
n
t
e
r
s
n
a
p
s
h
o
t
a
c
q
u
i
s
i
t
i
o
n
m
o
d
e
:
Physical device attached to South/NorthBridge
that sends a non-maskable interrupt
NIC with Wake-on-LAN-like feature
O
f
f
-
c
h
i
p
D
R
A
M
M
e
m
o
r
y
b
u
s
Snapshot acquisition mode
Vinod Ganapathy - EECS Symposium
32
M
e
m
o
r
y
c
o
n
t
r
o
l
l
e
r
C
r
y
p
t
o
l
o
g
i
c
C
P
U
•
Memory controller splits on-chip DRAM into
two parts:
Cache of off-chip DRAM memory
Copy-on-Write (CoW) area
C
a
c
h
e
C
o
W
1
O
f
f
-
c
h
i
p
D
R
A
M
M
e
m
o
r
y
b
u
s
Snapshot acquisition mode
Vinod Ganapathy - EECS Symposium
33
M
e
m
o
r
y
c
o
n
t
r
o
l
l
e
r
C
r
y
p
t
o
l
o
g
i
c
C
P
U
•
Hardware brings one page frame of off-chip
DRAM at a time to on-chip DRAM cache
C
a
c
h
e
C
o
W
F
i
F
i
2
O
f
f
-
c
h
i
p
D
R
A
M
M
e
m
o
r
y
b
u
s
Snapshot acquisition mode
Vinod Ganapathy - EECS Symposium
34
M
e
m
o
r
y
c
o
n
t
r
o
l
l
e
r
C
r
y
p
t
o
l
o
g
i
c
C
P
U
•
Crypto logic digitally signs contents of page:
Random nonce used to prevent replay attacks
Same nonce used for all pages in snapshot
C
a
c
h
e
C
o
W
3
O
f
f
-
c
h
i
p
D
R
A
M
M
e
m
o
r
y
b
u
s
Snapshot acquisition mode
Vinod Ganapathy - EECS Symposium
35
M
e
m
o
r
y
c
o
n
t
r
o
l
l
e
r
C
r
y
p
t
o
l
o
g
i
c
C
P
U
•
Hardware instructs OS to write signed page
to external medium:
Even if OS is infected, it cannot cheat, since
integrity of page is protected by the hardware
C
a
c
h
e
C
o
W
D
i
s
k
4
O
f
f
-
c
h
i
p
D
R
A
M
M
e
m
o
r
y
b
u
s
•
CPU continues to execute concurrently:
I
f
i
t
w
r
i
t
e
s
t
o
p
a
g
e
F
j
t
h
a
t
h
a
s
n
o
t
y
e
t
b
e
e
n
c
o
p
i
e
d
M
e
m
o
r
y
c
o
n
t
r
o
l
l
e
r
m
a
k
e
s
a
c
o
p
y
o
f
t
h
e
o
r
i
g
i
n
a
l
p
a
g
e
i
n
t
h
e
C
o
p
y
-
o
n
-
W
r
i
t
e
a
r
e
a
W
h
e
n
h
a
r
d
w
a
r
e
r
e
a
d
y
t
o
s
n
a
p
s
h
o
t
F
j
,
c
o
p
y
c
r
e
a
t
e
d
f
r
o
m
C
o
p
y
-
o
n
-
W
r
i
t
e
a
r
e
a
Snapshot acquisition mode
Vinod Ganapathy - EECS Symposium
36
M
e
m
o
r
y
c
o
n
t
r
o
l
l
e
r
C
r
y
p
t
o
l
o
g
i
c
C
P
U
M
e
m
o
r
y
b
u
s
C
a
c
h
e
C
o
W
5
F
j
O
f
f
-
c
h
i
p
D
R
A
M
F
j
At conclusion of acquisition
•
C
o
n
s
i
s
t
e
n
t
s
n
a
p
s
h
o
t
o
f
o
f
f
-
c
h
i
p
m
e
m
o
r
y
a
t
i
n
s
t
a
n
t
w
h
e
n
a
c
q
u
i
s
i
t
i
o
n
w
a
s
i
n
i
t
i
a
t
e
d
•
S
n
a
p
s
h
o
t
i
s
t
a
m
p
e
r
-
r
e
s
i
s
t
a
n
t
e
v
e
n
t
o
a
c
o
r
r
u
p
t
e
d
O
S
•
O
b
t
a
i
n
e
d
w
i
t
h
o
u
t
p
a
u
s
i
n
g
t
a
r
g
e
t
m
a
c
h
i
n
e
Vinod Ganapathy - EECS Symposium
37
…
Security analysis
Malicious OS cannot:
C
o
r
r
u
p
t
p
a
g
e
s
i
n
s
n
a
p
s
h
o
t
:
I
n
t
e
g
r
i
t
y
H
i
d
e
p
a
g
e
s
f
r
o
m
s
n
a
p
s
h
o
t
:
C
o
m
p
l
e
t
e
n
e
s
s
R
e
p
l
a
y
o
l
d
s
n
a
p
s
h
o
t
:
F
r
e
s
h
n
e
s
s
“
C
l
e
a
n
”
i
t
s
e
l
f
d
u
r
i
n
g
s
n
a
p
s
h
o
t
a
c
q
u
i
s
i
t
i
o
n
b
e
c
a
u
s
e
C
o
p
y
-
o
n
-
W
r
i
t
e
s
t
o
r
e
s
o
r
i
g
i
n
a
l
p
a
g
e
:
E
x
t
e
r
n
a
l
c
o
n
t
r
o
l
Verified SnipSnap’s design using the TLA+
model checker [Lamport]
Vinod Ganapathy - EECS Symposium
38
…
Evaluation
•
Hardware emulator using a NUMA system.
•
Evaluated:
–
Memory-intensive workloads
•
c
a
n
n
e
a
l
,
d
e
d
u
p
,
m
e
m
c
a
c
h
e
d
,
g
r
a
p
h
5
0
0
,
m
c
f
,
c
f
a
r
1
0
,
m
n
i
s
t
–
Effectiveness of performance-isolation claim
–
Time to procure full snapshot of memory
39
Application performance with virtual
machine-based snapshotting
40
Application performance with
SnipSnap-based snapshotting
41
T
h
a
n
k
y
o
u
V
i
n
o
d
G
a
n
a
p
a
t
h
y
G
o
o
g
l
e
“
S
e
c
u
r
e
,
C
o
n
s
i
s
t
e
n
t
,
H
i
g
h
-
P
e
r
f
o
r
m
a
n
c
e
M
e
m
o
r
y
S
n
a
p
s
h
o
t
t
i
n
g
”
f
o
r
a
c
o
p
y
o
f
t
h
e
p
a
p
e
r
Explore the concept of memory snapshotting through images and insights shared by Vinod Ganapathy at the EECS Symposium. Delve into topics like malware detection, the layer-below principle, and the risks of OS infections due to exploits and social engineering attacks.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
How to snapshot memory Vinod Ganapathy vg@iisc.ac.in
The layer-below principle The lower you go, the more control you have Least control User app User app gcc Utilities & Libraries ls, ps, & bash utilities libc Syscalls IDT Operating System Kernel Code Process List Hardware Most control CPU Memory I/O devices Vinod Ganapathy - EECS Symposium 2
Example: Malware detection User app Utilities & Libraries Operating System Hardware Vinod Ganapathy - EECS Symposium 3
Example: Malware detection User app Malware detector Utilities & Libraries Operating System Hardware Vinod Ganapathy - EECS Symposium 4
Example: Malware detection User app Malware detector Trusted Layer Utilities & Libraries TCB ps ls cat Operating System Hardware Vinod Ganapathy - EECS Symposium 5
Libraries may be compromised! User app Malware detector Utilities & Libraries ps ls cat Operating System Hardware Vinod Ganapathy - EECS Symposium 6
Solution: Query the OS Query with syscall 1 2 OS reads file Returns true file content 3 User app Malware detector Utilities & Libraries 1 2 3 Operating System System call API TCB Hardware Vinod Ganapathy - EECS Symposium 7
What if the OS is malicious? User app Malware detector Utilities & Libraries Operating System System call API Hardware Vinod Ganapathy - EECS Symposium 8
How does an OS get infected? Exploits of kernel vulnerabilities: Injecting malicious code by exploiting a memory error in the kernel Privilege escalation attacks: Exploit a root process and use resulting administrative privileges to update the kernel Social engineering attacks: Trick user into installing fake kernel updates Defeated via signature verification of kernel updates Trivial to perform prior to the Windows Vista OS Vinod Ganapathy - EECS Symposium 9
How prevalent are OS infections? 2010 Microsoft report: 7% of all infections from client machines due to rootkits[1] 2016 HummingBad Android rootkit:[2] Up to 85 million Android devices infected? Earns malware authors $300,000 each week through fraudulent mobile advertisements Used in many high-profile incidents: Torpig and Storm botnets Sony BMG (2005), Greek wiretapping (2004/5) Microsoft Malware Protection Center, Some Observations on Rootkits, January 2010, https://blogs.technet.microsoft.com/mmpc/2010/01/07/some-observations-on-rootkits [1] CheckPoint Software, From HummingBad to Worse, July 2016, http://blog.checkpoint.com/wp-content/uploads/2016/07/HummingBad-Research-report_FINAL-62916.pdf Vinod Ganapathy - EECS Symposium [2] 10
How can we detect OS infection? Ask for help from the layers below User app Malware detector Utilities & Libraries Operating System System call API TCB Hypervisor (a.k.a. Virtual Machine Monitor) Hardware Vinod Ganapathy - EECS Symposium 11
How low can we go? User app Malware detector Utilities & Libraries Operating System [Bluepill, Subvert] Hypervisor Hardware TCB Vinod Ganapathy - EECS Symposium 12
How low can we go? User app Malware detector Utilities & Libraries Operating System Hardware ??? [Stuxnet, Trojaned ICs] TCB Vinod Ganapathy - EECS Symposium 13
Todays talk Memory forensics: Using memory snapshots to detect system infection. Volatility Foundation Vinod Ganapathy - EECS Symposium 14
Modus operandi Target machine Potentially rootkit-infected User app User app Utilities & Libraries Syscall Operating System Kernel Code Process List Hardware Physical Memory TCB Vinod Ganapathy - EECS Symposium 15
Modus operandi Target machine Potentially rootkit-infected Analysis machine Trusted User app User app Utilities & Libraries Syscall Operating System Kernel Code Process List Hardware Physical Memory TCB Vinod Ganapathy - EECS Symposium 16
Modus operandi Target machine Potentially rootkit-infected Analysis machine Trusted User app User app Utilities & Libraries Syscall Operating System Kernel Code Process List Snapshot of memory pages Hardware Physical Memory TCB Vinod Ganapathy - EECS Symposium 17
Research questions How do we analyze memory snapshots? Large body of work (~15 years) on this topic. How can we snapshot memory pages without involving the target s OS? Focus of this talk! Vinod Ganapathy - EECS Symposium 18
Snapshot acquisition mechanism Tamper resistance Performance isolation Snapshot consistency 1 2 3 Vinod Ganapathy - EECS Symposium 19
Tamper resistance Tamper resistance Performance isolation Snapshot consistency Target should not interfere with snapshot acquisition Vinod Ganapathy - EECS Symposium 20
Tamper resistance Tamper resistance Performance isolation Snapshot consistency Virtualization Target should not interfere with snapshot acquisition Hypervisor can fetch memory from virtual machine without OS involvement Operating System Virtual Hardware Physical Memory Hypervisor Vinod Ganapathy - EECS Symposium 21
Tamper resistance Tamper resistance Performance isolation Snapshot consistency Virtualization Co-processor Target should not interfere with snapshot acquisition Co-processor uses DMA OS on target involved in DMA setup Malicious OS can hide portions of memory with malicious content Operating System Hardware Physical Memory Vinod Ganapathy - EECS Symposium 22
Performance isolation Tamper resistance Performance isolation Snapshot consistency Virtualization Co-processor Do not halt the target during snapshot acquisition Necessary for situations where frequent snapshot acquisition is necessary Hypervisor-based acquisition requires pausing the virtual machine Co-processor can operate in concert with target Vinod Ganapathy - EECS Symposium 23
Snapshot consistency Inconsistent snapshot Consistent snapshot 24
Snapshot consistency Tamper resistance Performance isolation Snapshot consistency Virtualization Co-processor Snapshot should faithfully represent target s state at a given instant in time Operating System CONSISTENT F2 F1 T Hardware CONSISTENT Physical Memory NULL T + F2 F1 Vinod Ganapathy - EECS Symposium 25
Snapshot consistency Tamper resistance Performance isolation Snapshot consistency Virtualization Co-processor Snapshot should faithfully represent target s state at a given instant in time Operating System INCONSISTENT F2 T + F1 T Hardware Co-processor cannot pause target. Snapshot may contain pages obtained at different instants in time Physical Memory Vinod Ganapathy - EECS Symposium 26
Introducing SnipSnap Tamper resistance Performance isolation Snapshot consistency Virtualization Co-processor SnipSnap New hardware-based design for memory snapshotting. Proposed deployment? Via die-stacked manufacturing. Vinod Ganapathy - EECS Symposium 27
Die-stacked chip On-chip memory (high-speed) CPU and Memory controller Picture courtesy of AMD Vinod Ganapathy - EECS Symposium 28
Die-stacked chip Traditional (off-chip) DRAM memory On-chip memory (high-speed) Memory bus CPU and Memory controller Picture courtesy of AMD Vinod Ganapathy - EECS Symposium 29
Our use of die-stacking Cache of off-chip DRAM memory Off-chip DRAM On-chip DRAM Memory bus Memory controller Crypto logic CPU On-chip DRAM treated as a page-granularity cache of off-chip DRAM memory Every address accessed by the CPU will result in the page frame being fetched to on-chip DRAM Vinod Ganapathy - EECS Symposium 30
Triggering snapshot acquisition Off-chip DRAM On-chip DRAM Memory bus Memory controller Crypto logic CPU Trigger = Device that communicates to the CPU to enter snapshot acquisition mode: Physical device attached to South/NorthBridge that sends a non-maskable interrupt NIC with Wake-on-LAN-like feature Vinod Ganapathy - EECS Symposium 31
Snapshot acquisition mode 1 Off-chip DRAM CoW Cache Memory bus Memory controller Crypto logic CPU Memory controller splits on-chip DRAM into two parts: Cache of off-chip DRAM memory Copy-on-Write (CoW) area Vinod Ganapathy - EECS Symposium 32
Snapshot acquisition mode 2 Off-chip DRAM CoW Cache Fi Fi Memory bus Memory controller Crypto logic CPU Hardware brings one page frame of off-chip DRAM at a time to on-chip DRAM cache Vinod Ganapathy - EECS Symposium 33
Snapshot acquisition mode 3 Off-chip DRAM + Page# + Rand# Fi CoW Cache Memory bus Memory controller Crypto logic CPU Crypto logic digitally signs contents of page: Random nonce used to prevent replay attacks Same nonce used for all pages in snapshot Vinod Ganapathy - EECS Symposium 34
Snapshot acquisition mode 4 Off-chip DRAM + Page# + Rand# Fi CoW Cache Memory bus Memory controller Crypto logic CPU Disk Hardware instructs OS to write signed page to external medium: Even if OS is infected, it cannot cheat, since integrity of page is protected by the hardware Vinod Ganapathy - EECS Symposium 35
Snapshot acquisition mode 5 Off-chip DRAM CoW Cache Fj Fj Memory bus Memory controller Crypto logic CPU CPU continues to execute concurrently: If it writes to page Fj that has not yet been copied Memory controller makes a copy of the original page in the Copy-on-Write area When hardware ready to snapshot Fj, copy created from Copy-on-Write area Vinod Ganapathy - EECS Symposium 36
At conclusion of acquisition + 1 + R + N + R + 0 + R F1 FN F0 Consistent snapshot of off-chip memory at instant when acquisition was initiated Snapshot is tamper-resistant even to a corrupted OS Obtained without pausing target machine Vinod Ganapathy - EECS Symposium 37
Security analysis + 1 + R + N + R + 0 + R F1 FN F0 Malicious OS cannot: Corrupt pages in snapshot: Integrity Hide pages from snapshot: Completeness Replay old snapshot: Freshness Clean itself during snapshot acquisition because Copy-on-Write stores original page: External control Verified SnipSnap s design using the TLA+ model checker [Lamport] Vinod Ganapathy - EECS Symposium 38
Evaluation Hardware emulator using a NUMA system. Evaluated: Memory-intensive workloads canneal, dedup, memcached, graph500, mcf, cfar10, mnist Effectiveness of performance-isolation claim Time to procure full snapshot of memory 39
Application performance with virtual machine-based snapshotting 3 Normalized performance degradation 2.5 2 1.5 1 512M 2G 8GB DRAM size net-100 net-10 ssd 40
Application performance with SnipSnap-based snapshotting 2.8 Normalized performance degradation net-100 net-10 ssd 2.6 2.4 2.2 2 1.8 1.6 1.4 1.2 1 512M 2G 8GB 512M 2G 8GB Virtual machine SnipSnap DRAM sizes 41
Thank you Vinod Ganapathy Google Secure, Consistent, High-Performance Memory Snapshotting for a copy of the paper
