Exploring Variability and Noise in Neural Networks
Biological Modeling of
Neural Networks
W
e
e
k
7
–
V
a
r
i
a
b
i
l
i
t
y
a
n
d
N
o
i
s
e
:
T
h
e
q
u
e
s
t
i
o
n
o
f
t
h
e
n
e
u
r
a
l
c
o
d
e
Wulfram Gerstner
EPFL, Lausanne, Switzerland
7
.
1
V
a
r
i
a
b
i
l
i
t
y
o
f
s
p
i
k
e
t
r
a
i
n
s
- experiments
7
.
2
S
o
u
r
c
e
s
o
f
V
a
r
i
a
b
i
l
i
t
y
?
-
Is variability equal to noise?
7
.
3
P
o
i
s
s
o
n
M
o
d
e
l
-
T
h
r
e
e
d
e
f
i
n
i
t
i
o
n
s
o
f
R
a
t
e
c
o
d
e
7
.
4
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
a
r
r
i
v
a
l
-
M
e
m
b
r
a
n
e
p
o
t
e
n
t
i
a
l
f
l
u
c
t
u
a
t
i
o
n
s
7
.
5
.
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
f
i
r
i
n
g
- stochastic integrate-and-fire
W
e
e
k
7
–
p
a
r
t
1
:
V
a
r
i
a
b
i
l
i
t
y
Neuronal Dynamics –
7.1. Variability
visual
cortex
motor
cortex
frontal
cortex
to motor
output
Crochet et al., 2011
awake mouse, cortex, freely whisking,
Spontaneous activity
in vivo
Neuronal Dynamics –
7
.1
Variability in vivo
Variability
- of membrane potential?
-
o
f
s
p
i
k
e
t
i
m
i
n
g
?
Detour: Receptive fields in V5/MT
visual
cortex
c
ells in visual cortex MT/V5
respond to motion stimuli
Neuronal Dynamics –
7
.1
Variability in vivo
adapted from Bair and Koch 1996;
data from Newsome 1989
1
5
r
e
p
e
t
i
t
i
o
n
s
o
f
t
h
e
s
a
m
e
r
a
n
d
o
m
d
o
t
m
o
t
i
o
n
p
a
t
t
e
r
n
Human Hippocampus
Neuronal Dynamics –
7
.1
Variability in vivo
Quiroga, Reddy,
Kreiman, Koch,
and Fried (2005).
Nature, 435:1102-1107.
Neuronal Dynamics –
7
.1
Variability in vitro
4 repetitions of the same time-dependent stimulus,
I(t)
brain slice
Neuronal Dynamics –
7
.1
Variability
F
l
u
c
t
u
a
t
i
o
n
s
-
of membrane potential
-
of spike times
fluctuations=noise?
model of fluctuations?
relevance for coding?
source of fluctuations?
In vivo data
looks ‘noisy’
In vitro data
fluctuations
W
e
e
k
7
–
V
a
r
i
a
b
i
l
i
t
y
a
n
d
N
o
i
s
e
:
T
h
e
q
u
e
s
t
i
o
n
o
f
t
h
e
n
e
u
r
a
l
c
o
d
e
Wulfram Gerstner
EPFL, Lausanne, Switzerland
7
.
1
V
a
r
i
a
b
i
l
i
t
y
o
f
s
p
i
k
e
t
r
a
i
n
s
- experiments
7
.
2
S
o
u
r
c
e
s
o
f
V
a
r
i
a
b
i
l
i
t
y
?
-
Is variability equal to noise?
7
.
3
T
h
r
e
e
d
e
f
i
n
i
t
i
o
n
s
o
f
R
a
t
e
c
o
d
e
-
P
o
i
s
s
o
n
M
o
d
e
l
7
.
4
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
a
r
r
i
v
a
l
-
M
e
m
b
r
a
n
e
p
o
t
e
n
t
i
a
l
f
l
u
c
t
u
a
t
i
o
n
s
7
.
5
.
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
f
i
r
i
n
g
- stochastic integrate-and-fire
W
e
e
k
7
–
p
a
r
t
2
:
S
o
u
r
c
e
s
o
f
V
a
r
i
a
b
i
l
i
t
y
Biological Modeling of
Neural Networks
- Intrinsic noise (ion channels)
Na
+
K
+
-
Finite number of channels
-
Finite temperature
Neuronal Dynamics –
7
.2.
Sources of Variability
Na+ channel from rat heart
(Patlak and Ortiz 1985)
A
t
r
a
c
e
s
f
r
o
m
a
p
a
t
c
h
c
o
n
t
a
i
n
i
n
g
s
e
v
e
r
a
l
c
h
a
n
n
e
l
s
.
Bottom: average gives current time course.
B
.
O
p
e
n
i
n
g
t
i
m
e
s
o
f
s
i
n
g
l
e
c
h
a
n
n
e
l
e
v
e
n
t
s
Steps:
Different number
of channels
Ca
2+
Na
+
K
+
I
o
n
s
/
p
r
o
t
e
i
n
s
Review from 2.5
Ion channels
stochastic opening and closing
- Intrinsic noise (ion channels)
Na
+
K
+
-
Finite number of channels
-
Finite temperature
Neuronal Dynamics –
7
.2.
Sources of Variability
Neuronal Dynamics –
7
.2
Variability in vitro
Image adapted from
Mainen&Sejnowski 1995
I(t)
neurons are fairly reliable
REVIEW from
1.5:
How good are integrate-and-fire models?
Aims: - predict spike initiation times
- predict subthreshold voltage
Badel et al., 2008
only possible, because
neurons are fairly reliable
- Intrinsic noise (ion channels)
Na
+
K
+
-
Finite number of channels
-
Finite temperature
Neuronal Dynamics –
7
.2.
Sources of Variability
small contribution!
The Brain: a highly connected system
Brain
Distributed architecture
High connectivity:
systematic, organized in local populations
b
u
t
s
e
e
m
i
n
g
l
y
r
a
n
d
o
m
Neuronal Dynamics –
7
.2
Sources of Variability
Population
- 50 000 neurons
- 20 percent inhibitory
-
r
a
n
d
o
m
l
y
c
o
n
n
e
c
t
e
d
R
a
n
d
o
m
f
i
r
i
n
g
i
n
a
p
o
p
u
l
a
t
i
o
n
o
f
L
I
F
n
e
u
r
o
n
s
100
200
time [ms]
50
10
A [Hz]
Neuron #
32340
32440
Brunel, J. Comput. Neurosc. 2000
Mayor and Gerstner, Phys. Rev E. 2005
Vogels et al., 2005
N
e
t
w
o
r
k
o
f
d
e
t
e
r
m
i
n
i
s
t
i
c
leaky integrate-and-fire:
‘fluctuations’
Population
- 50 000 neurons
- 20 percent inhibitory
-
r
a
n
d
o
m
l
y
c
o
n
n
e
c
t
e
d
R
a
n
d
o
m
f
i
r
i
n
g
i
n
a
p
o
p
u
l
a
t
i
o
n
o
f
L
I
F
n
e
u
r
o
n
s
100
200
time [ms]
Neuron # 32374
50
u [mV]
100
0
10
A [Hz]
Neuron #
32340
32440
100
200
time [ms]
50
- Variability of interspike intervals (ISI)
Variability of spike trains:
broad ISI distribution
here in simulations,
but also
in vivo
Brunel,
J. Comput. Neurosc. 2000
Mayor and Gerstner,
Phys. Rev E. 2005
Vogels and Abbott,
J. Neuroscience, 2005
Neuronal Dynamics –
7
.2.
Interspike interval distribution
- Intrinsic noise (ion channels)
Na
+
K
+
-
Network noise
Neuronal Dynamics –
7
.2.
Sources of Variability
small contribution!
big contribution!
In vivo data
looks ‘noisy’
In vitro data
small fluctuations
nearly deterministic
Neuronal Dynamics –
Quiz 7.1.
B
–
I
n
t
e
r
s
p
i
k
e
I
n
t
e
r
v
a
l
D
i
s
t
r
i
b
u
t
i
o
n
(
I
S
I
)
[ ] An isolated deterministic leaky integrate-and-fire neuron driven by a constant
current can have a broad ISI
[ ] A deterministic leaky integrate-and-fire neuron embedded into a randomly
connected network of integrate-and-fire neurons can have a broad ISI
[ ] A deterministic Hodgkin-Huxley model as in week 2 embedded into a randomly
connected network of Hodgkin-Huxley neurons can have a broad ISI
A
-
S
p
i
k
e
t
i
m
i
n
g
i
n
v
i
t
r
o
a
n
d
i
n
v
i
v
o
[ ] Reliability of spike timing can be assessed by repeating several times the
same stimulus
[ ] Spike timing in vitro is more reliable under injection of constant current than
with fluctuating current
[ ] Spike timing in vitro is more reliable than spike timing in vivo
W
e
e
k
7
–
V
a
r
i
a
b
i
l
i
t
y
a
n
d
N
o
i
s
e
:
T
h
e
q
u
e
s
t
i
o
n
o
f
t
h
e
n
e
u
r
a
l
c
o
d
e
Wulfram Gerstner
EPFL, Lausanne, Switzerland
7
.
1
V
a
r
i
a
b
i
l
i
t
y
o
f
s
p
i
k
e
t
r
a
i
n
s
- experiments
7
.
2
S
o
u
r
c
e
s
o
f
V
a
r
i
a
b
i
l
i
t
y
?
-
Is variability equal to noise?
7
.
3
P
o
i
s
s
o
n
M
o
d
e
l
- Poisson Model
- 3 definitions of rate coding
7
.
4
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
a
r
r
i
v
a
l
-
M
e
m
b
r
a
n
e
p
o
t
e
n
t
i
a
l
f
l
u
c
t
u
a
t
i
o
n
s
7
.
5
.
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
f
i
r
i
n
g
- stochastic integrate-and-fire
W
e
e
k
7
–
p
a
r
t
3
:
P
o
i
s
s
o
n
M
o
d
e
l
–
r
a
t
e
c
o
d
i
n
g
Biological Modeling of
Neural Networks
Homogeneous Poisson model: constant rate
stochastic spiking
Poisson model
Blackboard:
Poisson model
Neuronal Dynamics –
7
.3
Poisson Model
Neuronal Dynamics –
7
.3
Interval distrib
ution
?
(
i) Continuous time
(
ii) Discrete time steps
prob to ‘survive’
Blackboard:
Poisson model
Exercise 1.1 and 1.2: Poisson neuron
stimulus
Poisson rate
1.1. - Probability of NOT firing during time t?
1.2. - Interval distribution
p(s)?
s
1.3.- How can we detect if rate switches from
S
t
a
r
t
9
:
5
0
-
N
e
x
t
l
e
c
t
u
r
e
a
t
1
0
:
1
5
(1.4 at home:)
-2 neurons fire stochastically (Poisson) at 20Hz.
Percentage of spikes that coincide within +/-2 ms?)
W
e
e
k
7
–
V
a
r
i
a
b
i
l
i
t
y
a
n
d
N
o
i
s
e
:
T
h
e
q
u
e
s
t
i
o
n
o
f
t
h
e
n
e
u
r
a
l
c
o
d
e
Wulfram Gerstner
EPFL, Lausanne, Switzerland
7
.
1
V
a
r
i
a
b
i
l
i
t
y
o
f
s
p
i
k
e
t
r
a
i
n
s
- experiments
7
.
2
S
o
u
r
c
e
s
o
f
V
a
r
i
a
b
i
l
i
t
y
?
-
Is variability equal to noise?
7
.
3
P
o
i
s
s
o
n
M
o
d
e
l
- Poisson Model
- 3 definitions of rate coding
7
.
4
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
a
r
r
i
v
a
l
-
M
e
m
b
r
a
n
e
p
o
t
e
n
t
i
a
l
f
l
u
c
t
u
a
t
i
o
n
s
7
.
5
.
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
f
i
r
i
n
g
- stochastic integrate-and-fire
W
e
e
k
7
–
p
a
r
t
3
:
P
o
i
s
s
o
n
M
o
d
e
l
–
r
a
t
e
c
o
d
i
n
g
Biological Modeling of
Neural Networks
rate changes
Neuronal Dynamics –
7
.3
Inhomogeneous Poisson Process
Survivor function
Interval distribution
Neuronal Dynamics –
Quiz 7.2.
A
H
o
m
o
g
e
n
e
o
u
s
P
o
i
s
s
o
n
P
r
o
c
e
s
s
:
A spike train is generated by a homogeneous Poisson
process with rate 25Hz with time steps of 0.1ms.
[ ] The most likely interspike interval is 25ms.
[ ] The most likely interspike interval is 40 ms.
[ ] The most likely interspike interval is 0.1ms
[ ] We can’t say.
B
I
n
h
o
m
o
g
e
n
e
o
u
s
P
o
i
s
s
o
n
P
r
o
c
e
s
s
:
A spike train is generated by an inhomogeneous
Poisson process with a rate that oscillates periodically
(sine wave) between 0 and 50Hz (mean 25Hz). A first
spike has been fired at a time when the rate was at its
maximum. Time steps are 0.1ms.
[ ] The most likely interspike interval is 25ms.
[ ] The most likely interspike interval is 40 ms.
[ ] The most likely interspike interval is 0.1ms.
[ ] We can’t say.
Neuronal Dynamics –
7
.3.
Three definitions of Rate Codes
3
d
e
f
i
n
i
t
i
o
n
s
-
Temporal averaging
-
Averaging across repetitions
-
Population averaging (‘spatial’ averaging)
Variability of spike timing
stim
T=1s
trial 1
Neuronal Dynamics –
7
.3.
Rate codes: spike count
Brain
rate as a (normalized) spike count:
single neuron/single trial:
temporal average
Neuronal Dynamics –
7
.3.
Rate codes: spike count
single neuron/single trial:
temporal average
measure regularity
stim
T
trial 1
trial 2
trial
K
Neuronal Dynamics –
7
.3.
Spike count
: FANO factor
Brain
Fano factor
Neuronal Dynamics –
7
.3.
Three definitions of Rate Codes
3
d
e
f
i
n
i
t
i
o
n
s
-
Temporal averaging (spike count)
ISI distribution (regularity of spike train)
Fano factor (repeatability across repetitions)
-
Averaging across repetitions
-
Population averaging (‘spatial’ averaging)
Problem: slow!!!
Neuronal Dynamics –
7
.3.
Three definitions of Rate Codes
3
d
e
f
i
n
i
t
i
o
n
s
-
Temporal averaging
-
Averaging across repetitions
-
Population averaging
Problem: slow!!!
Variability of spike timing
stim
trial 1
trial 2
trial
K
Neuronal Dynamics –
7
.3.
Rate codes: PSTH
Brain
t
Averaging across repetitions
K
repetitions
Neuronal Dynamics –
7
.3.
Rate codes: PSTH
single neuron/many trials:
average across trials
Neuronal Dynamics –
7
.3.
Three definitions of Rate Codes
3
d
e
f
i
n
i
t
i
o
n
s
-
Temporal averaging
-
Averaging across repetitions
-
Population averaging
Problem: not useful
for animal!!!
population of neurons
with similar properties
stim
neuron 1
neuron 2
Neuron
K
Neuronal Dynamics –
7
.3.
Rate codes: population activity
Brain
population activity - rate defined by population average
t
Neuronal Dynamics –
7
.3.
Rate codes: population activity
‘natural readout’
Neuronal Dynamics –
7
.3.
Three definitions of Rate codes
T
h
r
e
e
a
v
e
r
a
g
i
n
g
m
e
t
h
o
d
s
-
over time
-
over repetitions
-
over population (space)
Not possible
for animal!!!
Too slow
for animal!!!
‘natural’
single neuron
single neuron
many neurons
T
n
sp
inhomogeneous Poisson model consistent with rate coding
population
activity
t
Neuronal Dynamics –
7.3
Inhomogeneous Poisson Process
Neuronal Dynamics –
Quiz 7.3.
R
a
t
e
c
o
d
e
s
.
S
u
p
p
o
s
e
t
h
a
t
i
n
s
o
m
e
b
r
a
i
n
a
r
e
a
w
e
h
a
v
e
a
g
r
o
u
p
o
f
5
0
0
n
e
u
r
o
n
s
.
A
l
l
n
e
u
r
o
n
s
h
a
v
e
i
d
e
n
t
i
c
a
l
p
a
r
a
m
e
t
e
r
s
a
n
d
t
h
e
y
a
l
l
r
e
c
e
i
v
e
t
h
e
s
a
m
e
i
n
p
u
t
.
I
n
p
u
t
i
s
g
i
v
e
n
b
y
s
e
n
s
o
r
y
s
t
i
m
u
l
a
t
i
o
n
a
n
d
p
a
s
s
e
s
t
h
r
o
u
g
h
2
p
r
e
l
i
m
i
n
a
r
y
n
e
u
r
o
n
a
l
p
r
o
c
e
s
s
i
n
g
s
t
e
p
s
b
e
f
o
r
e
i
t
a
r
r
i
v
e
s
a
t
o
u
r
g
r
o
u
p
o
f
5
0
0
n
e
u
r
o
n
s
.
W
i
t
h
i
n
t
h
e
g
r
o
u
p
,
n
e
u
r
o
n
s
a
r
e
n
o
t
c
o
n
n
e
c
t
e
d
t
o
e
a
c
h
o
t
h
e
r
.
I
m
a
g
i
n
e
t
h
e
b
r
a
i
n
a
s
a
m
o
d
e
l
n
e
t
w
o
r
k
c
o
n
t
a
i
n
i
n
g
1
0
0
0
0
0
n
o
n
l
i
n
e
a
r
i
n
t
e
g
r
a
t
e
-
a
n
d
-
f
i
r
e
n
e
u
r
o
n
s
,
s
o
t
h
a
t
w
e
k
n
o
w
e
x
a
c
t
l
y
h
o
w
e
a
c
h
n
e
u
r
o
n
f
u
n
c
t
i
o
n
s
.
E
x
p
e
r
i
m
e
n
t
a
l
i
s
t
A
m
a
k
e
s
a
m
e
a
s
u
r
e
m
e
n
t
i
n
a
s
i
n
g
l
e
t
r
i
a
l
o
n
a
l
l
5
0
0
n
e
u
r
o
n
s
u
s
i
n
g
a
m
u
l
t
i
-
e
l
e
c
t
r
o
d
e
a
r
r
a
y
,
d
u
r
i
n
g
a
p
e
r
i
o
d
o
f
s
e
n
s
o
r
y
s
t
i
m
u
l
a
t
i
o
n
.
E
x
p
e
r
i
m
e
n
t
a
l
i
s
t
B
p
i
c
k
s
a
n
a
r
b
i
t
r
a
r
y
s
i
n
g
l
e
n
e
u
r
o
n
a
n
d
r
e
p
e
a
t
s
t
h
e
s
a
m
e
s
e
n
s
o
r
y
s
t
i
m
u
l
a
t
i
o
n
5
0
0
t
i
m
e
s
(
w
i
t
h
l
o
n
g
p
a
u
s
e
s
i
n
b
e
t
w
e
e
n
,
s
a
y
o
n
e
p
e
r
d
a
y
)
.
E
x
p
e
r
i
m
e
n
t
a
l
i
s
t
C
r
e
p
e
a
t
s
t
h
e
s
a
m
e
s
e
n
s
o
r
y
s
t
i
m
u
l
a
t
i
o
n
5
0
0
t
i
m
e
s
(
1
p
e
r
d
a
y
)
,
b
u
t
e
v
e
r
y
d
a
y
h
e
p
i
c
k
s
a
r
a
n
d
o
m
n
e
u
r
o
n
(
a
m
o
n
g
s
t
t
h
e
5
0
0
n
e
u
r
o
n
s
)
.
All three determine the time-dependent firing rate.
[ ] A and B and C are expected to find the same result.
[ ] A and B are expected to find the same result, but that of C is expected to be different.
[ ] B and C are expected to find the same result, but that of A is expected to be different.
[ ] None of the above three options is correct.
Start at 10:50,
Discussion at 10:55
Neuronal Dynamics:
Computational Neuroscience
of Single Neurons
W
e
e
k
7
–
V
a
r
i
a
b
i
l
i
t
y
a
n
d
N
o
i
s
e
:
T
h
e
q
u
e
s
t
i
o
n
o
f
t
h
e
n
e
u
r
a
l
c
o
d
e
Wulfram Gerstner
EPFL, Lausanne, Switzerland
7
.
1
V
a
r
i
a
b
i
l
i
t
y
o
f
s
p
i
k
e
t
r
a
i
n
s
- experiments
7
.
2
S
o
u
r
c
e
s
o
f
V
a
r
i
a
b
i
l
i
t
y
?
-
Is variability equal to noise?
7
.
3
T
h
r
e
e
d
e
f
i
n
i
t
i
o
n
s
o
f
R
a
t
e
c
o
d
e
-
P
o
i
s
s
o
n
M
o
d
e
l
7
.
4
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
a
r
r
i
v
a
l
-
M
e
m
b
r
a
n
e
p
o
t
e
n
t
i
a
l
f
l
u
c
t
u
a
t
i
o
n
s
7
.
5
.
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
f
i
r
i
n
g
- stochastic integrate-and-fire
W
e
e
k
7
–
p
a
r
t
4
:
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
a
r
r
i
v
a
l
Crochet et al., 2011
awake mouse, freely whisking
,
Spontaneous activity
in vivo
Neuronal Dynamics –
7
.4
Variability in vivo
Variability
of membrane potential?
Population
- 50 000 neurons
- 20 percent inhibitory
-
r
a
n
d
o
m
l
y
c
o
n
n
e
c
t
e
d
R
a
n
d
o
m
f
i
r
i
n
g
i
n
a
p
o
p
u
l
a
t
i
o
n
o
f
L
I
F
n
e
u
r
o
n
s
100
200
time [ms]
Neuron # 32374
50
u [mV]
100
0
10
A [Hz]
Neuron #
32340
32440
100
200
time [ms]
50
Neuronal Dynamics –
7
.4
Membrane potential fluctuat
ions
big contribution!
‘Network noise’
Pull out one neuron
from neuron’s point
of view:
s
t
o
c
h
a
s
t
i
c
s
p
i
k
e
a
r
r
i
v
a
l
Neuronal Dynamics –
7
.4
. Stochastic Spike Arrival
Take
Total spike train of K presynaptic neurons
spike train
expectation
Pull out one neuron
Blackboard
now!
Neuronal Dynamics –
Exercise 2.1 NOW
A leaky integrate-and-fire neuron without threshold
(=passive membrane) receives stochastic spike arrival,
described as a homogeneous Poisson process.
C
a
l
c
u
l
a
t
e
t
h
e
m
e
a
n
m
e
m
b
r
a
n
e
p
o
t
e
n
t
i
a
l
.
T
o
d
o
s
o
,
u
s
e
t
h
e
a
b
o
v
e
f
o
r
m
u
l
a
.
Start at 11:35,
Discussion at 11:48
Neuronal Dynamics –
Quiz 7.4
A linear (=passive) membrane has a potential given by
Suppose the neuronal dynamics are given by
[ ] the filter
f
is exponential with time constant
[ ] the constant
a
is equal to the time constant
[ ] the constant
a
is equal to
[ ] the amplitude of the filter
f
is proportional to
q
[ ] the amplitude of the filter
f
is q
Neuronal Dynamics –
7
.4
. Calculating the mean
mean: assume Poisson process
use for exercise
Biological Modeling and
Neural Networks
W
e
e
k
7
–
V
a
r
i
a
b
i
l
i
t
y
a
n
d
N
o
i
s
e
:
T
h
e
q
u
e
s
t
i
o
n
o
f
t
h
e
n
e
u
r
a
l
c
o
d
e
Wulfram Gerstner
EPFL, Lausanne, Switzerland
7
.
1
V
a
r
i
a
b
i
l
i
t
y
o
f
s
p
i
k
e
t
r
a
i
n
s
- experiments
7
.
2
S
o
u
r
c
e
s
o
f
V
a
r
i
a
b
i
l
i
t
y
?
-
Is variability equal to noise?
7
.
3
T
h
r
e
e
d
e
f
i
n
i
t
i
o
n
s
o
f
R
a
t
e
c
o
d
e
-
P
o
i
s
s
o
n
M
o
d
e
l
7
.
4
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
a
r
r
i
v
a
l
-
M
e
m
b
r
a
n
e
p
o
t
e
n
t
i
a
l
f
l
u
c
t
u
a
t
i
o
n
s
7
.
5
.
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
f
i
r
i
n
g
- Stochastic Integrate-and-fire
W
e
e
k
7
–
p
a
r
t
5
:
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
f
i
r
i
n
g
i
n
i
n
t
e
g
r
a
t
e
-
a
n
d
-
f
i
r
e
m
o
d
e
l
s
Neuronal Dynamics –
7
.5
. Fluctuation of current/potential
Neuronal Dynamics –
7
.5
. Fluctuation of potential
for a passive membrane, we
can analytically predict the
mean of membrane potential
fluctuations
Passive membrane
=Leaky integrate-and-fire
without threshold
ADD THRESHOLD
Leaky Integrate-and-Fire
u(t)
Neuronal Dynamics –
7
.5
. Stochastic leaky integrate-and-fire
noisy input/ diffusive noise/
stochastic spike arrival
subthreshold regime:
- firing driven by fluctuations
-
b
r
o
a
d
I
S
I
d
i
s
t
r
i
b
u
t
i
o
n
-
in vivo
like
ISI distribution
Neuronal Dynamics
week 5
–
References and Suggested Reading
R
e
a
d
i
n
g
:
W
.
G
e
r
s
t
n
e
r
,
W
.
M
.
K
i
s
t
l
e
r
,
R
.
N
a
u
d
a
n
d
L
.
P
a
n
i
n
s
k
i
,
Neuronal Dynamics: from single neurons to networks and models of cognition.
Ch. 7,8
:
Cambridge, 2014
O
R
W
.
G
e
r
s
t
n
e
r
a
n
d
W
.
M
.
K
i
s
t
l
e
r
,
S
p
i
k
i
n
g
N
e
u
r
o
n
M
o
d
e
l
s
,
C
h
a
p
t
e
r
5
,
C
a
m
b
r
i
d
g
e
,
2
0
0
2
-
Rieke, F., Warland, D., de Ruyter van Steveninck, R., and Bialek, W. (1996).
Spikes - Exploring the neural code
. MIT Press.
-Faisal, A., Selen, L., and Wolpert, D. (2008). Noise in the nervous system.
Nat. Rev. Neurosci
., 9:202
-Gabbiani, F. and Koch, C. (1998). Principles of spike train analysis. In Koch, C. and Segev, I., editors,
Methods in Neuronal Modeling, chapter 9, pages 312-360. MIT press, 2nd edition.
-
Softky, W. and Koch, C. (1993). The highly irregular firing pattern of cortical cells is inconsistent with temporal integration of random
epsps.
J . Neurosci.
, 13:334-350.
-
Stein, R. B. (1967). Some models of neuronal variability.
Biophys. J
., 7:37-68.
-
Siegert, A. (1951). On the first passage time probability problem. Phys. Rev., 81:617{623.-
Konig, P., et al. (1996). Integrator or coincidence detector? the role of the cortical
neuron revisited.
Trends Neurosci,
19(4):130-137.
THE END
Biological Modeling and
Neural Networks
W
e
e
k
7
–
V
a
r
i
a
b
i
l
i
t
y
a
n
d
N
o
i
s
e
:
T
h
e
q
u
e
s
t
i
o
n
o
f
t
h
e
n
e
u
r
a
l
c
o
d
e
Wulfram Gerstner
EPFL, Lausanne, Switzerland
7
.
1
V
a
r
i
a
b
i
l
i
t
y
o
f
s
p
i
k
e
t
r
a
i
n
s
- experiments
7
.
2
S
o
u
r
c
e
s
o
f
V
a
r
i
a
b
i
l
i
t
y
?
-
Is variability equal to noise?
7
.
3
T
h
r
e
e
d
e
f
i
n
i
t
i
o
n
s
o
f
R
a
t
e
c
o
d
e
-
P
o
i
s
s
o
n
M
o
d
e
l
7
.
4
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
a
r
r
i
v
a
l
-
M
e
m
b
r
a
n
e
p
o
t
e
n
t
i
a
l
:
m
e
a
n
v
a
l
u
e
7
.
5
.
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
f
i
r
i
n
g
- Integrate-and-fire
7
.
6
S
u
b
t
h
r
e
s
h
o
l
d
a
n
d
s
u
p
e
r
t
h
.
7
.
7
.
C
a
l
c
u
l
a
t
e
f
l
u
c
t
u
a
t
i
o
n
s
W
e
e
k
7
–
p
a
r
t
5
:
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
f
i
r
i
n
g
i
n
i
n
t
e
g
r
a
t
e
-
a
n
d
-
f
i
r
e
m
o
d
e
l
s
Neuronal Dynamics –
7
.4
. Fluctuation of current/potential
fluctuating input current
fluctuating potential
Random spike arrival
stochastic spike arrival in I&F – interspike intervals
ISI distribution
LIF with Diffusive noise (stochastic spike arrival)
Superthreshold vs. Subthreshold regime
Neuronal Dynamics –
5.4b
. Fluctuation of potential
fluctuating potential
Passive membrane
for a passive membrane, we
can analytically predict the
amplitude of membrane
potential fluctuations
Leaky integrate-and-fire
in subthreshold regime
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
a
r
r
i
v
a
l
:
Neuronal Dynamics –
review: Fluctuations of potential
effective noise current
u(t)
Neuronal Dynamics –
7
.5
. Stochastic leaky integrate-and-fire
noisy input/
diffusive noise/
stochastic spike
arrival
LIF
u(t)
Neuronal Dynamics –
7
.5
. Stochastic leaky integrate-and-fire
noisy input/ diffusive noise/
stochastic spike arrival
subthreshold regime:
- firing driven by fluctuations
- broad ISI distribution
-
in vivo
like
ISI distribution
Crochet et al., 2011
awake mouse, freely whisking
,
Spontaneous activity
in vivo
Neuronal Dynamics –
7.5
Variability in vivo
membrane potential
most of the time subthreshold
spikes are rare events
Biological Modeling and
Neural Networks
W
e
e
k
7
–
V
a
r
i
a
b
i
l
i
t
y
a
n
d
N
o
i
s
e
:
T
h
e
q
u
e
s
t
i
o
n
o
f
t
h
e
n
e
u
r
a
l
c
o
d
e
Wulfram Gerstner
EPFL, Lausanne, Switzerland
7
.
1
V
a
r
i
a
b
i
l
i
t
y
o
f
s
p
i
k
e
t
r
a
i
n
s
7
.
2
S
o
u
r
c
e
s
o
f
V
a
r
i
a
b
i
l
i
t
y
?
-
Is variability equal to noise?
7
.
3
T
h
r
e
e
d
e
f
i
n
i
t
i
o
n
s
o
f
R
a
t
e
c
o
d
7
.
4
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
a
r
r
i
v
a
l
-
M
e
m
b
r
a
n
e
p
o
t
e
n
t
i
a
l
:
m
e
a
n
v
a
l
u
e
7
.
5
.
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
f
i
r
i
n
7
.
6
S
u
b
t
h
r
e
s
h
o
l
d
a
n
d
s
u
p
e
r
t
h
.
7
.
7
.
C
a
l
c
u
l
a
t
e
f
l
u
c
t
u
a
t
i
o
n
s
-
a
u
t
o
c
o
r
r
e
l
a
t
i
o
n
o
f
m
e
m
b
r
a
n
e
p
o
t
.
- standard deviation of membr. Pot.
- autocorrelation of Poisson
W
e
e
k
7
–
p
a
r
t
5
:
S
t
o
c
h
a
s
t
i
c
s
p
i
k
e
f
i
r
i
n
g
i
n
i
n
t
e
g
r
a
t
e
-
a
n
d
-
f
i
r
e
m
o
d
e
l
s
Crochet et al., 2011
awake mouse, freely whisking
,
Spontaneous activity
in vivo
Neuronal Dynamics –
7.7
Variability in vivo
Variability
of membrane potential?
Subthreshold regime
Neuronal Dynamics –
7.7. Fluctuations of potential
Neuronal Dynamics –
7.7. Fluctuations of potential
Input: step + fluctuations
Neuronal Dynamics –
7.7. Calculating autocorrelations
Autocorrelation
Mean:
Neuronal Dynamics –
7.7. Autocorrelation of Poisson
spike train
math detour
now!
Probability of spike
i
n
s
t
e
p
n
A
N
D
s
t
e
p
k
Autocorrelation (continuous time)
Neuronal Dynamics –
7.7. Fluctuation of potential
fluctuating potential
Passive membrane
for a passive membrane, we
can analytically predict the
amplitude of membrane
potential fluctuations
Leaky integrate-and-fire
in subthreshold regime
First Lecture: 7.1, 7.2, and 7.3 (HOMOGENEOUS Poisson Model). Dann um 9:50h vor der Pause, Exercise 1 (Poisson Model)
Second Lecture: 7.3 (INHOMOGENEOUS Poission model, Quiz 1, and three definitions of rate). Quiz 2, five minutes before the end.
Third Lecture: 7.4 Stochastic spike arrival, on the blackboard, then exericise 2
Understanding the variability of spike trains and sources of variability in neural networks, dissecting if variability is equivalent to noise. Delving into the Poisson model, stochastic spike arrival, and firing, and biological modeling of neural networks. Examining variability in different brain regions and experimental setups both in vivo and in vitro. Investigating the implications of fluctuations in membrane potential and spike times on neural coding.
Uploaded on Oct 05, 2024 | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Week 7 part 1 :Variability 7.1Variability of spike trains - experiments 7.2 Sources of Variability? - Is variability equal to noise? 7.3 Poisson Model -Three definitions of Rate code 7.4Stochastic spike arrival - Membrane potential fluctuations 7.5. Stochastic spike firing - stochastic integrate-and-fire Biological Modeling of Neural Networks Week 7 Variability and Noise: The question of the neural code Wulfram Gerstner EPFL, Lausanne, Switzerland
Neuronal Dynamics 7.1. Variability motor cortex frontal cortex visual cortex to motor output
Neuronal Dynamics 7.1Variability in vivo Variability - of membrane potential? - of spike timing? Spontaneous activity in vivo awake mouse, cortex, freely whisking, Crochet et al., 2011
Detour: Receptive fields in V5/MT visual cortex cells in visual cortex MT/V5 respond to motion stimuli
Neuronal Dynamics 7.1Variability in vivo 15 repetitions of the same random dot motion pattern adapted from Bair and Koch 1996; data from Newsome 1989
Neuronal Dynamics 7.1Variability in vivo Human Hippocampus Quiroga, Reddy, Kreiman, Koch, and Fried (2005). Nature, 435:1102-1107. Sidne y opera Sidney opera Sidne y opera
Neuronal Dynamics 7.1Variability in vitro 4 repetitions of the same time-dependent stimulus, brain slice I(t)
Neuronal Dynamics 7.1Variability Fluctuations -of membrane potential -of spike times fluctuations=noise? In vivo data looks noisy relevance for coding? In vitro data fluctuations source of fluctuations? model of fluctuations?
Week 7 part 2 : Sources of Variability 7.1Variability of spike trains - experiments 7.2 Sources of Variability? - Is variability equal to noise? 7.3 Three definitions of Rate code - Poisson Model 7.4Stochastic spike arrival - Membrane potential fluctuations 7.5. Stochastic spike firing - stochastic integrate-and-fire Biological Modeling of Neural Networks Week 7 Variability and Noise: The question of the neural code Wulfram Gerstner EPFL, Lausanne, Switzerland
Neuronal Dynamics 7.2.Sources of Variability - Intrinsic noise (ion channels) -Finite number of channels -Finite temperature Na+ K+
Review from 2.5 Ion channels Na+ K+ Steps: Different number of channels Ca2+ Ions/proteins Na+ channel from rat heart (Patlak and Ortiz 1985) A traces from a patch containing several channels. Bottom: average gives current time course. B. Opening times of single channel events
Neuronal Dynamics 7.2.Sources of Variability - Intrinsic noise (ion channels) -Finite number of channels -Finite temperature Na+ K+ -Network noise (background activity) -Spike arrival from other neurons -Beyond control of experimentalist Check intrinisic noise by removing the network
Neuronal Dynamics 7.2Variability in vitro neurons are fairly reliable I(t) Image adapted from Mainen&Sejnowski 1995
REVIEW from 1.5: How good are integrate-and-fire models? Badel et al., 2008 Aims: - predict spike initiation times - predict subthreshold voltage only possible, because neurons are fairly reliable
Neuronal Dynamics 7.2.Sources of Variability - Intrinsic noise (ion channels) -Finite number of channels -Finite temperature Na+ K+ -Network noise (background activity) -Spike arrival from other neurons -Beyond control of experimentalist Check network noise by simulation!
Neuronal Dynamics 7.2Sources of Variability The Brain: a highly connected system Brain High connectivity: systematic, organized in local populations but seemingly random Distributed architecture 10 10 neurons 4 10 connections/neurons
Random firing in a population of LIF neurons A [Hz] 10 32440 Neuron # -low rate -high rate input 32340 time [ms] 200 50 100 Brunel, J. Comput. Neurosc. 2000 Mayor and Gerstner, Phys. Rev E. 2005 Vogels et al., 2005 Population - 50 000 neurons - 20 percent inhibitory - randomly connected
Random firing in a population of LIF neurons A [Hz] 10 32440 Neuron # -low rate -high rate input 32340 time [ms] 200 50 100 100 Neuron # 32374 u [mV] Population - 50 000 neurons - 20 percent inhibitory - randomly connected 0 time [ms] 200 50 100
Neuronal Dynamics 7.2.Interspikeinterval distribution here in simulations, but also in vivo - Variability of interspike intervals (ISI) 100 u [mV] ISI ISI distribution 0 100 100 t [ms] 500 time [ms] 200 50 100 Brunel, J. Comput. Neurosc. 2000 Mayor and Gerstner, Phys. Rev E. 2005 Vogels and Abbott, J. Neuroscience, 2005 Variability of spike trains: broad ISI distribution
Neuronal Dynamics 7.2.Sources of Variability - Intrinsic noise (ion channels) Na+ In vivo data looks noisy K+ -Network noise In vitro data small fluctuations nearly deterministic
Neuronal Dynamics Quiz 7.1. A- Spike timing in vitro and in vivo [ ] Reliability of spike timing can be assessed by repeating several times the same stimulus [ ] Spike timing in vitro is more reliable under injection of constant current than with fluctuating current [ ] Spike timing in vitro is more reliable than spike timing in vivo B Interspike Interval Distribution (ISI) [ ] An isolated deterministic leaky integrate-and-fire neuron driven by a constant current can have a broad ISI [ ] A deterministic leaky integrate-and-fire neuron embedded into a randomly connected network of integrate-and-fire neurons can have a broad ISI [ ] A deterministic Hodgkin-Huxley model as in week 2 embedded into a randomly connected network of Hodgkin-Huxley neurons can have a broad ISI
Week 7 part 3 : Poisson Model rate coding 7.1Variability of spike trains - experiments 7.2 Sources of Variability? - Is variability equal to noise? 7.3 Poisson Model - Poisson Model - 3 definitions of rate coding 7.4Stochastic spike arrival - Membrane potential fluctuations 7.5. Stochastic spike firing - stochastic integrate-and-fire Biological Modeling of Neural Networks Week 7 Variability and Noise: The question of the neural code Wulfram Gerstner EPFL, Lausanne, Switzerland
Neuronal Dynamics 7.3 Poisson Model Homogeneous Poisson model: constant rate Blackboard: Poisson model t = FP t Probability of finding a spike 0 stochastic spiking Poisson model
Neuronal Dynamics 7.3 Interval distribution ? Probability of firing: = P t 0 F t (i) Continuous time (ii) Discrete time steps prob to survive Blackboard: Poisson model t 0 dS t t dt = ( | ) ( | ) S t t 1 0 0 1 0
Exercise 1.1 and 1.2: Poisson neuron Start 9:50 - Next lecture at 10:15 Poisson rate s stimulus 1 t t 0 1.1. - Probability of NOT firing during time t? 1.2. - Interval distribution p(s)? 1.3.- How can we detect if rate switches from 0 1 (1.4 at home:) -2 neurons fire stochastically (Poisson) at 20Hz. Percentage of spikes that coincide within +/-2 ms?)
Week 7 part 3 : Poisson Model rate coding 7.1Variability of spike trains - experiments 7.2 Sources of Variability? - Is variability equal to noise? 7.3 Poisson Model - Poisson Model - 3 definitions of rate coding 7.4Stochastic spike arrival - Membrane potential fluctuations 7.5. Stochastic spike firing - stochastic integrate-and-fire Biological Modeling of Neural Networks Week 7 Variability and Noise: The question of the neural code Wulfram Gerstner EPFL, Lausanne, Switzerland
Neuronal Dynamics 7.3 Inhomogeneous Poisson Process rate changes t = ( ) t FP t Probability of firing t = ( ') t dt ( | ) S t t exp( ') Survivor function t t Interval distribution = ( )exp( t ( ') t dt ( | ) P t t ') t
Neuronal Dynamics Quiz 7.2. A Homogeneous Poisson Process: A spike train is generated by a homogeneous Poisson process with rate 25Hz with time steps of 0.1ms. [ ] The most likely interspike interval is 25ms. [ ] The most likely interspike interval is 40 ms. [ ] The most likely interspike interval is 0.1ms [ ] We can t say. B Inhomogeneous Poisson Process: A spike train is generated by an inhomogeneous Poisson process with a rate that oscillates periodically (sine wave) between 0 and 50Hz (mean 25Hz). A first spike has been fired at a time when the rate was at its maximum. Time steps are 0.1ms. [ ] The most likely interspike interval is 25ms. [ ] The most likely interspike interval is 40 ms. [ ] The most likely interspike interval is 0.1ms. [ ] We can t say.
Neuronal Dynamics 7.3.Three definitions of Rate Codes 3 definitions -Temporal averaging - Averaging across repetitions - Population averaging ( spatial averaging)
Neuronal Dynamics 7.3.Rate codes: spike count Variability of spike timing trial 1 rate as a (normalized) spike count: sp n T = ( ) t single neuron/single trial: temporal average T=1s Brain stim
Neuronal Dynamics 7.3.Rate codes: spike count single neuron/single trial: temporal average sp n T = ( ) t Variability of interspike intervals (ISI) measure regularity 100 u [mV] ISI ISI distribution 0 100 100 t [ms] 500 time [ms] 200 50 100
Neuronal Dynamics 7.3.Spike count: FANO factor = = sp 5 n trial 1 1 sp 6 n trial 2 2 = sp K 4 n trial K Fano factor ( ) T 2 sp k sp k n n Brain stim = F sp k n
Neuronal Dynamics 7.3.Three definitions of Rate Codes 3 definitions -Temporal averaging (spike count) ISI distribution (regularity of spike train) Fano factor (repeatability across repetitions) Problem: slow!!! - Averaging across repetitions - Population averaging ( spatial averaging)
Neuronal Dynamics 7.3.Three definitions of Rate Codes 3 definitions -Temporal averaging Problem: slow!!! - Averaging across repetitions - Population averaging
Neuronal Dynamics 7.3.Rate codes: PSTH Variability of spike timing trial 1 trial 2 trial K Brain stim
Neuronal Dynamics 7.3.Rate codes: PSTH Averaging across repetitions single neuron/many trials: average across trials t K repetitions + ; ( ) n t t t = ( ) PSTH t K t PSTH(t) Stim(t) K=50 trials
Neuronal Dynamics 7.3.Three definitions of Rate Codes 3 definitions -Temporal averaging - Averaging across repetitions Problem: not useful for animal!!! - Population averaging
Neuronal Dynamics 7.3.Rate codes: population activity population of neurons with similar properties neuron 1 neuron 2 Neuron K Brain stim
Neuronal Dynamics 7.3.Rate codes: population activity population activity - rate defined by population average t t + ( ; ) n t t t population activity natural readout = ( ) A t N t
Neuronal Dynamics 7.3.Three definitions of Rate codes Three averaging methods -over time Too slow for animal!!! single neuron - over repetitions single neuron Not possible for animal!!! - over population (space) natural many neurons
Neuronal Dynamics 7.3 Inhomogeneous Poisson Process t A(t) I(t) nsp + ( ; ) n t t t + ; ( ) n t t t = ( ) A t = ( ) PSTH t N t K t T population activity inhomogeneous Poisson model consistent with rate coding
Neuronal Dynamics Quiz 7.3. Rate codes. Suppose that in some brain area we have a group of 500 neurons. All neurons have identical parameters and they all receive the same input. Input is given by sensory stimulation and passes through 2 preliminary neuronal processing steps before it arrives at our group of 500 neurons. Within the group, neurons are not connected to each other. Imagine the brain as a model network containing 100 000 nonlinear integrate-and- fire neurons, so that we know exactly how each neuron functions. Experimentalist A makes a measurement in a single trial on all 500 neurons using a multi- electrode array, during a period of sensory stimulation. Experimentalist B picks an arbitrary single neuron and repeats the same sensory stimulation 500 times (with long pauses in between, say one per day). Experimentalist C repeats the same sensory stimulation 500 times (1 per day), but every day he picks a random neuron (amongst the 500 neurons). Start at 10:50, Discussion at 10:55 All three determine the time-dependent firing rate. [ ] A and B and C are expected to find the same result. [ ] A and B are expected to find the same result, but that of C is expected to be different. [ ] B and C are expected to find the same result, but that of A is expected to be different. [ ] None of the above three options is correct.
Week 7 part 4 :Stochastic spike arrival 7.1Variability of spike trains - experiments 7.2 Sources of Variability? - Is variability equal to noise? 7.3 Three definitions of Rate code - Poisson Model 7.4Stochastic spike arrival - Membrane potential fluctuations 7.5. Stochastic spike firing - stochastic integrate-and-fire Neuronal Dynamics: Computational Neuroscience of Single Neurons Week 7 Variability and Noise: The question of the neural code Wulfram Gerstner EPFL, Lausanne, Switzerland
Neuronal Dynamics 7.4Variability in vivo Spontaneous activity in vivo Variability of membrane potential? awake mouse, freely whisking, Crochet et al., 2011
Random firing in a population of LIF neurons A [Hz] 10 32440 Neuron # -low rate -high rate input 32340 time [ms] 200 50 100 100 Neuron # 32374 u [mV] Population - 50 000 neurons - 20 percent inhibitory - randomly connected 0 time [ms] 200 50 100
Neuronal Dynamics 7.4Membrane potential fluctuations from neuron s point of view: stochastic spike arrival Pull out one neuron Network noise
Neuronal Dynamics 7.4. Stochastic Spike Arrival Total spike train of K presynaptic neurons Blackboard now! t spike train Pull out one neuron Probability of spike arrival: = 0 FP K t 0 t expectation Take K = t t f ( ) ( ) S t k = 1 k f
Neuronal Dynamics Exercise 2.1 NOW t Passive membrane d syn = t t = + f ( ) ( ) ( ) ( ) ( ) u t ds f s s u u u R I t rest k dt f A leaky integrate-and-fire neuron without threshold (=passive membrane) receives stochastic spike arrival, described as a homogeneous Poisson process. Calculate the mean membrane potential. To do so, use the above formula. Start at 11:35, Discussion at 11:48
Neuronal Dynamics Quiz 7.4 A linear (=passive) membrane has a potential given by = ') ( ' + f ( ) ' ( ) u t dt f t t t t a k f Suppose the neuronal dynamics are given by du dt = u u + t t f ( ) ( ) q rest f [ ] the filter f is exponential with time constant [ ] the constant a is equal to the time constant [ ] the constant a is equal to rest [ ] the amplitude of the filter f is proportional to q u [ ] the amplitude of the filter f is q
Neuronal Dynamics 7.4. Calculating the mean ( k f syn = ) ( ) RI t w t t k k f t 1 k = ') ( ' f syn f = ( ) x t ' ( ) dt f t t t t ( ) ' ( ) ' t ' ( ) I t w dt t t t k k R k f f mean: assume Poisson process k f = f f syn ( ) x t ' ( ') ( ' t ) dt f t t t = = 1 ( ) ' ( ) ' ' ( ) I I t w dt t t t t 0 k k R k f = ( ) x t ' ( ') ( ') t dt f t t k = ' 1 ( ) ' I w dt t t 0 k k R rate of inhomogeneous Poisson process
