Neural Network Control for Seismometer Temperature Stabilization
N
e
u
r
a
l
N
e
t
w
o
r
k
C
o
n
t
r
o
l
f
o
r
S
e
i
s
m
o
m
e
t
e
r
T
e
m
p
e
r
a
t
u
r
e
S
t
a
b
i
l
i
z
a
t
i
o
n
Ella Buehner Gattis
Supervised by Rana Adhikari, Koji Arai, Tega Edo
1
LIGO-T2000291–vX
I
n
t
r
o
d
u
c
t
i
o
n
2
LIGO-T2000291–vX
M
o
t
i
v
a
t
i
o
n
•
Increase overall performance of controller
•
Nonlinear control has better disturbance rejection, faster set
point convergence
•
Training a neural network controller for temperature stabilization
will then be applicable across many other stabilization problems
in LIGO
•
Mirrors, pendulum stabilization, etc
•
Sets premise for nonlinear controllers throughout LIGO
3
LIGO-T2000291–vX
O
b
j
e
c
t
i
v
e
s
•
Design feedback system to keep temperature constant within
seismometer enclosure
•
Non-linear control for
better accuracy
•
Neural Network control
has ability for self-learning
and may also reduce
control energy
4
seismometer
foam insulation
stainless steel enclosure
resistive heating
mesh
LIGO-T2000291–vX
H
o
w
a
N
e
u
r
a
l
N
e
t
w
o
r
k
W
o
r
k
s
•
Many individual neurons
connected in dense
network
•
Each neuron receives
inputs multiplied by a
weight
•
Layers are made of rows
of neurons
5
LIGO-T2000291–vX
H
o
w
a
N
e
u
r
a
l
N
e
t
w
o
r
k
W
o
r
k
s
•
Neuron outputs sum of
values received
multiplied by their
respective weights
•
Connections initially have
randomly assigned
weights and biases,
which are adjusted in
process of training
6
LIGO-T2000291–vX
H
o
w
a
N
e
u
r
a
l
N
e
t
w
o
r
k
W
o
r
k
s
•
Network iterates over
training set, adjusting
weights until desired
output is received for a
particular input
•
Maps inputs to outputs
•
Train by supervised
learning or reinforcement
learning
7
LIGO-T2000291–vX
S
u
p
e
r
v
i
s
e
d
L
e
a
r
n
i
n
g
8
input training
data
neural network
output
ideal output
training data
difference
adjust
weights
LIGO-T2000291–vX
I
n
i
t
i
a
l
N
e
u
r
a
l
N
e
t
w
o
r
k
•
To better understand the concept,
first built a simple Convolutional
Neural Network to imitate the
output of a digital filter
•
Supervised learning
•
Tensorflow, Keras
•
Digital filter applied to random
noise using second order sections
•
Grey noisy signal is input of
network
•
Blue smoothed signal is the target
output of network
9
LIGO-T2000291–vX
I
n
i
t
i
a
l
N
e
u
r
a
l
N
e
t
w
o
r
k
•
Labelled set of input (random noise) and output (filtered signal)
•
As network goes through more training iterations, the weights
and biases are adjusted to achieve the desired output
•
Input shape matters!
(1, data shape, 1)
•
(batch size, number of steps, channels)
10
LIGO-T2000291–vX
C
o
n
v
o
l
u
t
i
o
n
a
l
N
e
t
w
o
r
k
S
t
r
u
c
t
u
r
e
•
Convolutional network adapts to broader pattern across data
set
•
Convolution layer searches for features across a particular
“window”, acts like a filter
•
Takes element-wise product of filter and input
•
Also need a convolutional neural network for optimum
temperature control
11
LIGO-T2000291–vX
C
o
n
v
o
l
u
t
i
o
n
a
l
N
e
t
w
o
r
k
S
t
r
u
c
t
u
r
e
12
input
output
extracted
features
window size 3
dot prod w/ weights
time
LIGO-T2000291–vX
S
i
m
p
l
e
n
e
u
r
a
l
n
e
t
w
o
r
k
13
•
loss is defined
here by mean
squared error
•
able to achieve
very high
accuracy after
around 220
training epochs
LIGO-T2000291–vX
T
a
r
g
e
t
a
n
d
r
e
a
l
n
e
t
w
o
r
k
o
u
t
p
u
t
a
l
m
o
s
t
c
o
m
p
l
e
t
e
l
y
o
v
e
r
l
a
p
14
LIGO-T2000291–vX
15
PSD of neural network output and
target output
LIGO-T2000291–vX
S
u
p
e
r
v
i
s
e
d
L
e
a
r
n
i
n
g
v
s
R
e
i
n
f
o
r
c
e
m
e
n
t
L
e
a
r
n
i
n
g
16
input training
data
neural network
output
ideal output
training data
difference
adjust
weights
agent
action
environment
state
reward
LIGO-T2000291–vX
T
e
m
p
e
r
a
t
u
r
e
c
o
n
t
r
o
l
n
e
u
r
a
l
n
e
t
w
o
r
k
•
Also use convolutional neural network
•
Control: interacts directly with environment- need to use
reinforcement learning
•
Whereas supervised learning uses a labelled data set of input
and desired output to train, reinforcement learning generates its
own labels – the “reward”
•
Need continuous action space (output will be continuous –
power supplied to heater)
•
Actor critic most suitable
17
LIGO-T2000291–vX
R
e
i
n
f
o
r
c
e
m
e
n
t
l
e
a
r
n
i
n
g
:
O
p
e
n
A
I
g
y
m
•
Several different environments
available to train networks with
game-like scenarios
•
In our case want to train network
for temperature stabilization
•
Built custom environment for this
based on pendulum example (both
are stabilization problems)
18
stabilizing an inverted pendulum
LIGO-T2000291–vX
A
c
t
o
r
C
r
i
t
i
c
N
e
t
w
o
r
k
19
a
c
t
o
r
observation
action
c
r
i
t
i
c
observation
action
Q-value
LIGO-T2000291–vX
A
c
t
o
r
C
r
i
t
i
c
N
e
t
w
o
r
k
•
take a single step, for
which a reward and
the next state are
obtained
•
as more steps are
taken both the actor
and the critic improve
at their respective
roles
•
actor is policy-based,
critic is value-based
20
LIGO-T2000291–vX
A
c
t
o
r
C
r
i
t
i
c
N
e
t
w
o
r
k
•
the policy maps states to actions
•
the value function finds expected reward for a state - judges
which actions were good or not
•
attempt to merge concepts of policy-based (better in continuous
action spaces) and value-based algorithms
•
idea is for the overall network to outperform the actor and critic
separately
21
LIGO-T2000291–vX
S
u
m
m
a
r
y
•
Neural network for nonlinear control will reach temperature set
point more quickly, with less over/under-shoot
•
Built a simple convolutional neural network to fit frequency
response of filter
•
Aiming to train a convolutional neural network for control
•
Built gym environment for temp control – no ambient
temperature fluctuation
•
Control with DQN converged on set point very slowly
•
Setting up Actor Critic as training algorithm
22
LIGO-T2000291–vX
G
o
i
n
g
f
o
r
w
a
r
d
•
Get actor critic network up and running in the gym environment
•
Increase the complexity of both the environment and the
network
•
Add ambient temperature fluctuation to training
•
Fine tune the network, e.g. adjust hyperparameters for best
performance
•
Run network control in simulation to evaluate efficacy
•
Apply neural network controller to other scenarios in LIGO
23
LIGO-T2000291–vX
A
c
k
n
o
w
l
e
d
g
e
m
e
n
t
s
•
Thank you to Rana, Koji and Tega for your supervision & all
your help throughout
•
Thank you also to Sanika and Rushabha, it’s been really fun
working with you guys!
24
LIGO-T2000291–vX
Utilizing neural networks, this project aims to enhance seismometer temperature stabilization by implementing nonlinear control to address system nonlinearities. The goal is to improve control performance, decrease overshoot, and allow adaptability to unpredictable parameters. The implementation of a neural network controller offers advantages such as better disturbance rejection and faster convergence, setting the groundwork for nonlinear controllers in various stabilization applications within LIGO. The neural network design includes feedback mechanisms to maintain temperature stability within the seismometer enclosure, promoting accuracy and energy efficiency through self-learning capabilities.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Neural Network Control for Seismometer Temperature Stabilization Ella Buehner Gattis Supervised by Rana Adhikari, Koji Arai, Tega Edo LIGO-T2000291 vX 1
Introduction Building on linear temperature control with PID to develop nonlinear control using a neural network Real systems are nonlinear, so this will be better suited to adapt to nonlinearities such as radiative losses Ability to deal with slow time variations and unpredictable parameters Decrease overshoot and undershoot LIGO-T2000291 vX 2
Motivation Increase overall performance of controller Nonlinear control has better disturbance rejection, faster set point convergence Training a neural network controller for temperature stabilization will then be applicable across many other stabilization problems in LIGO Mirrors, pendulum stabilization, etc Sets premise for nonlinear controllers throughout LIGO LIGO-T2000291 vX 3
Objectives Design feedback system to keep temperature constant within seismometer enclosure Non-linear control for better accuracy Neural Network control has ability for self-learning and may also reduce control energy foam insulation stainless steel enclosure seismometer resistive heating mesh LIGO-T2000291 vX 4
How a Neural Network Works Many individual neurons connected in dense network Each neuron receives inputs multiplied by a weight Layers are made of rows of neurons LIGO-T2000291 vX 5
How a Neural Network Works Neuron outputs sum of values received multiplied by their respective weights Connections initially have randomly assigned weights and biases, which are adjusted in process of training ?1 ?1 ? ?2 ? ?2 ???? ?=1 ?3 ?3 LIGO-T2000291 vX 6
How a Neural Network Works Network iterates over training set, adjusting weights until desired output is received for a particular input Maps inputs to outputs Train by supervised learning or reinforcement learning LIGO-T2000291 vX 7
Supervised Learning input training data neural network adjust weights output difference ideal output training data LIGO-T2000291 vX 8
Initial Neural Network To better understand the concept, first built a simple Convolutional Neural Network to imitate the output of a digital filter Supervised learning Tensorflow, Keras Digital filter applied to random noise using second order sections Grey noisy signal is input of network Blue smoothed signal is the target output of network LIGO-T2000291 vX 9
Initial Neural Network Labelled set of input (random noise) and output (filtered signal) As network goes through more training iterations, the weights and biases are adjusted to achieve the desired output Input shape matters! (1, data shape, 1) (batch size, number of steps, channels) LIGO-T2000291 vX 10
Convolutional Network Structure Convolutional network adapts to broader pattern across data set Convolution layer searches for features across a particular window , acts like a filter Takes element-wise product of filter and input Also need a convolutional neural network for optimum temperature control LIGO-T2000291 vX 11
Convolutional Network Structure window size 3 input time extracted features dot prod w/ weights output LIGO-T2000291 vX 12
Simple neural network loss is defined here by mean squared error able to achieve very high accuracy after around 220 training epochs LIGO-T2000291 vX 13
Target and real network output almost completely overlap LIGO-T2000291 vX 14
PSD of neural network output and target output LIGO-T2000291 vX 15
Supervised Learning vs Reinforcement Learning input training data agent neural network reward state action adjust weights environment output difference ideal output training data LIGO-T2000291 vX 16
Temperature control neural network Also use convolutional neural network Control: interacts directly with environment- need to use reinforcement learning Whereas supervised learning uses a labelled data set of input and desired output to train, reinforcement learning generates its own labels the reward Need continuous action space (output will be continuous power supplied to heater) Actor critic most suitable LIGO-T2000291 vX 17
Reinforcement learning: OpenAI gym Several different environments available to train networks with game-like scenarios In our case want to train network for temperature stabilization Built custom environment for this based on pendulum example (both are stabilization problems) stabilizing an inverted pendulum LIGO-T2000291 vX 18
Actor Critic Network actor selects best action based on the current state critic estimates the value function of the action observation action Q-value observation action actor critic LIGO-T2000291 vX 19
Actor Critic Network take a single step, for which a reward and the next state are obtained as more steps are taken both the actor and the critic improve at their respective roles actor is policy-based, critic is value-based LIGO-T2000291 vX 20
Actor Critic Network the policy maps states to actions the value function finds expected reward for a state - judges which actions were good or not attempt to merge concepts of policy-based (better in continuous action spaces) and value-based algorithms idea is for the overall network to outperform the actor and critic separately LIGO-T2000291 vX 21
Summary Neural network for nonlinear control will reach temperature set point more quickly, with less over/under-shoot Built a simple convolutional neural network to fit frequency response of filter Aiming to train a convolutional neural network for control Built gym environment for temp control no ambient temperature fluctuation Control with DQN converged on set point very slowly Setting up Actor Critic as training algorithm LIGO-T2000291 vX 22
Going forward Get actor critic network up and running in the gym environment Increase the complexity of both the environment and the network Add ambient temperature fluctuation to training Fine tune the network, e.g. adjust hyperparameters for best performance Run network control in simulation to evaluate efficacy Apply neural network controller to other scenarios in LIGO LIGO-T2000291 vX 23
Acknowledgements Thank you to Rana, Koji and Tega for your supervision & all your help throughout Thank you also to Sanika and Rushabha, it s been really fun working with you guys! LIGO-T2000291 vX 24
