Accelerator Safety Systems and Controls Overview
undefined
Interlocks, Access
Controls, and
Configuration
Management:
JLab and SNS
Rev. 1
2014 Accelerator Safety Workshop
Kelly Mahoney,
mahoneykl@ornl.gov
ORNL/SNS
(Nee Jlab)
A Tale of Two Accelerator Facilities
J
L
a
b
•
Single Purpose Lab
•
JLab Mission: Nuclear Physics
•
Recirculating Linac
–
Superconducting
–
CW (Duty Factor 100%)
•
Recent Milestones:
–
10.5 GeV beam
–
CD4A Completion of Accelerator and
Experimental Hall D
S
N
S
•
Multipurpose Lab (ORNL)
•
SNS Mission: Neutron Science
•
Linac
Accumulator Ring
–
Warm RFQ/DTL; SC Linac
–
Pulsed (Linac Duty Factor ~ 6%)
•
Recent Milestones:
–
Sustained 1.4 MW Operation
–
Completion of baseline instrument suite
A Tale of Two Accelerator Facilities
J
L
a
b
•
Beam:
e
-
•
Energy:
12 GeV
•
Power:
1 MW
•
Fixed Target
•
Experiment Beamlines: 4
•
Long-term Experiments
•
Future:
–
12GeV Commissioning/Operations
–
ELIC Design
S
N
S
•
Beam:
H
-
H
+
N
•
Energy:
1 GeV
•
Power:
1.4 MW
•
Fixed Target
•
Experiment Beamlines: 18
•
Short-term Experiments
•
Future
–
High Power (Flux) operations
–
Ion Source Upgrades
–
Inner Reflector Plug Replacement
–
STS Design
Common Approach to Safety Systems
•
Accelerator Safety Envelope Credited Controls and Defense in Depth
Controls
•
Primarily active engineering controls for ionizing radiation safety
•
Additional Safety Functions Include:
–
Oxygen Deficiency Monitoring
–
Non-ionizing radiation
–
High Voltage Interlocks
–
Magnet Interlocks
–
Explosive Gas Monitoring
–
Beam Envelope Monitoring
•
PLC Based Safety Systems
–
Older areas Industrial PLCs
–
New units use Safety PLCs
•
New SNS Instruments use ‘one-box’ safety PLCs
SNS Specific
•
Environmental Monitoring
–
Transfer Bay Differential Pressure
–
Stack Radiation Monitoring
•
Toxic Gas Monitoring
•
Target Protection System
–
Process Controls
SNS lHg Target in Hot Cell
SNS 18 Primary Neutron Shutters
SNS H
2
Used in Experiment
SNS Stack
Common
Concerns
•
Management
of safety systems other than credited controls
–
Excluded/Exempt Systems; Systems other than radiological protection
•
Rise to level of defense-in-depth at most
–
Still perform safety
functions
•
Must meet regulatory and best practices
–
Tend to not
get the level of attention of Credited Controls (CCs)
•
May
be included in FSAD but not ASE
•
Not
accountable through the ASO/ASE Process
–
Internal configuration control process must be able to manage these systems
–
Examples:
•
Non-Accelerator Test Facilities
•
Non-ionizing equipment facilities
–
Laser Labs
–
High Power RF
•
High Field Magnets
•
Hazardous/Explosive Gases not managed under ASE Process
•
Environmental Compliance Monitoring
Common Concerns
-
Cont’d
•
Management of Safety System Configuration During
Construction, D&D, and Major Maintenance
–
Problem:
•
Intentional change to accelerator systems, facilities, and
infrastructure (SFI); Unintentional change to safety systems
.
•
Construction/D&D to upgrade existing facilities, systems
containing safety systems
–
Presumption on workers’ part that safety systems can be altered
with other infrastructure during construction or D&D
–
Gap in work planning information and communication
•
Configuration management (Integrity Management) must
ensure continuity of safety system integrity throughout
lifecycle
–
What lifecycle model includes mini D&D !!?
•
Not adequately addressed in standards and guidance
•
Collection minor incidents at JLab and SNS
Photos of Config
Concerns
Note: None of the below resulted in
reduced safety. All were during
major outage. The concern was the
behavior and the trend.
Clockwise from upper left:
Trapped Key - mating gate removed
Typical safety interface to field
equipment among
non-safety
connections.
Safety switches cast in concrete
Safety message display in trash bin
ODH Sensor bagged during
painting. (no gases in area yet)
Others:
Changes to electrical feeds to
Safety System components.
Defacto removal of
configuration control locks
when locked device is
removed.
Common Concerns
-
Cont’d
•
Management of Safety System Configuration During
Construction, D&D, and Major Maintenance
–
Solutions:
•
Customized training; re-enforced awareness
•
Recognition for appropriate behavior
•
Integrated work planning tools
•
Escalating consequences based on combination of potential
consequence, worker experience,
egregiousness, …
–
i.e. treated like an EH&S violation
Common Concerns – Cont’d
•
Requirements flow down/flow up
–
Change management must assess impact at requirements level
•
Requirements are the most likely source of common mode failures
•
Validate Assumptions, Hazard Analysis, Interfaces,…
–
Difficult to track changes, impacts
–
Ideally should track high level through to implementation, test,
operation, and management systems
–
RM tools like DOORS are expensive and require large learning curve
•
Solution (SNS)
–
Expanding capability of existing Project Lifecycle Management (PLM)
tools to include document relationships and hierarchy.
(JLab was starting this effort in 2014)
Other Concerns – Controversial Stuff
Active Engineered Controls
•
Use of Machine Safety Standard
s and Guidance does not adequately
address
Institutional
Risk
of large accelerator operations
–
Plus
•
Directly applicable to interlocks
•
Large body of knowledge
•
‘Cookie cutter’ solutions
•
Does not require dedicated safety professional
–
Minus
•
‘Dumbed down’ implementation of safety instrumented functions
•
Does not adequately address complex system of systems
•
Does not address high reliability systems management
•
Does not require dedicated safety professional
•
R
e
c
o
m
m
e
n
d
c
o
n
t
i
n
u
e
d
i
n
t
e
g
r
a
t
i
o
n
o
f
D
O
E
,
P
r
o
c
e
s
s
S
a
f
e
t
y
,
A
e
r
o
s
p
a
c
e
,
a
n
d
I
n
t
e
r
n
a
t
i
o
n
a
l
N
u
c
l
e
a
r
s
t
a
n
d
a
r
d
s
a
n
d
p
r
a
c
t
i
c
e
s
•
F
o
r
I
n
s
t
i
t
u
t
i
o
n
a
l
R
i
s
k
,
S
e
e
S
u
p
p
l
e
m
e
n
t
a
r
y
s
l
i
d
e
RISK
MITIGATION
Reliability
Models
O
L
D
•
Safety system management
follows a lifecycle from concept
to retirement…
–
Defined set of processes with
outcomes
–
Risk is quantified
•
Residual Risk Recorded and
Accepted before Operations
–
Risk is not considered
dynamic
N
E
W
•
Safety system management is
like a control system with
feedback…
–
Continuous change
–
Perturbations to the system
create vulnerability or risk
•
Human Performance
•
Regulatory Environment
•
Resource quality and availability
–
Controls must detect
perturbations and correct
problem before an accident
–
Better fits realistic accident
models
New Approach to Interlocks
•
Models:
–
1990 – 2000:
Central Processing/Concentrated I/O
–
2000 – 2012:
Central Processing/Distributed I/O
–
2012 – 2014:
Distributed Processing/Distributed I/O
–
2014+
Distributed Safety Instrumented
Subsystems
Summary
•
Experience at two accelerator facilities show there
are common concerns related to
–
Integrity management
during construction/D&D
–
Management of non-CECs
–
Requirements management
•
Topics for Conversation
–
Similar
Experiences and Solutions
–
Future Safety System Architectures
–
New Reliability Models
–
New Approach to Safety System Architectures
Backup Slides
Institutional Risk Assessment
•
New
Risk Analysis Based on Multiple Risk Categories
–
Direct Risk of Harm to People
–
Direct Risk of Harm to Environment
–
Direct Risk of Harm to Tangible Property
–
Direct Risk of Financial
Loss
–
Direct Risk of Impact to Continuity of Operations/Mission
–
Direct Risk of Enforcement Action
•
Each area is scored individually as well as summed
•
Data shows that items scoring low risk in individual categories can add
up to significant cumulative risk
Distributed Safety Instrumented
Subsystems
•
Autonomous functions
•
Simplified hardware, software, configuration management
•
Easier
implementation of graded approach (SIL1-3 as needed)
•
‘Black Box’ inputs, outputs, interface
•
Scalable from very small to very large facilities
•
JLab Uses
–
Beam Envelope
Limit System (BELS)
–
Injection Magnet Monitoring
–
Beam Transport Monitor (BTM)
•
Future Direction
•
Integrated Equipment Interlocks (SIL 2/3 Built-in to CCs)
•
Semi-autonomous interlocks and access controls
•
Multiple functional redundancy, checks, communication
Achieves Simultaneous:
•
High Safety Availability
•
High Reliability
•
Fault Tolerance
•
On-Line Self Test
•
On-line Maintenance
Single Safety PLC
•
Single CPU (internally redundant architecture)
–
Redundant I/O Count remains unchanged
–
Should still use redundant power supplies
–
Plus
•
Easier configuration management
•
Able to safely pass information between A/B Programs
•
Able to take advantage of built-in safety functions like evaluation of
redundant switches, Output-Feedback evaluation,
–
Minus
•
Single Mode vulnerability to issues like network errors
Some Architectures
Key to Above Slide
SIL1 = Average Probability of Dangerous Failure
10
0
to
10
-1
SIL2 = Average Probability of Dangerous Failure
10
-1
to 10
-2
SIL3 = Average Probability of Dangerous Failure
10
-2
to 10
-3
I/O Configuration
1oo1, 1oo2, 2oo3 = m out of n
n = # of systems (redundancy)
m = # required to trip
D = Fail-safe diagnostic coverage over 99%. No ‘D’ assumes
diagnostic coverage between 60 and 90 %.
* - ODH is a monitoring and alarm function. It may not act to
automatically
mitigate the hazard.
NIF Target Cell
Enterprise Warp Core – The Ultimate
Repurposing!
This document presents an insightful overview of the safety systems, controls, and management approaches implemented at accelerator facilities such as JLab and SNS. It covers topics including interlocks, access controls, configuration management, safety envelopes, environmental monitoring, and common safety concerns. The content emphasizes the importance of active engineering controls, defense in depth strategies, and specific safety functions like oxygen deficiency monitoring and beam envelope monitoring. Illustrative images and descriptions enhance the understanding of safety measures and operational milestones achieved at these facilities.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Interlocks, Access Controls, and Configuration Management: JLab and SNS Rev. 1 2014 Accelerator Safety Workshop Kelly Mahoney, mahoneykl@ornl.gov ORNL/SNS (Nee Jlab) ORNL is managed by UT-Battelle for the US Department of Energy
A Tale of Two Accelerator Facilities JLab SNS Single Purpose Lab Multipurpose Lab (ORNL) JLab Mission: Nuclear Physics SNS Mission: Neutron Science Recirculating Linac Superconducting CW (Duty Factor 100%) Linac Accumulator Ring Warm RFQ/DTL; SC Linac Pulsed (Linac Duty Factor ~ 6%) Recent Milestones: 10.5 GeV beam CD4A Completion of Accelerator and Experimental Hall D Recent Milestones: Sustained 1.4 MW Operation Completion of baseline instrument suite 2 ASW-2104: Experience JLab & SNS
A Tale of Two Accelerator Facilities JLab SNS Beam: e- Beam: H- H+ N Energy: 12 GeV Energy: 1 GeV Power: 1 MW Power: 1.4 MW Fixed Target Fixed Target Experiment Beamlines: 4 Experiment Beamlines: 18 Long-term Experiments Short-term Experiments Future: 12GeV Commissioning/Operations ELIC Design Future High Power (Flux) operations Ion Source Upgrades Inner Reflector Plug Replacement STS Design 3 ASW-2104: Experience JLab & SNS
Common Approach to Safety Systems Accelerator Safety Envelope Credited Controls and Defense in Depth Controls Primarily active engineering controls for ionizing radiation safety Additional Safety Functions Include: Oxygen Deficiency Monitoring Non-ionizing radiation High Voltage Interlocks Magnet Interlocks Explosive Gas Monitoring Beam Envelope Monitoring PLC Based Safety Systems Older areas Industrial PLCs New units use Safety PLCs New SNS Instruments use one-box safety PLCs 4 ASW-2104: Experience JLab & SNS
SNS Specific Environmental Monitoring Transfer Bay Differential Pressure Stack Radiation Monitoring Toxic Gas Monitoring Target Protection System Process Controls SNS Stack SNS H2 Used in Experiment SNS lHg Target in Hot Cell SNS 18 Primary Neutron Shutters 5 ASW-2104: Experience JLab & SNS
Common Concerns Management of safety systems other than credited controls Excluded/Exempt Systems; Systems other than radiological protection Rise to level of defense-in-depth at most Still perform safety functions Must meet regulatory and best practices Tend to not get the level of attention of Credited Controls (CCs) May be included in FSAD but not ASE Not accountable through the ASO/ASE Process Internal configuration control process must be able to manage these systems Examples: Non-Accelerator Test Facilities Non-ionizing equipment facilities Laser Labs High Power RF High Field Magnets Hazardous/Explosive Gases not managed under ASE Process Environmental Compliance Monitoring 6 ASW-2104: Experience JLab & SNS
Common Concerns - Contd Management of Safety System Configuration During Construction, D&D, and Major Maintenance Problem: Intentional change to accelerator systems, facilities, and infrastructure (SFI); Unintentional change to safety systems. Construction/D&D to upgrade existing facilities, systems containing safety systems Presumption on workers part that safety systems can be altered with other infrastructure during construction or D&D Gap in work planning information and communication Configuration management (Integrity Management) must ensure continuity of safety system integrity throughout lifecycle What lifecycle model includes mini D&D !!? Not adequately addressed in standards and guidance Collection minor incidents at JLab and SNS 7 ASW-2104: Experience JLab & SNS
Note: None of the below resulted in reduced safety. All were during major outage. The concern was the behavior and the trend. Photos of Config Concerns Clockwise from upper left: Trapped Key - mating gate removed Typical safety interface to field equipment among non-safety connections. Safety switches cast in concrete Safety message display in trash bin ODH Sensor bagged during painting. (no gases in area yet) Others: Changes to electrical feeds to Safety System components. Defacto removal of configuration control locks when locked device is removed. 8 ASW-2104: Experience JLab & SNS
Common Concerns - Contd Management of Safety System Configuration During Construction, D&D, and Major Maintenance Solutions: Customized training; re-enforced awareness Recognition for appropriate behavior Integrated work planning tools Escalating consequences based on combination of potential consequence, worker experience, egregiousness, i.e. treated like an EH&S violation 9 ASW-2104: Experience JLab & SNS
Common Concerns Contd Requirements flow down/flow up Change management must assess impact at requirements level Requirements are the most likely source of common mode failures Validate Assumptions, Hazard Analysis, Interfaces, Difficult to track changes, impacts Ideally should track high level through to implementation, test, operation, and management systems RM tools like DOORS are expensive and require large learning curve Solution (SNS) Expanding capability of existing Project Lifecycle Management (PLM) tools to include document relationships and hierarchy. (JLab was starting this effort in 2014) 10 ASW-2104: Experience JLab & SNS
Other Concerns Controversial Stuff Active Engineered Controls Use of Machine Safety Standards and Guidance does not adequately address Institutional Risk of large accelerator operations Plus Directly applicable to interlocks Large body of knowledge Cookie cutter solutions Does not require dedicated safety professional Minus Dumbed down implementation of safety instrumented functions Does not adequately address complex system of systems Does not address high reliability systems management Does not require dedicated safety professional MITIGATION RISK Recommend continued integration of DOE, Process Safety, Aerospace, and International Nuclear standards and practices For Institutional Risk, See Supplementary slide 11 ASW-2104: Experience JLab & SNS
Reliability Models OLD Safety system management follows a lifecycle from concept to retirement Defined set of processes with outcomes Risk is quantified Residual Risk Recorded and Accepted before Operations Risk is not considered dynamic NEW Safety system management is like a control system with feedback Continuous change Perturbations to the system create vulnerability or risk Human Performance Regulatory Environment Resource quality and availability Controls must detect perturbations and correct problem before an accident Better fits realistic accident models 12 ASW-2104: Experience JLab & SNS
New Approach to Interlocks Models: 1990 2000: Central Processing/Concentrated I/O 2000 2012: Central Processing/Distributed I/O 2012 2014: Distributed Processing/Distributed I/O 2014+ Distributed Safety Instrumented Subsystems 13 ASW-2104: Experience JLab & SNS
Summary Experience at two accelerator facilities show there are common concerns related to Integrity management during construction/D&D Management of non-CECs Requirements management Topics for Conversation Similar Experiences and Solutions Future Safety System Architectures New Reliability Models New Approach to Safety System Architectures 14 ASW-2104: Experience JLab & SNS
Backup Slides 15 ASW-2104: Experience JLab & SNS
Institutional Risk Assessment New Risk Analysis Based on Multiple Risk Categories Direct Risk of Harm to People Direct Risk of Harm to Environment Direct Risk of Harm to Tangible Property Direct Risk of Financial Loss Direct Risk of Impact to Continuity of Operations/Mission Direct Risk of Enforcement Action Each area is scored individually as well as summed Data shows that items scoring low risk in individual categories can add up to significant cumulative risk Any Individual Score (IS) Risk Acceptance Level Combined Score (CS) Intolerable Unacceptable Tolerable Acceptable > 10 4 2 < IS < 4 1 < IS 2 1 5 < CS 10 1 < CS 5 1 16 ASW-2104: Experience JLab & SNS
Distributed Safety Instrumented Subsystems Autonomous functions Simplified hardware, software, configuration management Easier implementation of graded approach (SIL1-3 as needed) Black Box inputs, outputs, interface Scalable from very small to very large facilities Achieves Simultaneous: High Safety Availability High Reliability Fault Tolerance On-Line Self Test On-line Maintenance JLab Uses Beam Envelope Limit System (BELS) Injection Magnet Monitoring Beam Transport Monitor (BTM) Future Direction Integrated Equipment Interlocks (SIL 2/3 Built-in to CCs) Semi-autonomous interlocks and access controls 17 ASW-2104: Experience JLab & SNS Multiple functional redundancy, checks, communication
Single Safety PLC Single CPU (internally redundant architecture) Redundant I/O Count remains unchanged Should still use redundant power supplies Plus Easier configuration management Able to safely pass information between A/B Programs Able to take advantage of built-in safety functions like evaluation of redundant switches, Output-Feedback evaluation, Minus Single Mode vulnerability to issues like network errors 18 ASW-2104: Experience JLab & SNS
Some Architectures Accelerator Function IEC 61508 SIL Capability CPU I/O Software Configuration Configuration Configuration (Trip Conditions) SNS Instrument Interlocks 2 1 Safety 1oo2 Two Programs Fermi ODH 1 Safety 1oo1* One Program JLab ODH 2 1 Safety 1oo1D* One Program JLab BELS 2 2 Standard 2oo3 Two Programs JLab BTM 2 (Capable of 3) 1 Safety 2oo3D One Program JLab Access Controls/ Interlocks Radiation Interlocks 3 2 Safety 1oo2D Two Programs CERN 3 2 Safety 2oo2D (+HW Chain) One Program 19 ASW-2104: Experience JLab & SNS
Key to Above Slide SIL1 = Average Probability of Dangerous Failure SIL2 = Average Probability of Dangerous Failure SIL3 = Average Probability of Dangerous Failure 100 to 10-1 10-1 to 10-2 10-2 to 10-3 I/O Configuration 1oo1, 1oo2, 2oo3 = m out of n n = # of systems (redundancy) m = # required to trip D = Fail-safe diagnostic coverage over 99%. No D assumes diagnostic coverage between 60 and 90 %. * - ODH is a monitoring and alarm function. It may not act to automatically mitigate the hazard. 20 ASW-2104: Experience JLab & SNS
NIF Target Cell 21 ASW-2104: Experience JLab & SNS
Enterprise Warp Core The Ultimate Repurposing! 22 ASW-2104: Experience JLab & SNS
