Conducting Surveillance of CNS Providers

  
CHAPTER 2
CHAPTER 2
Conducting Surveillance of CNS
Conducting Surveillance of CNS
Providers
Providers
Definitions
Definitions
 
 
Surveillance:
Surveillance:
 A process through which the State proactively verifies
 A process through which the State proactively verifies
through inspections, audits and other activities that aviation licence,
through inspections, audits and other activities that aviation licence,
certificate, authorization or approval holders continue to meet the
certificate, authorization or approval holders continue to meet the
established requirements and function at the level of competency
established requirements and function at the level of competency
and safety required by the State.
and safety required by the State.
Audit: 
Audit: 
An in-depth review of the activities of an organization to verify
An in-depth review of the activities of an organization to verify
conformance to regulatory requirements.
conformance to regulatory requirements.
Inspection: 
Inspection: 
The basic activity of an audit, involving the systematic
The basic activity of an audit, involving the systematic
assessment of a specific characteristic of an organization to verify
assessment of a specific characteristic of an organization to verify
conformance to regulatory requirements or company approved
conformance to regulatory requirements or company approved
procedures.
procedures.
         
Overview
Overview
 
Audits/inspections of CNS service providers enable the CNS
Audits/inspections of CNS service providers enable the CNS
Inspectorate to:
Inspectorate to:
verify the CNS Providers' level of compliance with
verify the CNS Providers' level of compliance with
requirements.
requirements.
validate that preventive controls established by the CNS
validate that preventive controls established by the CNS
provider to mitigate risks are effective.
provider to mitigate risks are effective.
CAA therefore uses audits to answer the question, 
CAA therefore uses audits to answer the question, 
“Is it
“Is it
working?”
working?”
Audits may be systemic or targeted to specific facilities or
Audits may be systemic or targeted to specific facilities or
stations, and
stations, and
Audits may be conducted on-site or virtually, and include
Audits may be conducted on-site or virtually, and include
the review of documents, records, and other data provided
the review of documents, records, and other data provided
by the ANSP.
by the ANSP.
         
Overview
Overview
 
Safety Audits/inspections;
Safety Audits/inspections;
Provide the oversight organization and other authorities
Provide the oversight organization and other authorities
with evidence of ANSP’s compliance with applicable safety
with evidence of ANSP’s compliance with applicable safety
processes, regulatory requirements, and implementation
processes, regulatory requirements, and implementation
arrangements, and attempt to identify the need for
arrangements, and attempt to identify the need for
improvement or corrective action;
improvement or corrective action;
Are independent of CNS provider’s internal safety audits
Are independent of CNS provider’s internal safety audits
undertaken as part of its safety, quality control, quality
undertaken as part of its safety, quality control, quality
assurance, and/or quality management systems;
assurance, and/or quality management systems;
Are conducted by qualified CNS oversight inspectors; and
Are conducted by qualified CNS oversight inspectors; and
Lead to the correction of any identified non-compliance
Lead to the correction of any identified non-compliance
issues.
issues.
         
TYPES OF AUDIT
TYPES OF AUDIT
 
Pre-certification Audit/inspection (phase 4);
Post certification/routine conformance
Audit/inspection;
Additional authority Audit/inspection;
Special-purpose Audit/inspection.
         
TYPES OF AUDIT
TYPES OF AUDIT
 
Pre-certification Audit/Inspection
Pre-certification Audit/Inspection
Pre-certification Audit/inspection will be
conducted prior to the issuance of the certificate.
This occurs at the fourth stage- Demonstration and
inspection phases of the ANSP certification
process.
         
TYPES OF AUDIT
TYPES OF AUDIT
 
Post certification/routine conformance
Post certification/routine conformance
Inspection/Audit;
Inspection/Audit;
A Post certification/ routine conformance
Audit/inspection will be conducted for the
purpose of determining an organization’s overall
level of conformance to regulatory requirements
after the ANSP have been certificated. All
applicable characteristics of the organization will
be subject to review.
         
TYPES OF AUDIT
TYPES OF AUDIT
 
Additional authority Audit/Inspection
Additional authority Audit/Inspection
An additional authority Audit/inspection may be
conducted prior to the granting of an additional
authority. Prior notification to the organization is
not required.
         
TYPES OF AUDIT
TYPES OF AUDIT
 
Special-purpose Audit/Inspection
Special-purpose Audit/Inspection
A special-purpose Audit/Inspection is one conducted to
respond to safety concerns or circumstances other than
those requiring a pre-certification Audit/inspection, an
additional authority Inspection/Audit or a routine
conformance Audit/inspection.
A special purpose audit may be convened with little or
no notice arising from safety concern (e.g. Adhoc
surveillance).
A special purpose Audit/inspection may preclude certain
team-member activities and responsibilities that would be
normally associated with other types of Audit/inspections.
   
Audit/Inspection Phases
Audit/Inspection Phases
 
THREE distinct phases:
Pre-Audit/Inspection;
Physical Audit/Inspection;
Post- Audit/Inspection; and
Audit/Inspection follow-up
   
Audit/Inspection Phases
Audit/Inspection Phases
 
Pre-Audit/Inspection Phase
Pre-Audit/Inspection Phase
Pre-Audit/Inspection activities involve
notifying the organization, selecting an
Audit/Inspection team, developing an
Audit/Inspection plan, reviewing files and
related documentation and convening
the pre- Audit/Inspection team meeting.
   
Audit/Inspection Phases
Audit/Inspection Phases
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
The physical Audit/Inspection phase will be
implemented in accordance with the Audit/Inspection
plan.
The purpose of this phase is to verify conformance with
regulatory requirements and to assign findings where
non-conformance has been confirmed.
Audit/Inspection results will be communicated to the
organization under review at daily meetings and/or the
exit meeting.
Once an Audit/Inspection has commenced, it should
not be terminated without approval of the Head of the
inspectorate
   
Audit/Inspection Phases
Audit/Inspection Phases
 
Post- Audit/Inspection & follow up Phase
Post- Audit/Inspection & follow up Phase
Post-Audit/Inspection activities include
completion of administrative details and
production of the Audit/Inspection report
 
Planning a CNS Inspection
Planning a CNS Inspection
 
Audit/Inspection Programme
Audit/Inspection Programme
The ANS- CNS oversight department is required to establish
The ANS- CNS oversight department is required to establish
and implement, a surveillance programme which should
and implement, a surveillance programme which should
include, at a minimum, the elements below;
include, at a minimum, the elements below;
the types of surveillance activities (e.g. audits, inspections,
the types of surveillance activities (e.g. audits, inspections,
tests, safety events analyses);
tests, safety events analyses);
the timeframe or frequency of the activities;
the timeframe or frequency of the activities;
items to be covered or scope of the activities; and
items to be covered or scope of the activities; and
related methodology/procedures, job aids and guidance
related methodology/procedures, job aids and guidance
on how the activity should be conducted, starting from
on how the activity should be conducted, starting from
the notification of the service provider, if applicable, to
the notification of the service provider, if applicable, to
the closure of the deficiencies identified during the
the closure of the deficiencies identified during the
activities
activities
 
Planning a CNS Inspection
 
Audit/Inspection programme
Audit/Inspection programme
The oversight audit program should;
The oversight audit program should;
Enable audit/inspection of areas of potential
Enable audit/inspection of areas of potential
safety concern with the highest identified risk, or
safety concern with the highest identified risk, or
with a focus on those areas where problems have
with a focus on those areas where problems have
been identified;
been identified;
Cover all the functions/stations within CNS
Cover all the functions/stations within CNS
provider organizations, services and network;
provider organizations, services and network;
Ensure that audits are conducted in a manner
Ensure that audits are conducted in a manner
commensurate to the level of risk posed by the
commensurate to the level of risk posed by the
CNS provider’s activities;
CNS provider’s activities;
 
 
Planning a CNS Inspection
Planning a CNS Inspection
 
Audit/Inspection programme
Audit/Inspection programme
The oversight audit program should;
The oversight audit program should;
Ensure that sufficient audits are conducted on a
Ensure that sufficient audits are conducted on a
consistent basis, to check the compliance of all
consistent basis, to check the compliance of all
CNS organizations, as pertains to applicable safety
CNS organizations, as pertains to applicable safety
processes, and regulatory requirements in all the
processes, and regulatory requirements in all the
relevant areas of the functional system; and,
relevant areas of the functional system; and,
Enable conduct of follow-up activities to ensure
Enable conduct of follow-up activities to ensure
effective implementation of corrective actions.
effective implementation of corrective actions.
 
Planning a CNS Inspection
Planning a CNS Inspection
 
Elements of surveillance Program
Elements of surveillance Program
a)
The CNS provider/Station to be audited
The CNS provider/Station to be audited
b)
Audit/inspection type
Audit/inspection type
c)
The team composition
The team composition
d)
Budget
Budget
e)
Audit/inspection dates
Audit/inspection dates
 
Planning a CNS Inspection
Planning a CNS Inspection
 
Factors that influence the scope and depth of a
Factors that influence the scope and depth of a
surveillance programme
surveillance programme
type of the Inspection/Audit;
type of the Inspection/Audit;
the period back in time that systems are subject to review
the period back in time that systems are subject to review
(typically from the last Inspection/Audit to the present);
(typically from the last Inspection/Audit to the present);
the enforcement record of the organization;
the enforcement record of the organization;
the confidence in corrective actions taken by the
the confidence in corrective actions taken by the
organization as a result of a previous Inspection/Audit;
organization as a result of a previous Inspection/Audit;
and
and
Inspector/Auditor and financial resources available.
Inspector/Auditor and financial resources available.
                
                
Handout H2.1
Handout H2.1
 
 
Planning a CNS Inspection
Planning a CNS Inspection
 
Audit Plans
Audit Plans
For each individual CNS inspector approved to carry out
For each individual CNS inspector approved to carry out
aviation activities, a periodic surveillance plan should be
aviation activities, a periodic surveillance plan should be
developed based on the applicable surveillance
developed based on the applicable surveillance
programme.
programme.
For each surveillance activity on the service provider, a
For each surveillance activity on the service provider, a
surveillance plan should be developed
surveillance plan should be developed
The surveillance plan should detail the type of activities to be
The surveillance plan should detail the type of activities to be
performed and the specified timeframe as well as the scope
performed and the specified timeframe as well as the scope
of each activity, as applicable.
of each activity, as applicable.
The purpose of the safety oversight audit plan is to outline the
The purpose of the safety oversight audit plan is to outline the
sequential process of the on-site audit and provide the CNS
sequential process of the on-site audit and provide the CNS
provider with valuable information on the conduct of the
provider with valuable information on the conduct of the
audit
audit
 
Planning a CNS Inspection
Planning a CNS Inspection
 
Contents of an audit plan
Contents of an audit plan
dates of the on-site/virtual safety oversight audit;
dates of the on-site/virtual safety oversight audit;
objectives of the safety oversight audit;
objectives of the safety oversight audit;
scheduled dates for opening and closing meetings;
scheduled dates for opening and closing meetings;
identification of the CNS provider’s key personnel
identification of the CNS provider’s key personnel
(safety oversight coordinator would have been
(safety oversight coordinator would have been
already appointed and acknowledged to Authority);
already appointed and acknowledged to Authority);
scope of the on-site safety oversight audit to be
scope of the on-site safety oversight audit to be
conducted and audit areas to be covered;
conducted and audit areas to be covered;
identification of documents necessary to conduct the
identification of documents necessary to conduct the
safety oversight audit;
safety oversight audit;
 
Planning a CNS Inspection
 
Contents of an audit plan
Contents of an audit plan
travel and administration plans (security clearances, health
travel and administration plans (security clearances, health
requirements, etc.) as well as the travel schedules of audit
requirements, etc.) as well as the travel schedules of audit
members;
members;
the language to be used for the safety oversight audit, the
the language to be used for the safety oversight audit, the
audit preparation briefing material for team members and
audit preparation briefing material for team members and
the on-site audit mission package;
the on-site audit mission package;
identification of priority issues brought to the attention of
identification of priority issues brought to the attention of
ATS/SAR provider from reviewing material   provided and
ATS/SAR provider from reviewing material   provided and
other documents available to CAA or information received
other documents available to CAA or information received
on the status of Services
on the status of Services
Team members’ assignments and responsibilities.
Team members’ assignments and responsibilities.
                
                
Handout H 2.2
Handout H 2.2
 
Pre-Audit Activities
Pre-Audit Activities
 
Pre-Audit Activities
Pre-Audit Activities
 
Audit Research
Audit Research
Throughout the audit preparation process, audit team members should
Throughout the audit preparation process, audit team members should
thoroughly research the upcoming audit to ensure that all applicable
thoroughly research the upcoming audit to ensure that all applicable
requirements have been captured, and that they have not changed.
requirements have been captured, and that they have not changed.
Reference may be made to the ICAO standards as well as respective State
Reference may be made to the ICAO standards as well as respective State
regulations and directives, standard operating procedures, corrective action
regulations and directives, standard operating procedures, corrective action
plans, etc.
plans, etc.
Important research questions include:
Important research questions include:
What initiated the audit?
What initiated the audit?
What are the results of any past audits on the audit area?
What are the results of any past audits on the audit area?
Are there enforcement actions against the service provider related to this
Are there enforcement actions against the service provider related to this
audit area?
audit area?
Which requirements are applicable to this audit area?
Which requirements are applicable to this audit area?
 
Audit/Inspection Plan
Audit/Inspection Plan
 
 
C
C
a
a
s
s
e
e
 
 
S
S
c
c
e
e
n
n
a
a
r
r
i
i
o
o
P
P
r
r
a
a
c
c
t
t
i
i
c
c
a
a
l
l
 
 
E
E
x
x
e
e
r
r
c
c
i
i
s
s
e
e
-
-
 
 
P
P
r
r
e
e
p
p
a
a
r
r
a
a
t
t
i
i
o
o
n
n
 
 
o
o
f
f
 
 
a
a
n
n
 
 
I
I
n
n
s
s
p
p
e
e
c
c
t
t
i
i
o
o
n
n
 
 
P
P
l
l
a
a
n
n
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
The physical Audit/Inspection phase will be
The physical Audit/Inspection phase will be
implemented in accordance with the
implemented in accordance with the
Audit/Inspection plan.
Audit/Inspection plan.
The purpose of this phase is to verify conformance
The purpose of this phase is to verify conformance
with regulatory requirements and to assign findings
with regulatory requirements and to assign findings
where non-conformance has been confirmed.
where non-conformance has been confirmed.
Audit/Inspection results will be communicated to
Audit/Inspection results will be communicated to
the organization under review at daily meetings
the organization under review at daily meetings
and/or the exit meeting.
and/or the exit meeting.
Once an Audit/Inspection has commenced, it
Once an Audit/Inspection has commenced, it
should not be terminated without approval of the
should not be terminated without approval of the
Head of the inspectorate
Head of the inspectorate
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Audit teams are expected to:
Audit teams are expected to:
conduct an opening meeting with facility
conduct an opening meeting with facility
management;
management;
collect data, analyze collected data and verify
collect data, analyze collected data and verify
that analysis;
that analysis;
record findings;
record findings;
conduct team briefings; and
conduct team briefings; and
conduct a closing meeting with facility
conduct a closing meeting with facility
management while conducting on-site audits.
management while conducting on-site audits.
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Opening Meeting
Opening Meeting
The Audit/Inspection team should conduct an
The Audit/Inspection team should conduct an
entry meeting on the first day of the
entry meeting on the first day of the
audit/inspection with the audited party so that
audit/inspection with the audited party so that
both parties have an opportunity to meet and
both parties have an opportunity to meet and
discuss on relevant details and arrangements.
discuss on relevant details and arrangements.
The entry meeting should set the tone for the
The entry meeting should set the tone for the
physical Audit/Inspection and should be attended
physical Audit/Inspection and should be attended
by the CNS provider’s senior management and all
by the CNS provider’s senior management and all
members of the Audit/Inspection team.
members of the Audit/Inspection team.
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Opening Meeting Elements
Opening Meeting Elements
Thank the Auditee for cooperation
Introduce yourself and team member(s) present
Explain the purpose of the Audit
Review objectives and principles of the audit
(responsibilities and duties of the Auditee and the audit
team).
Describe the audit process and methods of gathering
information (e.g. observation, discussion, review of
documents) during the audit.
Briefly present the audit plan with the objective to adjust
and confirm it.
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Opening Meeting Elements
Opening Meeting Elements
Confirm the arrangements for the post-audit debriefing
(location, participants)
Explain the reporting system, including the audit report and
the action plan
Identify who is designated by the Auditee to receive the
audit report
Be receptive to any special concerns from the Auditee with
regard to the performance of the auditor areas to be
observed
Confirm the location of the audit team facilities and any
other administrative details
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Evidence Gathering Techniques
Evidence Gathering Techniques
The following methods will be used to gather
The following methods will be used to gather
information
information
Detailed documentation review
Detailed documentation review
Random sampling of documentation
Random sampling of documentation
Interviews
Interviews
Observations
Observations
Checklists
Checklists
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
Evidence Gathering Techniques
Evidence Gathering Techniques
 
1. Interviewing CNS Provider’s Personnel
1. Interviewing CNS Provider’s Personnel
 
 
 
 
 
 
 
 
 
 
 
Figure 2: Types of interviews
Figure 2: Types of interviews
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Figure 3: In-depth Interviews
Figure 3: In-depth Interviews
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Interviewing The CNS Provider’s Personnel
Interviewing The CNS Provider’s Personnel
In-depth interviews are the best since they involve a one on
In-depth interviews are the best since they involve a one on
one interaction and designed to provide a detailed picture
one interaction and designed to provide a detailed picture
of the audited/inspected entity.
of the audited/inspected entity.
 
Responses given by the auditee should be documented
Responses given by the auditee should be documented
during or as soon as possible following the interview.
during or as soon as possible following the interview.
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
2. Observation
2. Observation
It offers the inspector the opportunity to gather ‘live’ data
It offers the inspector the opportunity to gather ‘live’ data
from naturally occurring situations e.g. Checking status of
from naturally occurring situations e.g. Checking status of
systems in the equipment rooms
systems in the equipment rooms
In this way the auditor/inspector can look directly at what is
In this way the auditor/inspector can look directly at what is
taking place rather than relying on second-hand accounts.
taking place rather than relying on second-hand accounts.
What people do sometimes differs from what they say and
What people do sometimes differs from what they say and
observation provides a reality check.
observation provides a reality check.
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
2. Observation
2. Observation
When doing observations consider the following:
When doing observations consider the following:
Do I want to look, or listen, or hear or both?
Do I want to look, or listen, or hear or both?
Do I want to observe in real time, or later, or both?
Do I want to observe in real time, or later, or both?
If in real time, do I record on paper or electronically?
If in real time, do I record on paper or electronically?
If later, do I capture events by voice recorder, or video, or still
If later, do I capture events by voice recorder, or video, or still
camera, or in text
camera, or in text
Do I want to observe continuously, or for fixed periods, or at fixed
Do I want to observe continuously, or for fixed periods, or at fixed
intervals?
intervals?
Do I want to observe the whole arena, or just part of it?
Do I want to observe the whole arena, or just part of it?
Do I want to observe the same events/people all the time, or to
Do I want to observe the same events/people all the time, or to
change?
change?
Do I want to do the observation myself or involve others?
Do I want to do the observation myself or involve others?
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
An effective audit Team shall:
An effective audit Team shall:
Exercise good time management
Exercise good time management
Use checklists to collect evidence
Use checklists to collect evidence
Employ positive questioning techniques and listen to the
Employ positive questioning techniques and listen to the
auditee’s response
auditee’s response
Take notes
Take notes
Sample evidence provided
Sample evidence provided
Link evidence together for trends
Link evidence together for trends
Summarize the findings
Summarize the findings
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Daily briefings
Team briefings
ensure adherence to the Inspection/Audit plan;
validate confirmation requests and Inspection/Audit
findings;
resolve issues or problems arising from the day’s activities;
and
provide the team leader with the information necessary
to update the Authority where applicable
 
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
 
Daily briefings
ANSP Briefings
Scheduled to minimize any disruption
Update the ANSP’s management on Inspection/Audit
progress
D
iscuss any Inspection/Audit findings that have been
identified.
 
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Exit meeting.
Exit meeting.
On completion of an audit, the auditors convene a post-audit
On completion of an audit, the auditors convene a post-audit
meeting with the officer in-charge of the audited party.
meeting with the officer in-charge of the audited party.
The auditors have the discretion to either hold such meeting on the
The auditors have the discretion to either hold such meeting on the
same day of the audit or choose another date for this occasion.
same day of the audit or choose another date for this occasion.
The purpose of this meeting is to provide both the audit team and the
The purpose of this meeting is to provide both the audit team and the
audited party an opportunity to cover the following aspects:
audited party an opportunity to cover the following aspects:
a)
Review of the objective and scope of the audit;
Review of the objective and scope of the audit;
b)
Summary of the audit activities;
Summary of the audit activities;
c)
Presentation of draft findings and recommendations; and
Presentation of draft findings and recommendations; and
d)
Agree on post-audit actions by both the audit team and the
Agree on post-audit actions by both the audit team and the
audited party.
audited party.
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Completing Checklists.
Completing Checklists.
Purpose of Checklists
Purpose of Checklists
;
;
Checklists provide a systematic approach for the
Checklists provide a systematic approach for the
conduct of an Audit/Inspection and are designed
conduct of an Audit/Inspection and are designed
to identify specific items for review and make
to identify specific items for review and make
reference to the applicable regulatory requirement,
reference to the applicable regulatory requirement,
be it to a regulation, standard or control manual
be it to a regulation, standard or control manual
requirement.
requirement.
Checklists should not limit the Auditor/Inspector’s
Checklists should not limit the Auditor/Inspector’s
ability to explore other areas where warranted.
ability to explore other areas where warranted.
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Completing Checklists.
Completing Checklists.
Audit/Inspection checklists will:
Audit/Inspection checklists will:
be used to guide the Audit/Inspection;
be used to guide the Audit/Inspection;
be completed or have areas that were not
be completed or have areas that were not
completed so annotated; and
completed so annotated; and
be signed and dated by the team member using
be signed and dated by the team member using
the checklist.
the checklist.
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Completing Checklists.
Completing Checklists.
Use of the applicable checklist(s) is mandatory
A
ssist Inspectors/Auditors in determining the level of conformance to
regulatory requirements
Based on the results of the completed checklist, a determination of the
strengths and weaknesses of the ANSP’s control system will be possible
Although each item on the checklist need not be checked, this system
will be most effective if all items are evaluated
There may be times where it is not possible to review a particular
specialty area element.
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Completing Checklists.
Completing Checklists.
Checklists assist the inspection/audit team to;
Checklists assist the inspection/audit team to;
(a)
confirm whether the ANSP’s operation conforms to
confirm whether the ANSP’s operation conforms to
regulatory requirements;
regulatory requirements;
(b)
confirm whether controls are operating effectively and as
confirm whether controls are operating effectively and as
intended and specified in the appropriate manual; and
intended and specified in the appropriate manual; and
(c)
gather evidence or supporting documentation and
gather evidence or supporting documentation and
prepare an inspection/audit finding, where non-
prepare an inspection/audit finding, where non-
conformance with a regulatory requirement is identified
conformance with a regulatory requirement is identified
,
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Completing Checklists.
Completing Checklists.
The CNS inspector is expected to make entries/complete
The CNS inspector is expected to make entries/complete
the checklist to indicate the level of compliance by the
the checklist to indicate the level of compliance by the
service provider.
service provider.
The checklist is divided into several areas depending on
The checklist is divided into several areas depending on
State guidelines.
State guidelines.
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Completing Checklists.
Completing Checklists.
 
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
Completing Checklists.
Completing Checklists.
 
A CNS Inspection checklist would typically consist of various elements;
A CNS Inspection checklist would typically consist of various elements;
Organization/Administrative Issues/Documentation
Organization/Administrative Issues/Documentation
Processes/Procedures
Processes/Procedures
Personnel Requirements/Training
Personnel Requirements/Training
Facilities –Communication, Navigation, Surveillance, Auxiliary Systems
Facilities –Communication, Navigation, Surveillance, Auxiliary Systems
 
Note: The above elements are checked against the regulations in force
and approved operating procedures.
          
Handouts H2.3/2.4
 
Physical Audit/Inspection Phase
Physical Audit/Inspection Phase
 
 
 
 
   EX 2.2 – Case Scenario
   EX 2.2 – Case Scenario
Conducting Inspection/Completing CNS Inspection
Conducting Inspection/Completing CNS Inspection
Checklist
Checklist
 
Post-Audit/Inspection & follow up Phase
Post-Audit/Inspection & follow up Phase
 
Post-Audit/Inspection activities include completion of
Post-Audit/Inspection activities include completion of
administrative details and production of the
administrative details and production of the
Audit/Inspection report.
Audit/Inspection report.
Audit/Inspection follow-up includes acceptance of the
Audit/Inspection follow-up includes acceptance of the
organization’s corrective action plan, verification of full
organization’s corrective action plan, verification of full
implementation of that plan and formal closure of the
implementation of that plan and formal closure of the
Audit/Inspection by the team leader.
Audit/Inspection by the team leader.
Note:
Note:
 The CNS inspectorate should establish and implement,
 The CNS inspectorate should establish and implement,
a system for record-keeping and filing.
a system for record-keeping and filing.
Slide Note
Embed
Share

Conducting surveillance of CNS providers involves audits and inspections to ensure compliance with regulatory requirements and maintain safety standards. Various types of audits, such as pre-certification and post-certification audits, are conducted by qualified CNS oversight inspectors to identify non-compliance issues and implement corrective actions.

  • Surveillance
  • CNS Providers
  • Audits
  • Compliance
  • Safety

Uploaded on Sep 24, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. CHAPTER 2 Conducting Surveillance of CNS Providers

  2. Definitions Surveillance: A process through which the State proactively verifies through inspections, audits and other activities that aviation licence, certificate, authorization or approval holders continue to meet the established requirements and function at the level of competency and safety required by the State. Audit: An in-depth review of the activities of an organization to verify conformance to regulatory requirements. Inspection: The basic activity of an audit, involving the systematic assessment of a specific characteristic of an organization to verify conformance to regulatory requirements or company approved procedures.

  3. Overview Audits/inspections of CNS service providers enable the CNS Inspectorate to: verify the CNS Providers' level of compliance with requirements. validate that preventive controls established by the CNS provider to mitigate risks are effective. CAA therefore uses audits to answer the question, Is it working? Audits may be systemic or targeted to specific facilities or stations, and Audits may be conducted on-site or virtually, and include the review of documents, records, and other data provided by the ANSP.

  4. Overview Safety Audits/inspections; Provide the oversight organization and other authorities with evidence of ANSP s compliance with applicable safety processes, regulatory requirements, and implementation arrangements, and attempt to identify the need for improvement or corrective action; Are independent of CNS provider s internal safety audits undertaken as part of its safety, quality control, quality assurance, and/or quality management systems; Are conducted by qualified CNS oversight inspectors; and Lead to the correction of any identified non-compliance issues.

  5. TYPES OF AUDIT Pre-certification Audit/inspection (phase 4); Post certification/routine conformance Audit/inspection; Additional authority Audit/inspection; Special-purpose Audit/inspection.

  6. TYPES OF AUDIT Pre-certification Audit/Inspection Pre-certification Audit/inspection will be conducted prior to the issuance of the certificate. This occurs at the fourth stage- Demonstration and inspection phases of the ANSP certification process.

  7. TYPES OF AUDIT Post certification/routine conformance Inspection/Audit; A Post certification/ routine conformance Audit/inspection will be conducted for the purpose of determining an organization s overall level of conformance to regulatory requirements after the ANSP have been certificated. All applicable characteristics of the organization will be subject to review.

  8. TYPES OF AUDIT Additional authority Audit/Inspection An additional authority Audit/inspection may be conducted prior to the granting of an additional authority. Prior notification to the organization is not required.

  9. TYPES OF AUDIT Special-purpose Audit/Inspection A special-purpose Audit/Inspection is one conducted to respond to safety concerns or circumstances other than those requiring a pre-certification Audit/inspection, an additional authority Inspection/Audit or a routine conformance Audit/inspection. A special purpose audit may be convened with little or no notice arising from safety concern (e.g. Adhoc surveillance). A special purpose Audit/inspection may preclude certain team-member activities and responsibilities that would be normally associated with other types of Audit/inspections.

  10. Audit/Inspection Phases THREE distinct phases: Pre-Audit/Inspection; Physical Audit/Inspection; Post- Audit/Inspection; and Audit/Inspection follow-up

  11. Audit/Inspection Phases Pre-Audit/Inspection Phase Pre-Audit/Inspection activities involve notifying the organization, selecting an Audit/Inspection team, developing an Audit/Inspection plan, reviewing files and related documentation and convening the pre- Audit/Inspection team meeting.

  12. Audit/Inspection Phases Physical Audit/Inspection Phase The physical Audit/Inspection phase will be implemented in accordance with the Audit/Inspection plan. The purpose of this phase is to verify conformance with regulatory requirements and to assign findings where non-conformance has been confirmed. Audit/Inspection results will be communicated to the organization under review at daily meetings and/or the exit meeting. Once an Audit/Inspection has commenced, it should not be terminated without approval of the Head of the inspectorate

  13. Audit/Inspection Phases Post- Audit/Inspection & follow up Phase Post-Audit/Inspection activities include completion of administrative details and production of the Audit/Inspection report

  14. Planning a CNS Inspection Audit/Inspection Programme The ANS- CNS oversight department is required to establish and implement, a surveillance programme which should include, at a minimum, the elements below; the types of surveillance activities (e.g. audits, inspections, tests, safety events analyses); the timeframe or frequency of the activities; items to be covered or scope of the activities; and related methodology/procedures, job aids and guidance on how the activity should be conducted, starting from the notification of the service provider, if applicable, to the closure of the deficiencies identified during the activities

  15. Planning a CNS Inspection Audit/Inspection programme The oversight audit program should; Enable audit/inspection of areas of potential safety concern with the highest identified risk, or with a focus on those areas where problems have been identified; Cover all the functions/stations within CNS provider organizations, services and network; Ensure that audits are conducted in a manner commensurate to the level of risk posed by the CNS provider s activities;

  16. Planning a CNS Inspection Audit/Inspection programme The oversight audit program should; Ensure that sufficient audits are conducted on a consistent basis, to check the compliance of all CNS organizations, as pertains to applicable safety processes, and regulatory requirements in all the relevant areas of the functional system; and, Enable conduct of follow-up activities to ensure effective implementation of corrective actions.

  17. Planning a CNS Inspection Elements of surveillance Program a) The CNS provider/Station to be audited b) Audit/inspection type c) The team composition d) Budget e) Audit/inspection dates

  18. Planning a CNS Inspection Factors that influence the scope and depth of a surveillance programme type of the Inspection/Audit; the period back in time that systems are subject to review (typically from the last Inspection/Audit to the present); the enforcement record of the organization; the confidence in corrective actions taken by the organization as a result of a previous Inspection/Audit; and Inspector/Auditor and financial resources available. Handout H2.1

  19. Planning a CNS Inspection Audit Plans For each individual CNS inspector approved to carry out aviation activities, a periodic surveillance plan should be developed based on the applicable surveillance programme. For each surveillance activity on the service provider, a surveillance plan should be developed The surveillance plan should detail the type of activities to be performed and the specified timeframe as well as the scope of each activity, as applicable. The purpose of the safety oversight audit plan is to outline the sequential process of the on-site audit and provide the CNS provider with valuable information on the conduct of the audit

  20. Planning a CNS Inspection Contents of an audit plan dates of the on-site/virtual safety oversight audit; objectives of the safety oversight audit; scheduled dates for opening and closing meetings; identification of the CNS provider s key personnel (safety oversight coordinator would have been already appointed and acknowledged to Authority); scope of the on-site safety oversight audit to be conducted and audit areas to be covered; identification of documents necessary to conduct the safety oversight audit;

  21. Planning a CNS Inspection Contents of an audit plan travel and administration plans (security clearances, health requirements, etc.) as well as the travel schedules of audit members; the language to be used for the safety oversight audit, the audit preparation briefing material for team members and the on-site audit mission package; identification of priority issues brought to the attention of ATS/SAR provider from reviewing material provided and other documents available to CAA or information received on the status of Services Team members assignments and responsibilities. Handout H 2.2

  22. Pre-Audit Activities

  23. Pre-Audit Activities Audit Research Throughout the audit preparation process, audit team members should thoroughly research the upcoming audit to ensure that all applicable requirements have been captured, and that they have not changed. Reference may be made to the ICAO standards as well as respective State regulations and directives, standard operating procedures, corrective action plans, etc. Important research questions include: What initiated the audit? What are the results of any past audits on the audit area? Are there enforcement actions against the service provider related to this audit area? Which requirements are applicable to this audit area?

  24. Audit/Inspection Plan Case Scenario Practical Exercise- Preparation of an Inspection Plan

  25. Physical Audit/Inspection Phase The physical Audit/Inspection phase will be implemented in accordance with the Audit/Inspection plan. The purpose of this phase is to verify conformance with regulatory requirements and to assign findings where non-conformance has been confirmed. Audit/Inspection results will be communicated to the organization under review at daily meetings and/or the exit meeting. Once an Audit/Inspection has commenced, it should not be terminated without approval of the Head of the inspectorate

  26. Physical Audit/Inspection Phase Audit teams are expected to: conduct an opening meeting with facility management; collect data, analyze collected data and verify that analysis; record findings; conduct team briefings; and conduct a closing meeting with facility management while conducting on-site audits.

  27. Physical Audit/Inspection Phase Opening Meeting The Audit/Inspection team should conduct an entry meeting on the first day of the audit/inspection with the audited party so that both parties have an opportunity to meet and discuss on relevant details and arrangements. The entry meeting should set the tone for the physical Audit/Inspection and should be attended by the CNS provider s senior management and all members of the Audit/Inspection team.

  28. Physical Audit/Inspection Phase Opening Meeting Elements Thank the Auditee for cooperation Introduce yourself and team member(s) present Explain the purpose of the Audit Review objectives and principles of the audit (responsibilities and duties of the Auditee and the audit team). Describe the audit process and methods of gathering information (e.g. observation, discussion, review of documents) during the audit. Briefly present the audit plan with the objective to adjust and confirm it.

  29. Physical Audit/Inspection Phase Opening Meeting Elements Confirm the arrangements for the post-audit debriefing (location, participants) Explain the reporting system, including the audit report and the action plan Identify who is designated by the Auditee to receive the audit report Be receptive to any special concerns from the Auditee with regard to the performance of the auditor areas to be observed Confirm the location of the audit team facilities and any other administrative details

  30. Physical Audit/Inspection Phase Evidence Gathering Techniques The following methods will be used to gather information Detailed documentation review Random sampling of documentation Interviews Observations Checklists

  31. Physical Audit/Inspection Phase Evidence Gathering Techniques 1. Interviewing CNS Provider s Personnel Figure 2: Types of interviews

  32. Physical Audit/Inspection Phase Figure 3: In-depth Interviews

  33. Physical Audit/Inspection Phase Interviewing The CNS Provider s Personnel In-depth interviews are the best since they involve a one on one interaction and designed to provide a detailed picture of the audited/inspected entity. Responses given by the auditee should be documented during or as soon as possible following the interview.

  34. Physical Audit/Inspection Phase 2. Observation It offers the inspector the opportunity to gather live data from naturally occurring situations e.g. Checking status of systems in the equipment rooms In this way the auditor/inspector can look directly at what is taking place rather than relying on second-hand accounts. What people do sometimes differs from what they say and observation provides a reality check.

  35. Physical Audit/Inspection Phase 2. Observation When doing observations consider the following: Do I want to look, or listen, or hear or both? Do I want to observe in real time, or later, or both? If in real time, do I record on paper or electronically? If later, do I capture events by voice recorder, or video, or still camera, or in text Do I want to observe continuously, or for fixed periods, or at fixed intervals? Do I want to observe the whole arena, or just part of it? Do I want to observe the same events/people all the time, or to change? Do I want to do the observation myself or involve others?

  36. Physical Audit/Inspection Phase An effective audit Team shall: Exercise good time management Use checklists to collect evidence Employ positive questioning techniques and listen to the auditee s response Take notes Sample evidence provided Link evidence together for trends Summarize the findings

  37. Physical Audit/Inspection Phase Daily briefings Team briefings ensure adherence to the Inspection/Audit plan; validate confirmation requests and Inspection/Audit findings; resolve issues or problems arising from the day s activities; and provide the team leader with the information necessary to update the Authority where applicable

  38. Physical Audit/Inspection Phase Daily briefings ANSP Briefings Scheduled to minimize any disruption Update the ANSP s management on Inspection/Audit progress Discuss any Inspection/Audit findings that have been identified.

  39. Physical Audit/Inspection Phase Exit meeting. On completion of an audit, the auditors convene a post-audit meeting with the officer in-charge of the audited party. The auditors have the discretion to either hold such meeting on the same day of the audit or choose another date for this occasion. The purpose of this meeting is to provide both the audit team and the audited party an opportunity to cover the following aspects: a) Review of the objective and scope of the audit; b) Summary of the audit activities; c) Presentation of draft findings and recommendations; and d) Agree on post-audit actions by both the audit team and the audited party.

  40. Physical Audit/Inspection Phase Completing Checklists. Purpose of Checklists; Checklists provide a systematic approach for the conduct of an Audit/Inspection and are designed to identify specific items for review and make reference to the applicable regulatory requirement, be it to a regulation, standard or control manual requirement. Checklists should not limit the Auditor/Inspector s ability to explore other areas where warranted.

  41. Physical Audit/Inspection Phase Completing Checklists. Audit/Inspection checklists will: be used to guide the Audit/Inspection; be completed or have areas that were not completed so annotated; and be signed and dated by the team member using the checklist.

  42. Physical Audit/Inspection Phase Completing Checklists. Use of the applicable checklist(s) is mandatory Assist Inspectors/Auditors in determining the level of conformance to regulatory requirements Based on the results of the completed checklist, a determination of the strengths and weaknesses of the ANSP s control system will be possible Although each item on the checklist need not be checked, this system will be most effective if all items are evaluated There may be times where it is not possible to review a particular specialty area element.

  43. Physical Audit/Inspection Phase Completing Checklists. Checklists assist the inspection/audit team to; (a)confirm whether the ANSP s operation conforms to regulatory requirements; (b)confirm whether controls are operating effectively and as intended and specified in the appropriate manual; and (c)gather evidence or supporting documentation and prepare an inspection/audit finding, where non- conformance with a regulatory requirement is identified,

  44. Physical Audit/Inspection Phase Completing Checklists. The CNS inspector is expected to make entries/complete the checklist to indicate the level of compliance by the service provider. The checklist is divided into several areas depending on State guidelines.

  45. Physical Audit/Inspection Phase Completing Checklists. Checklist Item Inspection Item Comments Provides details of areas to be verified/checked for 1 compliance by the service provider Indication of the outcome on the level of assessment 2 Assessment Outcome Yes (Y) No (N) Not Checked (NC) Not Applicable (NA) Remarks Regulatory by the inspector Denotes full compliance that is acceptable Denotes non-compliance that is not acceptable Inspection item not checked Inspection item not applicable Remarks from the inspector Indication of the applicable regulations that need to 3 4 Requirements be complied with.

  46. Physical Audit/Inspection Phase Completing Checklists. A CNS Inspection checklist would typically consist of various elements; Organization/Administrative Issues/Documentation Processes/Procedures Personnel Requirements/Training Facilities Communication, Navigation, Surveillance, Auxiliary Systems Note: The above elements are checked against the regulations in force and approved operating procedures. Handouts H2.3/2.4

  47. Physical Audit/Inspection Phase EX 2.2 Case Scenario Conducting Inspection/Completing CNS Inspection Checklist

  48. Post-Audit/Inspection & follow up Phase Post-Audit/Inspection activities include completion of administrative details and production of the Audit/Inspection report. Audit/Inspection follow-up includes acceptance of the organization s corrective action plan, verification of full implementation of that plan and formal closure of the Audit/Inspection by the team leader. Note: The CNS inspectorate should establish and implement, a system for record-keeping and filing.

Related


More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#