Conducting Surveillance of CNS Providers
Conducting surveillance of CNS providers involves audits and inspections to ensure compliance with regulatory requirements and maintain safety standards. Various types of audits, such as pre-certification and post-certification audits, are conducted by qualified CNS oversight inspectors to identify non-compliance issues and implement corrective actions.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
CHAPTER 2 Conducting Surveillance of CNS Providers
Definitions Surveillance: A process through which the State proactively verifies through inspections, audits and other activities that aviation licence, certificate, authorization or approval holders continue to meet the established requirements and function at the level of competency and safety required by the State. Audit: An in-depth review of the activities of an organization to verify conformance to regulatory requirements. Inspection: The basic activity of an audit, involving the systematic assessment of a specific characteristic of an organization to verify conformance to regulatory requirements or company approved procedures.
Overview Audits/inspections of CNS service providers enable the CNS Inspectorate to: verify the CNS Providers' level of compliance with requirements. validate that preventive controls established by the CNS provider to mitigate risks are effective. CAA therefore uses audits to answer the question, Is it working? Audits may be systemic or targeted to specific facilities or stations, and Audits may be conducted on-site or virtually, and include the review of documents, records, and other data provided by the ANSP.
Overview Safety Audits/inspections; Provide the oversight organization and other authorities with evidence of ANSP s compliance with applicable safety processes, regulatory requirements, and implementation arrangements, and attempt to identify the need for improvement or corrective action; Are independent of CNS provider s internal safety audits undertaken as part of its safety, quality control, quality assurance, and/or quality management systems; Are conducted by qualified CNS oversight inspectors; and Lead to the correction of any identified non-compliance issues.
TYPES OF AUDIT Pre-certification Audit/inspection (phase 4); Post certification/routine conformance Audit/inspection; Additional authority Audit/inspection; Special-purpose Audit/inspection.
TYPES OF AUDIT Pre-certification Audit/Inspection Pre-certification Audit/inspection will be conducted prior to the issuance of the certificate. This occurs at the fourth stage- Demonstration and inspection phases of the ANSP certification process.
TYPES OF AUDIT Post certification/routine conformance Inspection/Audit; A Post certification/ routine conformance Audit/inspection will be conducted for the purpose of determining an organization s overall level of conformance to regulatory requirements after the ANSP have been certificated. All applicable characteristics of the organization will be subject to review.
TYPES OF AUDIT Additional authority Audit/Inspection An additional authority Audit/inspection may be conducted prior to the granting of an additional authority. Prior notification to the organization is not required.
TYPES OF AUDIT Special-purpose Audit/Inspection A special-purpose Audit/Inspection is one conducted to respond to safety concerns or circumstances other than those requiring a pre-certification Audit/inspection, an additional authority Inspection/Audit or a routine conformance Audit/inspection. A special purpose audit may be convened with little or no notice arising from safety concern (e.g. Adhoc surveillance). A special purpose Audit/inspection may preclude certain team-member activities and responsibilities that would be normally associated with other types of Audit/inspections.
Audit/Inspection Phases THREE distinct phases: Pre-Audit/Inspection; Physical Audit/Inspection; Post- Audit/Inspection; and Audit/Inspection follow-up
Audit/Inspection Phases Pre-Audit/Inspection Phase Pre-Audit/Inspection activities involve notifying the organization, selecting an Audit/Inspection team, developing an Audit/Inspection plan, reviewing files and related documentation and convening the pre- Audit/Inspection team meeting.
Audit/Inspection Phases Physical Audit/Inspection Phase The physical Audit/Inspection phase will be implemented in accordance with the Audit/Inspection plan. The purpose of this phase is to verify conformance with regulatory requirements and to assign findings where non-conformance has been confirmed. Audit/Inspection results will be communicated to the organization under review at daily meetings and/or the exit meeting. Once an Audit/Inspection has commenced, it should not be terminated without approval of the Head of the inspectorate
Audit/Inspection Phases Post- Audit/Inspection & follow up Phase Post-Audit/Inspection activities include completion of administrative details and production of the Audit/Inspection report
Planning a CNS Inspection Audit/Inspection Programme The ANS- CNS oversight department is required to establish and implement, a surveillance programme which should include, at a minimum, the elements below; the types of surveillance activities (e.g. audits, inspections, tests, safety events analyses); the timeframe or frequency of the activities; items to be covered or scope of the activities; and related methodology/procedures, job aids and guidance on how the activity should be conducted, starting from the notification of the service provider, if applicable, to the closure of the deficiencies identified during the activities
Planning a CNS Inspection Audit/Inspection programme The oversight audit program should; Enable audit/inspection of areas of potential safety concern with the highest identified risk, or with a focus on those areas where problems have been identified; Cover all the functions/stations within CNS provider organizations, services and network; Ensure that audits are conducted in a manner commensurate to the level of risk posed by the CNS provider s activities;
Planning a CNS Inspection Audit/Inspection programme The oversight audit program should; Ensure that sufficient audits are conducted on a consistent basis, to check the compliance of all CNS organizations, as pertains to applicable safety processes, and regulatory requirements in all the relevant areas of the functional system; and, Enable conduct of follow-up activities to ensure effective implementation of corrective actions.
Planning a CNS Inspection Elements of surveillance Program a) The CNS provider/Station to be audited b) Audit/inspection type c) The team composition d) Budget e) Audit/inspection dates
Planning a CNS Inspection Factors that influence the scope and depth of a surveillance programme type of the Inspection/Audit; the period back in time that systems are subject to review (typically from the last Inspection/Audit to the present); the enforcement record of the organization; the confidence in corrective actions taken by the organization as a result of a previous Inspection/Audit; and Inspector/Auditor and financial resources available. Handout H2.1
Planning a CNS Inspection Audit Plans For each individual CNS inspector approved to carry out aviation activities, a periodic surveillance plan should be developed based on the applicable surveillance programme. For each surveillance activity on the service provider, a surveillance plan should be developed The surveillance plan should detail the type of activities to be performed and the specified timeframe as well as the scope of each activity, as applicable. The purpose of the safety oversight audit plan is to outline the sequential process of the on-site audit and provide the CNS provider with valuable information on the conduct of the audit
Planning a CNS Inspection Contents of an audit plan dates of the on-site/virtual safety oversight audit; objectives of the safety oversight audit; scheduled dates for opening and closing meetings; identification of the CNS provider s key personnel (safety oversight coordinator would have been already appointed and acknowledged to Authority); scope of the on-site safety oversight audit to be conducted and audit areas to be covered; identification of documents necessary to conduct the safety oversight audit;
Planning a CNS Inspection Contents of an audit plan travel and administration plans (security clearances, health requirements, etc.) as well as the travel schedules of audit members; the language to be used for the safety oversight audit, the audit preparation briefing material for team members and the on-site audit mission package; identification of priority issues brought to the attention of ATS/SAR provider from reviewing material provided and other documents available to CAA or information received on the status of Services Team members assignments and responsibilities. Handout H 2.2
Pre-Audit Activities Audit Research Throughout the audit preparation process, audit team members should thoroughly research the upcoming audit to ensure that all applicable requirements have been captured, and that they have not changed. Reference may be made to the ICAO standards as well as respective State regulations and directives, standard operating procedures, corrective action plans, etc. Important research questions include: What initiated the audit? What are the results of any past audits on the audit area? Are there enforcement actions against the service provider related to this audit area? Which requirements are applicable to this audit area?
Audit/Inspection Plan Case Scenario Practical Exercise- Preparation of an Inspection Plan
Physical Audit/Inspection Phase The physical Audit/Inspection phase will be implemented in accordance with the Audit/Inspection plan. The purpose of this phase is to verify conformance with regulatory requirements and to assign findings where non-conformance has been confirmed. Audit/Inspection results will be communicated to the organization under review at daily meetings and/or the exit meeting. Once an Audit/Inspection has commenced, it should not be terminated without approval of the Head of the inspectorate
Physical Audit/Inspection Phase Audit teams are expected to: conduct an opening meeting with facility management; collect data, analyze collected data and verify that analysis; record findings; conduct team briefings; and conduct a closing meeting with facility management while conducting on-site audits.
Physical Audit/Inspection Phase Opening Meeting The Audit/Inspection team should conduct an entry meeting on the first day of the audit/inspection with the audited party so that both parties have an opportunity to meet and discuss on relevant details and arrangements. The entry meeting should set the tone for the physical Audit/Inspection and should be attended by the CNS provider s senior management and all members of the Audit/Inspection team.
Physical Audit/Inspection Phase Opening Meeting Elements Thank the Auditee for cooperation Introduce yourself and team member(s) present Explain the purpose of the Audit Review objectives and principles of the audit (responsibilities and duties of the Auditee and the audit team). Describe the audit process and methods of gathering information (e.g. observation, discussion, review of documents) during the audit. Briefly present the audit plan with the objective to adjust and confirm it.
Physical Audit/Inspection Phase Opening Meeting Elements Confirm the arrangements for the post-audit debriefing (location, participants) Explain the reporting system, including the audit report and the action plan Identify who is designated by the Auditee to receive the audit report Be receptive to any special concerns from the Auditee with regard to the performance of the auditor areas to be observed Confirm the location of the audit team facilities and any other administrative details
Physical Audit/Inspection Phase Evidence Gathering Techniques The following methods will be used to gather information Detailed documentation review Random sampling of documentation Interviews Observations Checklists
Physical Audit/Inspection Phase Evidence Gathering Techniques 1. Interviewing CNS Provider s Personnel Figure 2: Types of interviews
Physical Audit/Inspection Phase Figure 3: In-depth Interviews
Physical Audit/Inspection Phase Interviewing The CNS Provider s Personnel In-depth interviews are the best since they involve a one on one interaction and designed to provide a detailed picture of the audited/inspected entity. Responses given by the auditee should be documented during or as soon as possible following the interview.
Physical Audit/Inspection Phase 2. Observation It offers the inspector the opportunity to gather live data from naturally occurring situations e.g. Checking status of systems in the equipment rooms In this way the auditor/inspector can look directly at what is taking place rather than relying on second-hand accounts. What people do sometimes differs from what they say and observation provides a reality check.
Physical Audit/Inspection Phase 2. Observation When doing observations consider the following: Do I want to look, or listen, or hear or both? Do I want to observe in real time, or later, or both? If in real time, do I record on paper or electronically? If later, do I capture events by voice recorder, or video, or still camera, or in text Do I want to observe continuously, or for fixed periods, or at fixed intervals? Do I want to observe the whole arena, or just part of it? Do I want to observe the same events/people all the time, or to change? Do I want to do the observation myself or involve others?
Physical Audit/Inspection Phase An effective audit Team shall: Exercise good time management Use checklists to collect evidence Employ positive questioning techniques and listen to the auditee s response Take notes Sample evidence provided Link evidence together for trends Summarize the findings
Physical Audit/Inspection Phase Daily briefings Team briefings ensure adherence to the Inspection/Audit plan; validate confirmation requests and Inspection/Audit findings; resolve issues or problems arising from the day s activities; and provide the team leader with the information necessary to update the Authority where applicable
Physical Audit/Inspection Phase Daily briefings ANSP Briefings Scheduled to minimize any disruption Update the ANSP s management on Inspection/Audit progress Discuss any Inspection/Audit findings that have been identified.
Physical Audit/Inspection Phase Exit meeting. On completion of an audit, the auditors convene a post-audit meeting with the officer in-charge of the audited party. The auditors have the discretion to either hold such meeting on the same day of the audit or choose another date for this occasion. The purpose of this meeting is to provide both the audit team and the audited party an opportunity to cover the following aspects: a) Review of the objective and scope of the audit; b) Summary of the audit activities; c) Presentation of draft findings and recommendations; and d) Agree on post-audit actions by both the audit team and the audited party.
Physical Audit/Inspection Phase Completing Checklists. Purpose of Checklists; Checklists provide a systematic approach for the conduct of an Audit/Inspection and are designed to identify specific items for review and make reference to the applicable regulatory requirement, be it to a regulation, standard or control manual requirement. Checklists should not limit the Auditor/Inspector s ability to explore other areas where warranted.
Physical Audit/Inspection Phase Completing Checklists. Audit/Inspection checklists will: be used to guide the Audit/Inspection; be completed or have areas that were not completed so annotated; and be signed and dated by the team member using the checklist.
Physical Audit/Inspection Phase Completing Checklists. Use of the applicable checklist(s) is mandatory Assist Inspectors/Auditors in determining the level of conformance to regulatory requirements Based on the results of the completed checklist, a determination of the strengths and weaknesses of the ANSP s control system will be possible Although each item on the checklist need not be checked, this system will be most effective if all items are evaluated There may be times where it is not possible to review a particular specialty area element.
Physical Audit/Inspection Phase Completing Checklists. Checklists assist the inspection/audit team to; (a)confirm whether the ANSP s operation conforms to regulatory requirements; (b)confirm whether controls are operating effectively and as intended and specified in the appropriate manual; and (c)gather evidence or supporting documentation and prepare an inspection/audit finding, where non- conformance with a regulatory requirement is identified,
Physical Audit/Inspection Phase Completing Checklists. The CNS inspector is expected to make entries/complete the checklist to indicate the level of compliance by the service provider. The checklist is divided into several areas depending on State guidelines.
Physical Audit/Inspection Phase Completing Checklists. Checklist Item Inspection Item Comments Provides details of areas to be verified/checked for 1 compliance by the service provider Indication of the outcome on the level of assessment 2 Assessment Outcome Yes (Y) No (N) Not Checked (NC) Not Applicable (NA) Remarks Regulatory by the inspector Denotes full compliance that is acceptable Denotes non-compliance that is not acceptable Inspection item not checked Inspection item not applicable Remarks from the inspector Indication of the applicable regulations that need to 3 4 Requirements be complied with.
Physical Audit/Inspection Phase Completing Checklists. A CNS Inspection checklist would typically consist of various elements; Organization/Administrative Issues/Documentation Processes/Procedures Personnel Requirements/Training Facilities Communication, Navigation, Surveillance, Auxiliary Systems Note: The above elements are checked against the regulations in force and approved operating procedures. Handouts H2.3/2.4
Physical Audit/Inspection Phase EX 2.2 Case Scenario Conducting Inspection/Completing CNS Inspection Checklist
Post-Audit/Inspection & follow up Phase Post-Audit/Inspection activities include completion of administrative details and production of the Audit/Inspection report. Audit/Inspection follow-up includes acceptance of the organization s corrective action plan, verification of full implementation of that plan and formal closure of the Audit/Inspection by the team leader. Note: The CNS inspectorate should establish and implement, a system for record-keeping and filing.
