Internal Auditor Training: ISO 9001 & 19011 Overview

Internal Auditor Training 01/08/2017

Learning objectives Describe the purpose of internal auditing. Describe the responsibilities of an internal auditor. Plan, conduct and report an internal audit in accordance with ISO 19011. Describe purpose and structure of ISO 9001. Reference activities to the standard management system model. Reference activities to the audit criteria. Appreciate the model for continual improvement (plan-do-check-act cycle). 2

Time plan 09:30 09:45 10:00 10:45 11:00 11:15 11:45 12:15 12:30 13:00 13:30 14:00 14:45 15:00 15:30 16:15 16:30 Introduction What is auditing Auditing process overview, checklist and principles Break Approach to managing objectives ISO 9001 Processes and process-approach Improvement, risk and opportunities Lunch Methods for gathering evidence Preparing and using audit criteria checklist Checklist exercise, classifying audit findings Break Audit report and follow-up Test Summing up, Q&A session Finish 3

ISO 9001:2015 Requirements for a Quality Management System, where an organisation: needs to demonstrate its ability to meet customer and regulatory requirements, and aims to enhance customer satisfaction through the effective application of the systems, processes, continual improvement and the assurance of conformity to customer and regulatory requirements. Promotes a process approach and defines product as result of a process . Universally applicable to any type of organisation, regardless of size and product. By nature of its product, an organisation can exclude itself from certain requirements although this must not affect ability or responsibility towards the customer, statutory and regulatory requirements. It is a requirement that organisation periodically audits the effectiveness and conformity of its system. Sector specific variants e.g: ISO 13485:2009 on medical devices quality management system . Sector specific guidance standards, do not add, change or modify e.g: ISO 18091:2014 on Guidelines for the application of ISO 9001 in local government . Source: International Standards Organisation 4

5 ISO 9001 system model a system to establish policy and objectives and to achieve those objectives

ISO 19011:2011 Guidance standard on the principles and process for auditing management systems. Guidance standard on auditor competencies. Applicable to all organizations with a need to establish a programme for conducting internal or external audits. Applicable to other types of audits is possible, provided that consideration is given to the auditor s understanding of the type of system being audited e.g. quality, health and safety, stock, finance or environmental auditing. Source: International Standards Organisation 6

Audit objectives Effectiveness Assess degree to which the management system meets specified objectives. Identify risks that may need control or elimination. Conformance Verify that management system (or parts of it) conforms with audit criteria. Verify that activities conform with the intent of the management system. Assess capability of the management system conforming with legal and contractual requirements, to which the organisation is committed/obliged. Improvement Identify areas for potential improvement of the management system. Source: ISO19011:2011 7

Definitions Audit systematic, independent and documented process for obtaining audit evidence and evaluating it objectively to determine the extent to which the audit criteria are fulfilled Audit scope extent and boundaries of an audit Audit plan description of the activities and arrangements for an audit Audit evidence records, statements or other factual information which are relevant to the audit criteria and verifiable. Audit evidence can be qualitative or quantitative Audit criteria set of policies, procedures or requirements used as a reference against which audit evidence is compared Audit findings results of the evaluation of the collected audit evidence against audit criteria Audit conclusion the outcome of an audit, after consideration of all audit findings 8

Definitions (continued) Audit programme arrangements for a set of one or more audits planned for a specific time frame and directed towards a specific purpose Competence demonstrated ability to apply knowledge and skills to achieve intended results Conformity fulfilment of a requirement Non-conformity non-fulfilment of a requirement (minor, major) Requirement need or expectation that is stated, generally implied or obligatory Management system system to establish policy and objectives and to achieve those objectives Continual improvement recurring activity to increase the ability to fulfil requirements 9

Types of audit First-party performed by trained employee, for internal evaluation or verification purpose. Forms basis for self-declaration of conformity. Second-party performed by customer on its supplier. Third-party performed by external independent body, usually accredited to provide certification of conformity. Source: ISO9000:2005 10

Internal audit process Input: Resources, risks from non-conformance, previous audits outcomes Output: Schedule e.g. 6 to 8 audits per year, of 4 to 8 hours each Checklist Input: Regulation, standards, adopted code, binding agreements, Process definitions, documentation. Form: Checklist Checklist of Audit Criteria Regulation: - e.g. must CE-mark System evidence (defined) Conformity assessment process is not defined in system Process manual defines monthly meetings Process for assessing qualification Activity evidence (implemented) - - -? No purpose checking for evidence of this Records show 2 months between meetings last year Records for 10 staff all show assessments done Standard: - e.g. regular review Output: Records and observed activities Code of Practice: - e.g. qualified staff Output: Corrective actions plan Audit evidence leading to audit findings is recorded. Untraceable samples may be copied or photographed, and attached to report, for evidential purpose. If the auditor becomes aware of any new or changed assumptions or risks, then he/she should iterate back to re-determine the method and sampling. Output: Audit report, including open action for any non-conformance. Output: Closed report. 11

Checklist tool is key Preparation for ensuring coverage of subject Aids memory Aids time management Keeps you on track through the examination process Discovers evidence Collects evidence 12

Exercise 1 basic audit walk through ISO9001:2015, clause 8.2.3.1 Define audit criteria Describe system evidence Describe implementation evidence Describe audit finding Recommend actions 13

Exercise 2 complete by yourself ISO9001:2015, clause 8.4.3 Define audit criteria Describe system evidence Describe implementation evidence Describe audit finding Recommend actions 14

Principles of auditing Integrity Fair presentation foundation of professionalism obligation to report truthfully and accurately application of diligence and sound judgement security of information basis for the impartiality of the audit and objectivity of the audit conclusions Due professional care Confidentiality Independence Evidence-based approach rational method for reaching reliable and reproducible audit conclusions in a systematic audit process Source: ISO19011:2011 15

Auditor qualities Knowledgeable Risk aware Ethical Open-minded Observant Perceptive Versatile Tenacious Decisive Self-reliant Sensitive Collaborative sufficient to understand type of practices and terminology understand basic risk management principles fair, truthful, sincere, honest and discreet willing to consider alternative ideas or points of view actively observing physical surroundings and activities aware of and able to understand situations able to readily adapt to different situations persistent and focused on achieving objectives able to reach timely conclusions on logical reasoning and analysis able to act without being influenced by need for popularity observant and respectful to the culture of the auditee effectively interacting with auditee s personnel Source: ISO19011:2011 16

Congratulations You now know auditing Rest of today will focus on: Sources of audit criteria for the relevant management systems More on auditing methods and techniques 17

Time plan 09:30 09:45 10:00 10:45 11:00 11:15 11:45 12:15 12:30 13:00 13:30 14:00 14:45 15:00 15:30 16:15 16:30 Introduction What is auditing Auditing process overview, checklist and principles Break Approach to managing objectives ISO 9001 Processes and process-approach Improvement, risk and opportunities Lunch Methods for gathering evidence Preparing and using audit criteria checklist Checklist exercise, classifying audit findings Break Audit report and follow-up Test Summing up, Q&A session Finish 18

Policies and objectives Discuss: How to apply to managing personal finances? How to apply to an organisation managing quality? How to apply to an organisation managing OH&S? What else can policy and objectives be applied to? Significance of policies, objectives and defined processes? 19

PDCA management cycle PDCA cycle provides the underlying model for management. PDCA cycle is clear and easy enough, but how do we build a process-based management system around it? 20

ISO 9001 system model Everything that it takes to transform all of the input requirements into an output. The support element develops and maintains the appropriate competencies, capability and capacity in people, equipment, infrastructure and work environment. Suppliers are not part of the organisation, but they can influence outcomes and should therefore be engaged and managed similarly to the organisation s own resources. Measures, investigates and analyses the processes, product and outcomes, including customer satisfaction, for purpose of verifying that planned results are met and for identifying new risks and opportunities. Drives the PDCA cycle. Sets a unified direction and promotes coherence to planned objectives. Unblocks any obstacles and maintains conditions for achieving the objectives. Reactive and proactive activity for assuring the ability to meet requirements and for enhancing the satisfaction of customers and other interested parties. Determines the customer input, mandatory requirements and the organisational context, for translation into objectives. ISO universal model now in ISO 9001, 14001, 16001, 45001 etc 21

Functional vs process approach Optimisation result within a functionally divided organisation Optimisation result within a process-approach We all belong to both a functional structure and to a process structure. Everyone must integrate their efforts, with focus on the customer at the start and end of the core process. 22

Process approach Single process model (adapted from ISO 9001:2015) Process approach integrating multiple processes Process is an activity or set of activities using resources, and are managed in order to enable the transformation of inputs into outputs . Generally, the output from one process forms the input to the next, in a interlinking value-chain starting and ending with the customer. Process approach is the systematic definition and management of processes and their interactions . Definition says: This is how we want to perform the activity and this is what we want the output to look like . In this way everyone can be clear about the tasks and how they link to Company objectives. Definitions should balance risks and opportunities within the system overall. For example, defining a check point control may help prevent a deviance from the original intent. However, if this control is over-rigidly defined then it may prevent an opportunity for improving the process, by de-motivating or disallowing a potentially useful deviation of a new valuable idea. 23

Defining processes Showing the overall core process(es) in a single representation provides people with the understanding of the wider interrelationships, to enable them appreciate the contributions and impacts from their own localised decision making. Define and document any process that can affect the effective planning, operation and control of any Company objective or adopted standard. Do not define processes for trivial activities or those that professional people are reliably trained to know how to perform. 24

Process manual Home Page Links to Menu Page Links to Process Definition Links to Resources and Forms 25

Process definition Planning and deployment Template form Process model Process definition Process definition, supporting description Sub-process, shown in browser view 26

View Process Manual 27

What can make processes ineffective Lack of commitment from top management Not aligned to real needs/requirements not improving Lack of clear objectives and targets Lack of knowledge of and information about the system and documentation Lacking control over risks Inability to take opportunities Poorly defined interfaces (e.g. handover of items and information) Inadequate document control Poor record keeping Complicated documentation - bureaucracy Process or work instructions not followed Poor communication Poor training 28

Improvement Aims to increase the probability of the system enhancing satisfaction of customers and other interested parties. Actions include: Analysing and evaluating existing situation, to identify opportunities. Establishing objectives for improvement. Search for solutions. Evaluate optional solutions and make selection. Implement selected solution. Measure, verify, analyse and re-evaluate the result. Formalise the changes. Start over. Source: ISO9000:2005 29

Risks Severity Severe Slightly Extremely Highly unlikely Trivial risk Tolerable risk Moderate risk Likelihood Unlikely Tolerable risk Moderate risk Substantial risk Likely Moderate risk Substantial risk Risk level estimator Intolerable risk Risk-based thinking means to ensure that risks are identified, considered and controlled by a proactive approach. What could go wrong? How likely is it to happen? How severe is the effect? Forward reasoning from past experience and transferable knowledge. Addressing risk can include reducing or eliminating the probability of an adverse situation, or accepting and monitoring a tolerable amount of risk in order to enable pursuit of an opportunity. 30

Opportunities An opportunity is a set of circumstances that makes it possible to do something positive. Addressing opportunities can include taking on-board new improved practices, developing new products, opening new markets, reaching new customers or introducing new technology. Taking, or not taking, an opportunity presents varied levels of risk. Balancing risk and opportunity should be proportionate to their potential impact on the organisation and its customers. There should be capability, capacity and resilience of resources in being able to address opportunities as they arise. Inaction, in a changing market, can prove the greatest risk of them all. 31

Time plan 09:30 09:45 10:00 10:45 11:00 11:15 11:45 12:15 12:30 13:00 13:30 14:00 14:45 15:00 15:30 16:15 16:30 Introduction What is auditing Auditing process overview, checklist and principles Break Approach to managing objectives ISO 9001 Processes and process-approach Improvement, risk and opportunities Lunch Methods for gathering evidence Preparing and using audit criteria checklist Checklist exercise, classifying audit findings Break Audit report and follow-up Test Summing up, Q&A session Finish 32

Internal audit process Input: Resources, risks from non-conformance, previous audits outcomes Output: Schedule e.g. 6 to 8 audits per year, of 4 to 8 hours each Checklist Input: Regulation, standards, adopted code, binding agreements, Process definitions, documentation. Form: Checklist Checklist of Audit Criteria Regulation: - e.g. must CE-mark System evidence (defined) Conformity assessment process is not defined in system Process manual defines monthly meetings Process for assessing qualification Activity evidence (implemented) - - -? No purpose checking for evidence of this Records show 2 months between meetings last year Records for 10 staff all show assessments done Standard: - e.g. regular review Output: Records and observed activities Code of Practice: - e.g. qualified staff Output: Corrective actions plan Audit evidence leading to audit findings is recorded. Untraceable samples may be copied or photographed, and attached to report, for evidential purpose. If the auditor becomes aware of any new or changed assumptions or risks, then he/she should iterate back to re-determine the method and sampling. Output: Audit report, including open action for any non-conformance. Output: Closed report. 33

Methods for gathering objective evidence Intrusiveness Effectiveness Efficiency Backward tracing the audit trail Forward tracing the audit trail Method Observe activities Interview Email questions Review documentation Review records Review/test product Analyse data Random checks, if there is no path 34

Interviewing Prepare: Understand the defined audit trail Have an idea about what a conformance will look like Respectful of intrusiveness and sensitive to concern Stay focussed on your audit objectives Open questions: Can you show me how ....? Where is ....? What if ....? Does it sometimes go wrong? What then? Can you think of ways to improve it? Persist in getting to the bottom of things Record evidence Thank you 35

Sampling It is rarely practical or cost effective to examine everything. Samples should be appropriate and representative (unbiased). Target confidence level in audit findings depends on seriousness of a non-conformity. Only samples that are verifiable should be accepted as audit evidence. Verify and/or re-consider audit findings, when developing the audit conclusion. Judgement-based sampling relies on auditor knowledge and experience, for selecting samples in a complex/diverse/ interacting systems that are difficult to factualise . 36

Exercise 3 standardised checklist Divide into 2 teams. 15 minutes. Think about the purpose and benefits of checklist. Explore the idea of auditing an organisation that integrates ISO 9001 with other standards e.g. ISO 14001, ISO 45001. Report back on: how can we get best auditing time efficiency? what could a common standardised checklist look like? what are the pro s and con s of a standardised checklist? 37

Standardised checklist Requirements that are particular to the area being audited: Standards requirements Company policy requirements Objectives requirements Process defined requirements Follow-up from last audit Use standard checklist for the conformity checks and universal audit criteria, plus Select 5 to 10 further criteria, as appropriate. 38

Non-conformity Non-fulfilment of a requirement i.e. audit criteria is not met Minor: Audit criteria is partially met. Lapse in otherwise well-defined process, which has not significantly affected outcomes. A few discrepancies in records. Major: Audit criteria not addressed. Severe lapse in an important process or record keeping, where outcomes have been or could become severely affected. Multiple or repeated minor non-conformities, relating to shared root cause. 39

Not a non-conformity, but worth noting An issue that falls outside audit scope. When an approved plan for corrective action is already in place (it demonstrates that the management system works and is improving). Isolated, non-systemic lapse in procedure (insignificant as a failure). New staff not meeting competency requirement, while working supervised. Awkwardness in a procedure which could be clarified or improved. Grammar or spelling mistake in documents, which could potentially lead to misinterpretation and should be corrected. An improvement opportunity raised during the audit. 40

Non-conformity Report (NCR) Objective statement about why or how it is a non-conformance, including reference to a standard and its classification (minor/major). Statement is clear and concise - i.e. undisputable. Link multiple audit findings, to establish a root cause of non- conformity. Avoid personalising or contentious statements (blame system). 41

Exercise 4 ISO9001:2015, clause 7.5.3.2 Define audit criteria Describe audit evidence Audit criteria Audit evidence Control of documents shall address as applicable the distribution, access, retrieval, use, storage, preservation, retention and disposition. Team A operates to Supplier Evaluation process issue 1, 01/12/2016. Current document is issue 2, 01/08/2017. Evidence that suppliers SUP567, SUP789 were assessed to be unacceptable and still being used without addressing their recognised shortfalls. 42

Exercise 5 ISO9001:2015, clause 8.5.3 Define audit criteria Describe audit evidence Audit criteria Audit evidence Care is exercised with customer property, information and personal data, including identifying, verifying, protecting and safeguarding. Report to customer and document if property is lost, damaged or found unsuitable. Customer-owned product C on SO34567 damaged in storage. Damage not recorded and customer not notified. 43

Exercise 6 ISO9001:2015, clause 8.5.2 Define audit criteria Describe audit evidence Audit criteria Audit evidence Products are identifiable by suitable means, when necessary for conformity and service. Product status is identified with respect to monitoring and measurement requirements. Where traceability is a requirement, unique identification is controlled and records maintained. 1 example of Product A s/n:67890 carrying label of Product B s/n: 12345. Device History Record erroneously relate to Product B. Product A history is untraceable in database on 01/08/2017. 44

Exercise 7 Transfer audit findings into report Agree corrective actions (if any) Audit finding Note Team A operates to Supplier Evaluation process issue 1, 01/12/2016. Current document is issue 2, 01/08/2017. Evidence that suppliers SUP567, SUP789 were assessed to be unacceptable and still being used without addressing their shortfalls. Clause 7.5.3.2 requires distribution and use of documents are controlled. Customer-owned product C on SO34567 damaged in storage. Damage not recorded and customer not notified. Clause 8.5.3 requires care is exercised with customer property and any damage is identified, reported to customer and documented. Example of Audiometer A s/n:67890 carrying label of Audiometer B s/n: 12345. Device History Record erroneously relate to Audiometer B. Audiometer A history is untraceable in database on 01/08/2018. Clause 8.5.2 requires the unique identification of a medical device is controlled. OBS OBS NCR1 45

Time plan 09:30 09:45 10:00 10:45 11:00 11:15 11:45 12:15 12:30 13:00 13:30 14:00 14:45 15:00 15:30 16:15 16:30 Introduction What is auditing Auditing process overview, checklist and principles Break Approach to managing objectives ISO 9001 Processes and process-approach Improvement, risk and opportunities Lunch Methods for gathering evidence Preparing and using audit criteria checklist Checklist exercise, classifying audit findings Break Audit report and follow-up Test Summing up, Q&A session Finish 46

Audit report Unique reference and date What activity or procedure Who were involved (Auditees) Summary of findings: Notes Observation Improvement opportunity Non-conformity Any agreed action Conclusion Attach appropriate objective evidence 47

Follow-up Determine urgency i.e. when to follow-up: Early date Next audit Receive completion advice from Auditee Evaluate the action taken, re-audit if necessary Sign off (initialise, date) the non-conformity report If necessary, raise another non-conformity report 48

Q&A objectives tell me about Describe the purpose of internal auditing. Describe the responsibilities of an internal auditor. Plan, conduct and report an internal audit in accordance with ISO 19011. Describe purpose and structure of ISO 9001. Reference activities to the standard management system model. Reference activities to the audit criteria. Appreciate the model for continual improvement (plan-do-check-act cycle). 49

Test 30 minutes overall Part 1 (10 min): Answer 4 questions relating to ISO9001 requirements. Part 2 (20 min) Review a given scenario. Compare against standard audit criteria in a checklist. Write evidence section of audit report, including any NCR - if you consider that there is evidence of a non-conformity. 50