Published: 12 May 2017
In part four of our series, Richard Green explains performance evaluation to make sure the management system is producing the intended outcomes.
The first two articles in this series focused on governance. We started by considering the role management system auditors play in determining whether an organisation has clearly defined its intent. This means asking if top management has identified what their organisation is seeking to achieve and if this intent has been appropriately communicated.
Once satisfied that there is a shared vision, the auditor’s next duty is to ensure top management’s intent is fit for purpose. To find this out we should ask if the organisation’s practices realise the needs and meet the expectations of its stakeholders, or are they more likely to set stakeholders and top management on a collision course? If the answer to both of these governance questions is yes, the auditor can now switch their focus to assurance.
Last time we considered how we might confirm the existence of ‘the golden thread’ which ties together organisational policy, strategy, objectives, plans, projects, processes and individual tasks into a single, coherent business management system.
However, a joined-up system does not guarantee the organisation will generate the outcomes it has designed the system to deliver. Auditors need to ask a second assurance question: “Does the management system produce the intended outcomes?”
How to evaluate performance
Clause 9 of Annex SL-based management systems addresses performance evaluation. This ensures organisations are able to monitor, measure, analyse and evaluate their activities.
The organisation must determine:
- What it should monitor and measure
- How it should carryout monitoring and measuring in order to ensure valid results
- When it should monitor and measure
- When the results of monitoring and measuring should be analysed and evaluated.
- These requirements apply not just to the products and services being produced but also to the operation of the management system itself.
Raising the non-conformance
Although these are decisions for the organisation, the auditor should be prepared to challenge the decisions if audit evidence suggests that the organisation’s decision is wrong. For example, a high level non-conforming output can be directly credited to a decision to employ low-cost monitoring instead of more expensive measurement.
While the organisation must retain appropriate documented information to evidence the results of its performance evaluation, the auditor may find that this is stored in a number of formats. As a result auditors need to be comfortable working with electronic records as well as hard copy ones. This means not only being able to navigate the use of the ICT system but, crucially, interpreting what the data and information is telling them.
The results of internal audit (9.2) and management review (9.3) will further inform the auditor’s assessment of whether the system is producing its intended outcomes.
The latest version of ISO 19011, ‘Guidelines for Auditing Management Systems’, explains why auditors should understand performance evaluation. The Draft International Standard (DIS), produced in April 2017 now includes new guidance for auditors on a range of competence-based topics, including how to assess performance outcomes.
As audit professionals we should feel completely at home operating in the assurance arena. Providing evidence-based, impartial assurance is at the heart of what our profession was established to do.
If you are new to profession or if you feel you want to take your existing auditing skills to the next level, you may want to check out the range of audit-related courses offered by the CQI. These provide a stepping stone to full IRCA membership, the internationally recognised ‘gold standard’ of management system audit.
With business as usual under control, we can now look to the future. We’ll examine the role of the auditor in the third competence area: improvement.
Richard Green, CQP MCQI, is the CQI representative for ISO 17021-3, ISO 19011 and ISO 45001, and managing director of Kingsford Consultancy Services.