Is conformity assessment a quality process?

With around a million certificates issued to ISO 9001:2015 Quality management systems and sector derivatives such as IATF 16949 Automotive quality management systems and AS 9100, it seems quality management system (QMS) certification is 'fit for purpose'.  

However, you only have to browse social media platforms with an interest in this topic and it appears all is not well. There are many anecdotal examples of how the process is not working. So, the question becomes: ‘Is quality management systems certification a quality process?’  

I believe the answer is: ‘No – but it could be’. One way for the industry to move forward is by adopting a quality approach to QMS assessment and certification based on the ISO quality management principles that underpin ISO 9001 and the standard itself. 

This article references clauses of ISO 9001 and looks at how a conformity assessment body (CAB) can approach delivery of a credible assessment and certification service to its customers and, at the same time, satisfy the needs of the multiple stakeholders in accredited QMS certification. 

Context of ISO 9001 conformity assessment 

All CABs operate in a working environment (clause 4). They provide, among other services, assessment and certification of customers’ QMSs. The CAB should have a process in place for strategic planning of management systems’ conformity assessment activities. The CAB should ensure that its strategic direction is clear and is based on objectives for credible, reliable certification.  

To do this effectively, strategic planning should include: 

• An evaluation of internal strengths and weaknesses, based on evaluation of current performance as seen both internally and externally (from customer feedback and other interested parties). Effective analysis of strengths should consider first-line customers and other external stakeholders interested in the certification issued by the CAB.
• An evaluation of the market(s) in which it operates and factors of the environment that are likely to impact on its ability to provide independent conformity assessment services. Here the CAB needs to demonstrate a good understanding of internal and external issues that will affect its ability to deliver 'quality' QMS certification (to the topic of this article). Many of the issues the CAB needs to consider are related to an individual or a group of interested parties, and the analysis of issues and the needs and requirements of interested parties can be conducted at the same time.
• Internal issues will include CAB’s structure, culture, and any other services it provides alongside their activities. Internal measures provide information on its processes and the overall effectiveness of the QMS. Internal interested parties are also key:  
  • Where are the power bases within its current structure – what do current board meeting minutes and strategic plans say about the importance of its CAB activities? 
  • How effective is its QMS?  
  • Organisational complexity: regional delivery partners; joint ventures; franchisees; the mix of full-time and contracted-in auditors. 
• The CAB’s external stakeholders, including its customer base and their customer’s customers, among them major procuring organisations, accreditation bodies (AB), global and regional accreditation groups including the International Accreditation Forum (IAF). 

CAB processes 

The CAB needs to manage its core processes. 

Assessment, from capture of customer requirements through to final reporting of assessment results (clause 8.2, 8.5). 

Design and provision of new certification schemes and new methods of delivery (clause 8.3). 

Control of sub-contractors, to ensure they provide an equivalent service to full-time staff (clause 8.4). 

Certification decision-making, to ensure that process governance ensures issued certificates are valid (clause 8.6). 

It also needs to manage its support processes. For CABs, human error can have a significant impact on the credibility of third-party certification. The areas that can be affected by this include identifying of audit findings, judgement on classification of audit findings, and the certification decision. CAB competence processes are vital for certification credibility (clause 7.2). 

The CAB needs to ensure that internal and external performance measures are captured, analysed and used to maintain and improve its QMS (clause 9.1.1).  

The International Automotive Task Force (IATF) monitors CAB performance in auditing to IATF 16949, its ISO 9001-based standard. According to  current feedback  I have seen, CABs issue around twice as many nonconformities when the IATF is looking over its auditors’ shoulders than when they operate alone.  

The IAF publishes a mandatory document MD15 on Collection of Data to Provide Indicators of MS CBs’ Performance, but none relates to quality – all relate to volume of work.  

Where the CAB identifies a failure to follow its QMS, such as failure to follow the IAF MD5 requirements for sufficient audit days, then it must take effective corrective action to determine the extent of the problem to prevent recurrence across its client base (clause 10.2). This feedback can come from stakeholders such as an AB, customer or sector body. 

What can we do? 

This amounts to a call to action for all stakeholders of accredited certification to invest another 5% of effort into improving the credibility of the product. 

Auditors – do your job. Recognise that the audit process gathers objective evidence of the effectiveness of the organisation’s QMS to assess whether it satisfies ISO 9001 requirements and then report that accurately. The ISO 19011: 2018 Guidelines for auditing management systems audit principles cover all of this.  

Improve your understanding of ISO 9001requirements with: 

• Process approach (highlighting QMSs that parrot the clause structure of Annex SL).
• Risk-based thinking – focus on key areas of the organisation’s QMS.
• Context of the organisation – understand what is important to the company in the market it serves. 
• CABs – look after and enhance your reputation as a critical friend who will tell your customers how it is, without fear or favour.  
• ABs – understand the sources of variation in the QMS certification process and hold CABs to account to resolve this variability. Respond to feedback from interested parties, including sector schemes, and require CABs to convince you they have applied effective corrective action. Use the IAF peer review process to ensure the consistent application of ISO’s Committee on Conformity Assessment (CASCO) and IAF requirements across the CAB base. 
• Procurement organisations – when you experience poor quality of product or service, complain to your supplier and escalate complaints through the CB, AB and IAF regional groups. 
• Quality professionals – wherever you sit within the system, recognise the requirements under which you work, whether that is:  
  • ISO 9001 for a certified company;  
  • ISO/IEC 17021-1, ISO/IEC 17021-3 and the supporting IAF Mandatory Documents when working for a third-party certification body;  
  • ISO/IEC 17011 for an AB.  

Understanding and applying principles 

Your mission is simple, if not easy. Get to understand the underpinning principles for these requirements standards and apply them in your day job. Follow the CQI and IRCA professional code of conduct, including the below excerpts, and do your bit to improve the system one day at a time.  

Ensure that any stakeholders who may be affected by their activities are not misled or ill-informed about their level of competence to successfully discharge their responsibilities. 

Accept responsibility and accountability for their own professional actions and decisions. 

Always act in a way that supports and upholds the reputation of the quality profession. 

There are no overnight fixes. To make QMS certification into a quality product requires effort from everyone over a sustained period. 

To paraphrase an old joke for the QMS certification industry about how many clinical psychologists are required to change a lightbulb: 'Just one, but first the industry has to want to change.’