GUIDELINES FOR AUDITING QUALITY SYSTEMS (ISO 10011-1991) AND
GUIDELINES FOR ENVIRONMENTAL AUDITING (ISO/DIS 14010/11/12); A
COMPARATIVE ANALYSIS
by Walter Willborn, Ph.D, Certified (QMI) Auditor
58 Tunis Bay, Winnipeg, MB R3T 2X1, CANADA
April 1995
Provided to the internet community, with permission from Dr.
Willborn, by Dennis R. Arter, ASQC CQA, darter@mcimail.com
Introduction
The review of ISO Guidelines for auditing quality systems (ISO
10011) is to apply principles of continuous quality improvement;
as is searching for and identifying of improvements an objective
of any audit. For this review other audit standards can provide a
main source of information for potential improvements. We in
Canada were well served when comparing major audit standards prior
to drafting the Canadian standard for Quality Audits,
CAN3-Q395-1981, (Willborn, 1994).
The similar ISO Guidelines for Environmental Auditing ISO/DIS
14010/14011/140012 are at this time in the drafting stage. This is
a special opportune time for all concerned and interested to learn
from each others' work. Auditors of quality management systems
have gained valuable experience with the development,
implementation, and revision of the ISO Guidelines for Quality
Systems (ISO 10011) and are prepared to share this. Colleagues in
the environmental field have the advantage to start afresh in
describing features of sound auditing. They, however, prudently
used all along similar work and publications as one major
information source, as we did in Canada some years ago. When one
compares both documents, ISO10011 and ISO 14010, the cooperation
of both standard writing technical committees within the
International Organisation for Standards (ISO) becomes evident. In
the following both ISO audit guidelines are compared mainly to
identify main differences and allow for improvements.
In this comparative analysis we compare first form and structure
of both documents, and then consider major differences of the
contents. Finally, conclusions for the review of ISO 10011 are
drawn.
Both documents are called "Guidelines" using the word "should"
rather than "shall". The Audit Guideline 10011 has been prepared
during the late 1980s' by the Technical Committee ISO/TC 176 for
quality system standards, while work on the Guidelines for
Environmental Auditing commenced during the early 1990's within
the ISO TC207.
Preceding relevant documents
10011 was preceded by similar national audit guidelines in North
America (CSA Q395 and ANSI/ASQC Q1), in United Kingdom, and within
the North Atlantic Treaty Organisation (NATO). The committee for
the Canadian Q395 studied major auditing standard before drafting
their own document during the late 1970s'; Q395 was published in
1981. It was basically consistent with these other standards, but
had some innovative features. For instance it addressed quality
assurance of audits and the determination of sample size.
Unfortunately, these two important topics were subsequently
dropped in ISO 10011. A few years later, 1986, the newly published
American Q1 Guidelines included a flow-chart depicting individual
audit steps and loops. This flowchart was also discontinued in ISO
10011.
All audits in business normally share basic features and audit
principles, as we have found in our early analysis of existing
audit standards. An audit is foremost a systematic and independent
examination and/or verification of compliance.
Format and Structure
ISO 10011 divides into three parts: Auditing, Qualification
Criteria for Quality System Auditors, and Management of Audit
Programs. A bibliography of relevant ISO documents augments the
first part and a normative Annex (evaluating auditor candidates)
and an informative one (national auditor certification) is added
to the second part.
The ISO drafts for environmental auditing also consist of three
parts, but with one major difference: the first part outlines
General Principles (14010). The other two cover Audit Procedures
like 10011 Part 1 and Auditor Qualification Criteria like 10011
Part 2. Management of environmental an audit program similar to
that in 10011, Part 3 does not yet seem to have been covered in
the 14000 series. An audit program embraces a series of individual
audits that are connected by time and organization.
Early drafts for ISO14011 considered to address other related
audits than that of Environmental Management Systems, such as a
guideline for auditing site assessment. Now the ISO 14011 is
restricted to auditing of environmental management systems. These
management systems will be covered ISO14001, "Environmental
Management Systems - Specification with guidance for use". One
will probably avoid proliferation of these standards as has
happened in the ISO 9000 Series. One rather than several standards
for quality management systems would certainly enhance
effectiveness and consistency of auditing this system.
ISO standards share common features in form and structure; as they
evolve through identical standard writing procedures. Technical
committees, such as the TC 176 for Quality Management and TC 207
for Environment Management keep close contact in their work and
have official liaison. Each of the two standards compared, ISO
10011 and ISO14010, 14011, and 14012, comply with the ISO
prescribed topical layout that includes a preface, introduction,
scope, normative references, definitions, and in the body of the
documents, technical requirements. Notes are used to explain some
technical details "unofficially".
Style of writing is, in spite of notes and annexes often
unnecessarily technical and legalistic. This is in part usually
caused by the need for compromise in an international standard
writing forum. Translations and cultural differences create
another unavoidable source for misunderstanding and confusion.
Official interpretations, or even arbitration alleviate such
problems. Moreover these Guidelines with the adverb "should" allow
interpretation by experienced and knowledgeable auditors and by
standard writers at the national, industry, or even company
levels. Improved communication, understanding, comprehension, and
application of these Guidelines can come from reintroducing
flow-charts, for instance; or even special handbooks.
Major differences in content
Writers of the environmental auditing guidelines obviously studied
the then already published Guidelines for Auditing Quality Systems
(ISO 10011). The following listing of major differences in the
environmental auditing guideline, when compared with those for
quality management systems, can assist in reviewing ISO 10011. ISO
14000 standards can be expected to be a more in line with modern
technology than those of earlier date and now being subject for
revision.
"General Principles of Environmental Auditing" (ISO/DIS 14010)
emphasizes that any activity defined as an "environmental audit in
accordance with this International Standard should satisfy its
recommendations". This is an important point as it protects the
term, concept, and practice of "audit" against confusion with
other forms of quasi "audits", such as evaluations, inspections,
etc..
ISO10011, under Scope, in difference to ISO14010 states: "Each
organisation should develop its own specific procedure for
implementing these guidelines." 14010's General Principles are
declared applicable to all types of environmental audits.
This General Principles document further includes three main
topics: definitions, requirements for environmental audit, and
general principles. Introducing the reader to the terminology,
prerequisites, and principles of auditing before describing the
auditing process itself is logical and "user-friendly". After all,
these standards/guidelines serve both training and as reference
for audit participants.
Definitions in General Principles in part deviate significantly
from those in 10011, other are added. An "environmental audit" and
a "quality audit" are both "systematic" activities. But the "audit
as an examination" in 10011 is called in 14010 "a documented
verification process" and instead of auditing against "planned
arrangements" it uses the term "audit criteria". In fact, 14010
differentiates between "audit conclusion, criteria, and findings"
instead of 10011's summary term "observation". Whether using the
words "findings" or "observation" is not so important but
definitions of audit conclusions and criteria are. This is because
auditor's judgementary conclusion as a professional person is not,
and cannot, always be based on objective evidence. Still client
and auditee expect, and in 10011 are given to expect, that an
auditor concludes after the audit whether or not the system is
adequately and "effectively" implemented. Judgement by the auditor
is necessary as not all features of a system can be defined as
verifiable "audit criteria" against which evidence can be
compared. Following this example by introducing auditor judgement
more strongly into 10011 would allow broadening quality audits in
the direction of assessment and award evaluation.
Requirements for an environmental audit describe three major
prerequisites that are to be judged by the auditor: sufficient
information (i.e. evidence), audit resources, and auditee's
cooperations. It appears useful to list these esential basic
requirements of an audit early in the document, that is
immediately after clarification of terms, and before outlining
other audit principles. In 10011 these prerequisites are listed as
early considerations in the actual audit process. These are an
auditee's responsibility to cooperate and other requirements for
the audit, such as sufficient objective evidence.
Having basic requirements listed in 14010, topics und General
principles are: Objectives and scope, objectivity, independence
and competence, Dur professional care, Systematic procedure, Audit
criteria, evidence and findings, Reliability of audit findings and
conclusions, and Reporting. This topical layout indicates the
audit process flow from setting the audit objective, collecting
evidence, drawing conclusions, and reporting.
Of the principles outlined in this introductory environmental
guideline (14010) the following are either not mentioned in 10011,
or differ essentially:
+ Audit is based on objectives as defined by the client, and
both client and auditor then determine the audit scope. Both
objectives and scope are then communicated to the auditee. The
auditee in 10011 is contacted when determining the scope only "if
appropriate", but receives this, and more information, along with
the audit plan at a later stage and can then object.
+ The principle and concept of "due professional care" and
related quality assurance in audit performance are outlined here
in difference to 10011. This technical term and requirement of
"due professional care" would strengthen professionality of an
auditor. Quality Assurance of audits mentioned here in conjunction
with due professional care is obviously an important principle in
quality auditing that is currently more or less implied in 10011.
+ Under the topic "reliability of findings and conclusions"
auditor's risk of stating incorrect an finding or conclusion is
mentioned. This risks are an essential element of any audit due to
its nature of sampling and thus inherent uncertainty. Statistical
sampling techniques, however, are neither mentioned in 14010 nor
in 10011.
+ In environmental auditing the auditee should receive a copy
of the audit report, "unless specifically excluded by the client".
In a quality audit (10011) the client is held responsible to send
a copy to the auditee's senior management, which appears to be
more appropriate. All participants in an audit should be kept
informed about audit results as a matter of principle.
"Auditing of environmental management systems" (ISO/DIS14011) is
very specific, especially once the standard for environment
management systems (ISO 14001) is finalized and published.
Normative reference is to one management system standard rather
than currently to several in case of ISO 9000 series. Stipulations
are based on, and are consistent with, the preceding General
Principles outlined in ISO 14010. This approach from the general
aspects to the specific technical features of auditing leads the
reader and practitioner systematically, effectively, and
conveniently into the subject matter of auditing.
Topical layout and most of the content here are similar to 10011.
First are given some definitions, followed by audit objectives,
responsibilities and then the auditing process.
According to the definitions, environmental management system
(EMS) and quality system both are designed to implement
management; 14011 adds "maintaining" to "implementation". "Quality
management system" rather than "quality system" might be the more
precise term to be used in 10011. Environmental management system
is described in ISO 14001 Guideline. This single document rather
than three for quality systems, namely ISO 9001/2/3, or even more,
simplifies determination of audit criteria and auditing itself.
"Audit criteria", against which evidence is to be
examined/verified, is a concept not explicitly included in 10011.
In addition to audit criteria, that is requirements stipulated in
14001, other EMS requirements can be added as audit criteria "if
applicable". Audit criteria therefore are not restricted to 14010
requirements.
For audit objectives, or purpose/reason (10011), 14011 and 10011
list some examples. Practically identical in both documents are
responsibilities of lead auditor, auditor, audit team, client, and
auditee. Individual auditor functions are more detailed in 14011.
Responsibilities and activities are not separated. In both
guidelines can the lead auditor recommend improvements of the
management system "upon request" (10011), or "if agreed in the
scope of the audit" (14011).
The audit team is formed in 14011 by the lead auditor with the
client agreeing on its composition. In Quality audits the audit
program management (10011-3) selects auditors and lead auditor for
an audit project; audit program management is not addressed in
14011. In the selection process both consider similar factors.
Criteria for selecting audit team members in 14011 are:
qualification according to 14012, type of audit object, number,
language skills and expertise of the auditor, and any potential
conflict of interest. Independence of the auditor is covered in a
special clause in 10011. Observer in an audit team is not
mentioned in 14011.
The client should the provide resources while in 10011 audit
program management is responsible for it. Of course, ultimately
the client as "owner" of the audit will pay for it in both
instances.
Corrective action, initiated by the client and carried out be the
auditee remain implied responsibilities and audit completion
activities in 14011. In General Principles this desired audit
outcome of correction and usually improvement, was mentioned in a
note. Audit is completed after "all activitiesdefined in the audit
plan" are completed in 14011 and after report is submitted to the
client in 10011.
Actual auditing consists of the same logical sequential steps in
both guidelines: initiation, preliminary document review, audit
preparation, execution, and reporting. Guidelines remain very
similar and in part identical. 14011, however, in difference to
10011 addresses additional or differs in the following points:
+ change of audit scope by agreement of client and lead
auditor; in 10011 the same is possible in conjunction with change
of audit plan,
+ content of the audit plan is listed similar to 10011, but
plus: audit criteria, "procedures for auditing the auditee's EMSs
elements as appropriate for the auditee's organisation", report
format and structure, and document retention requirements,
+ auditee is kept fully informed about an impending audit and
the audit plan; 10011 allows withholding details "if premature
disclosure would compromise reliability of evidence",
+ during opening meeting promote active auditees' participation
and review site safety procedures,
+ non-verifiable statements during collecting evidence should
be recorded as such,
+ sufficiency of evidence and "auditors should examine the
basis of relevant sampling programs and the procedures for
ensuring effective quality control of sampling and measurement
processes",
+ significant audit findings should be recorded and significant
nonconformities should be documented,
+ in a note: with care, conformities might be reported, if
within the agreed scope of the audit,
+ during the closing meeting, findings should be presented in
such a manner that auditees' clearly understand them and
acknowledge the factual basis, that is not only at the time when
the findings are made,
+ disagreements should possibly be resolved before submitting
the report, final decision is with the lead auditor.
+ topics addressed in the report should be determined in
consultation with the client,
+ audit report content contains findings or a summary, should
include details about the audit itself, and may include any
obstacles encountered and general conclusions such as system
implementation and maintenance,
+ does not consider any communication by the lead auditor
between closing meeting and report submission,
+ report distribution is in accordance with the audit plan, the
client might not forward a copy to the auditee, and auditee must
permit outside distribution, it should be issued within the agreed
time period and in accordance with the audit plan,
+ auditor may not disclose any document without the permission
of client and auditee,
+ audit is completed once all activities, including possibly
corrective action follow-up, relating to the agreement between the
client, the auditee, and the lead auditor have been concluded,
(and not after report submission as in 10011).
Qualification criteria for environmental auditors are outlined in
ISO/DIS14012. The following lists topics in this and the 10011
document:
10011-2 14012
-----------------------------------------------------------
Education Education and work experience
Training Auditor training
Formal training
On-the-job training
Experience Objective evidence of education,
experience, and training
Personal attributes Personal attributes and skills
Management capabilities Lead auditor
Maintenance of competence Maintenance of competence
Selection of lead auditor
Due professional care
Language Language
Both documents have two annexes:
10011-2 14012
-----------------------------------------------------------
A (normative) Evaluating A (informative) Evaluating the
auditor candidates qualification of environmental
auditors
B (informative) National B (informative) Environmental
auditor certification auditor registration body
A "normative" annex is considered "an integral part of 10011"
while an "informative" one is not.
Major differences in stipulations of qualification criteria for
environmental auditors, when compared with those for quality
systems auditors are:
+ in 14012 it is explicitly stated "this international
Standard is applicable to both internal and external auditors,
internal auditors need the same set of competencies as external
auditors, but might not meet in all respects the detailed criteria
described herein, depending on such factors as the size, nature,
complexity and environmental impacts of the organisation, the rate
of development of the relevant expertise and experience within the
organisation,
+ auditors should have"appropriate work experience" in some or
allof the following: environmental science and technology,
technical and environmental aspects of facility operations,
relevant requirements of environmental laws, etc., environmental
management systems and standards, and audit procedures, processes
and techniques,
+ definition of 'degree" and "secondary education",
+ five years appropriate work experience( 10011: four years
full time),
+ auditor training may be provided by the auditor's own
organisation, or by an external organisation,
+ criterion for formal training in environmental science and
technology, technical and environmental aspects of facility
operations, relevant requirements of environmental laws, etc.,
environmental management systems and standards, and audit
procedures, processes and techniques may be waived if competence
can be demonstrated through accredited examinations or relevant
professional qualification,
+ auditors should have completed twenty equivalent work-days
on-the-job auditor training, for a minimum of four audits, within
not more than three consecutive years, under a lead auditor (time
requirement the same as in 10011),
+ individuals should keep objective evidence of their
education,
+ less personal attributes and skills are listed, but more
succinctly,
+ lead auditors can either demonstrate their qualification by
means such as interviews, observation, references, or
participation in the entire audit process for fifteen additional
work days of auditing, minimum of three additional complete
audits, in at least one of these participation as acting lead
auditor supervised by a lead auditor, within not more than three
consecutive years,
+ review of auditor's maintenance of competence is not
stipulated,
+ the auditor evaluation process (in the informative annex)
may be subject to a quality assurance program,
+ if an auditor registration body is established (informative
annex), the process should be subject to a quality assurance
program and a register of qualified auditors should be kept.
In general terms, qualification criteria for environmental
auditors are specified in less detail. But they demand about the
same level of competence as of a quality system auditor. This is
the more remarkable, as in difference to 10011-3 responsibilities
and activities of management of an audit program are not outlined
in a special guideline; at least not as yet. For instance,
consistency in auditing and of audit results is addressed for
environmental auditors only briefly in General Principles (14010,
clause 5.5). In the Guidelines for Auditing Quality Systems
(10011-3) management is directed and guided to monitor auditor
performance, consistency, and continuous training.
Conclusion
This comparative analysis of the Drafted International Standard
(DIS) "Guidelines for environmental auditing" (ISO /DIS14010,
14011, and 14012) was to identify possible improvements for
reviewing existing Guidelines for Auditing Quality Systems (ISO
10011). Their separate document for introducing general principles
of auditing appears to be positive particularly from the
standpoint of auditors and other users. Describing commonalities
of audits facilitates cooperation.of those that write saudit
standards and practice auditing. For quality management these
General Principles might readily and extensively be adopted. A
common or harmonized General Principles document in both
environmental and quality auditing would also allow joint audits
with mutual benefits. Clients and auditees will in due course
demand such joint audits of quality and environmental management
systems. Currently, environmental management systems auditor have
the advantage of one management system standard than three, or
even more, in case of quality management.
Audit program management, as has been pointed out, is not
considered for environmental auditing. Their guidelines deal
merely with the interplay of client, auditor, lead auditor, and
auditee in one audit project. and not with a program of individual
connected audits. Whether this restriction in the guidelines
suffices for effective auditing practice can be questioned. Lead
auditors and individual auditors normally report to supervisory
management in the auditing organisation, especially in third party
auditing. One particular individual audit seldomly stands alone;
it is connected with preceding, parallel, and subsequent audits.
Desired consistency between these audits calls for either close
cooperation between lead auditors and/or audit program management.
Organisational independence of audit program management support
also the necessary independence and "due professional care" of the
auditor in an individual audit project.
One major gap in both these audit standards is that of audit
methods and techniques, especially statistical sampling
techniques. After all, auditing is by its very nature sampling
from objective evidence. What confidence can one have in critical
audit results without application of such proven statistical
methods? As mentioned, the Canadian Q395-1981 had addressed this
topic and basic feature of any audit. The concept of materiality
that help an auditor to determine priorities in this sampling
situation is also lacking in both. Moreover, flowchart and
possibly other graphics would enhance comprehion by the various
interested and involved parties of an audit and would thus make
the documents more userfriendly.
Both standard writing groups face unique technical, political,
professional, and even personal conditions for performing their
work. Still their common mission and subject matter suggest some
communication and harmonization of the guidelines; as we have
recommended earlier for reviewing and revising10011.
References
Guidelines for environmental auditing-General principles of
environmental auditing (ISO/TC207/SC2/DIS 14010),
Guidelines for environmental auditing-Audit procedures-part
1:auditing of environmental management systems (ISO/TC207/SC2/DIS
14011)
Guidelines for environmental audting-Qualification criteria for
environmental auditors (ISO/TC207/SC2/DIS 14012)
Willborn, Walter, "Audit Standards, A Comparative Analysis" Second
Edition, ASQC Quality Press, Milwaukee, WI., 1994.