The ‘3 Ps’ for water quality management system auditing in a virtual world

The COVID-19 pandemic has flipped everything on its head, including the way we undertake water quality management system audits. 

While the ISO 19011 guidelines have provisions for remote auditing, it’s not something we normally do – we like to go out into the field as part of our evidence gathering. This is because water supply systems have big infrastructure and a multitude of management and operational systems, which usually means a requirement for physical inspection.

So, with remote auditing here to stay, let’s take a look at what it is and how you can get the most out of it to deliver a thorough water quality management system audit. 

What is a remote audit?

We have seen the words ‘remote’ and ‘virtual’ used interchangeably to describe auditing. It pays to understand the context and the grammar though, to fully understand what is meant. Auditing the virtual environment involves the collection of evidence to test how virtual systems such as algorithms, data veracity, data collection and data storage are performing.

In contrast, a ‘virtual audit’ or ‘a remote audit’ relates to the auditing conducted off-site, without physical face-to-face interaction. Data are exchanged via electronic means including Excel files, PDF files, database access and video audio linkages. 

It’s useful to think about it this way:

• If you are conducting an audit of an organisation whose products and services use an online environment, that can be accessed by any employee or authorised contractor regardless of their physical location, this is a virtual audit.
• If you are using technology to help gather information, interview personnel relevant to the audit scope, undertake a virtual tour of a water treatment plant or facilities in the absence of physical human-to-human interviews (where this method is unavailable or not preferred), then this is a remote audit.

Remote and virtual auditing in ISO 19011

ISO 19011:2018 (AS/NZS ISO 19011:2019) provides the foundational framework for auditing management systems (Davison and Loder, 2019). One of the updates in the 2018 version of ISO 19011 was the expansion of Annexe A to cover auditing guidance on relatively new concepts including:

• Organisational context
• Leadership and commitment
• Virtual audits
• Compliance and
• Supply chains.

The scope of an audit is important, as it helps to set the boundaries on what is and is not being audited – including the location. A virtual audit would also require description as part of the audit planning and records. 

Accessing audit information for the audit depends on the where, when and how components, independent of the storage, use or creation of that information.

ISO 19011 provides good guidance on remote and virtual auditing at Annexes A.15 and A.16. Auditors should familiarise themselves with the requirements of virtual and remote auditing including understanding the risks (both positive and negative – according to the ISO 31000 definition of risk i.e. the effect of uncertainty on objectives).

Benefits of remote auditing

There are a range of ‘upside risk’ opportunities to remote auditing, including:

• Increased time efficiencies for the auditor (considering matters like reduction in travel time).
• Reduction in compliance costs for auditees and auditors as there are in effect no travel-related costs.
• Reducing onsite health and safety risks in the context of COVID-19 or simply from eliminating exposures to any other physical risks associated with auditors/auditees being on-site.

Downside risks of remote auditing

Potential exists for a range of ‘downside risks’ with remote auditing, including: 

• Some government departments have very strict IT requirements, including access to third-party systems such as Dropbox (for evidence sharing) or Zoom (for video linkage). You may need to be agnostic and agile as an auditor to accommodate a diversity of sharing and access platforms.
• Having out-of-date documentation in relation to floor plans and facility maps, where that information is being relied on to conduct a virtual tour of a plant to view evidence. E.g. an out-of-date flow diagram for mapping and remote review of the process train of a water treatment plant.
• Unacceptable noise at the location being audited, causing auditee comments to be lost or misunderstood. For example blowers or pumps at a sewage treatment plant.
• Interruptions occurring at the site causing the audit to be delayed and/or disrupted. E.g. over-run on the audit time cutting into the scheduled ‘online’ window of the remote platform being used. 
• Non-observance of privacy during breaks. As with any online meeting platform, you will need to set out expectations before the remote audit interview including muting microphones, pausing of cameras and turning off of ‘smart glasses’ during audit breaks. This information should form part of your audit plan and is your responsibility, as a lead auditor, to reiterate at the start of every interview.
• Time zones. While remote tools facilitate cross-nation auditing, time zones need to be understood to pick the times for optimal concentration and availability of all parties.
• Cultural issues. Not all auditees will be comfortable with being videoed or photographed or having their information used in augmented reality environments, and care should be taken to manage auditee concerns (including the storage of any information featuring people). Permission should always be sought and received before the audit is conducted and reiterated at the start of the remote audit interview.
• Demographic issues. The water industry largely tends towards an older workforce. Auditors will need to be aware of their auditee ‘makeup’ to ensure that all auditees are comfortable with, and can use, the technologies used to perform the remote aspects of the audit. The implication of this aspect is that auditors will need to spend more time upfront, have more documentation on the process and the lead auditors will need to engage in comprehensive introductions to all in the audit team and with the auditees to ensure appropriate implementation of process.  
• Delays in getting reports to the auditees post the remote interviews. Ideally, you should get the close-out meeting and the list of non-conformances or non-compliances (for a statutory audit) to the auditees as soon as possible after the end of the close-out meeting. Audit tools are increasingly available or becoming available, which are helping to facilitate this process.
• Audit team members using unapproved devices and systems for collection of data during the remote audit, which may impact on privacy and security issues (live-streaming and mobile technologies).
• Drop-out of connections during the remote audit, meaning that the audit is unable to continue as planned.
• Lack of ‘peripheral vision’ as an auditor. With views and information flows being controlled by the auditee, observations may be limited to what the auditee thinks the auditor would like to see or ‘should’ see. When you’re there in person, you often see a lot of things along the way that give you a good feel for the risk culture, leadership and potential areas for concern to investigate further. When the auditee is holding the camera and showing you what they think you want to see, that ‘peripheral vision’ is lost, so you need to be even more focused on getting the full picture. This makes a well-designed sampling and testing methodology even more important, ensuring that findings are representative, that you have confidence in the outcomes and that recommendations are relevant.

How should you prepare?

Section 6 of ISO 19011 is comprehensive in terms of preparation for and requirements in conducting an audit. This segment of our paper concentrates on what you need to consider for the remote aspects of your audit.

As per your normal course of practice, your remote audit plan/program should be designed with the following considerations in mind:

• The risks and complexities of the audit.
• Results of previous audits.
• Technical, procedural or legislative developments relevant to the audit (noting that these may change compared to your ‘normal’ auditing considerations).

All auditors need to understand their obligations in a remote auditing world. Materials from the food and beverage industry provide a useful starting point for understanding the requirements of remote auditing as frameworks and knowledge are now in place. 

There may also be additional third-party audit requirements (such as ISO 7020 or 17021-1) for you and your organisation when developing and/or implementing remote audit practices. When exploring opportunities to implement remote audits you should take steps to ensure that you are keeping appropriately informed about applicable technical and/or legislative developments so as to maintain the capability to perform the audits, and as an auditor be competent in both the knowledge of your audit management system, and ability to implement procedures applicable to the remote audit activities being performed.

If you do not already have a privacy policy in place, you should have one. In any event, anyone certified as an auditor usually has to sign and comply with a code of conduct which will require you to not to discuss or disclose information unless required to do so by law e.g. Exemplar Global (2017). 

Make sure that your regulator is comfortable with allowing remote audits. Even in the food industry where remote auditing has been occurring for some time, there is still a mixed response in terms of acceptability, being mindful that not everyone has the technology, access to the internet or the inclination to achieve this.

As an auditee, if your organisation has a policy of not allowing externals on site, you will need to have a discussion with all audit stakeholders to come up with an acceptable solution. 

Summary

Be remote but not removed! Here are our top tips for remote water quality management system auditing in a virtual world – the 3 Ps.

Platforms (remote audit tools)

Always ensure you have trialled your remote audit platform and equipment before you begin. Some essential things to consider include: 

• Have your tablet/phone charged or charging.
• Clean your device. I.e. turn off all windows and apps running in the background (such as email) and block pop-ups.
• Make sure all applicable audit apps are updated with the latest version.
• Coordinate a time with your auditee in advance to 'test' the system (e.g. Zoom or Skype or Google Hangouts).
• Learn to utilise the ‘mute’ function as it is hugely useful when you can get it right.

Preparedness

In advance of undertaking the remote audit you should have done your homework (research and planning), including:

• The auditee should be given a clear expectation on the scope of the audit in advance.
• Ensure the auditee is the appropriate person to answer the questions you may have.
• Provide the auditee with a list of the type of documentation/data requirements before the remote audit to allow them to prepare and save time/stress during the remote audit. 
• Treat the audit as you would if you were actually onsite in person. Dress appropriately for the audience, be considerate of the situation and try to avoid external distractions. At least you won't need that hard hat (well, we hope…..).

Patience

A key virtue of any auditor. A great audit is built around being realistic that things may not always go to plan. You need to ‘plan for the worst and hope for the best'.

• An auditee who feels they are on the back foot or anxious will not be forthcoming with information or may make mistakes they would not ordinarily make, just because of the situation.
• Be mindful of providing positive feedback on the auditee's performance at the start of the remote audit with the aim of reducing potential stress and building confidence with your auditees. 
• For many water utilities, it may be the first time they have been subject to a water quality management system audit. Therefore, building a rapport with these first-time auditees is key to ensuring the audit is the best experience it can be, for all parties.
• If you need a change in behaviour, explain how this change will help the audit process.

References

Australian Institute of Food Science and Technology (2020) The virtues of remote audits - 29 April 2020. https://vimeo.com/413449577/4b6f6f5bd1.

Davison, A. and Loder, S. (2019) Using ISO 19011 for robust audits of drinking water. Certified Enterprise Risk Manager Risk Insights #254 23 November 2019.

Exemplar Global (2017) Code of Conduct for Exemplar Global certified persons. Document Ref : PCF01 Code of conduct form Edition: 7 Issued: August 2017. At: exemplarglobal.org/documents/certification-requirements/latest/pcf01-code-of-conduct.pdf

ISO/IEC 17020:2012 Conformity assessment — Requirements for the operation of various types of bodies performing inspection.

ISO/IEC 17021-1:2015 Conformity Assessment - Requirements for bodies providing audit and certification of management systems - Part 1: Requirements.

ISO 19011:2018 Guidelines for Auditing Management Systems. 

About the authors

Dr Annette Davison | Annette is the founder and Principal Risk Analyst of Risk Edge Pty Ltd (a risk assessment and management and auditing specialist company). Annette is also a co-founder of D2K Information Pty Ltd (an end to end water quality monitoring hardware, water quality intelligence software and solutions company). Annette’s background is in environmental and public health microbiology, water management system risk assessment and auditing and local government and environmental law. Annette has over 30 years’ experience nationally and internationally in the environment and water industries including lead-authoring the World Health Organisation’s first Water Safety Plan Manual. Annette is an Exemplar Global-qualified Lead Auditor and IPART (NSW) Lead Auditor in a number of categories.

Matt Parkinson | Matt is the Co-founder of Wai Comply Limited which is a specialist drinking-water compliance assessment company based in New Zealand. He has spent most of his career working as a public health enforcement and investigations officer where he specialised in drinking-water safety. His work to date includes operating as a Drinking Water Assessor (statutory officer), aiding in developing national guidelines and standards for drinking-water quality and management, and aiding with mentoring and developing future drinking-water assessors

Sarah Loder | Sarah is a qualified engineer, specialising in strategic risk management and process improvement for the water industry, with a background in both engineering consulting and management consulting. She also has postgraduate qualifications in management accounting and is an IASSC Certified Green Belt, reflecting her passion for data-driven and value-added process improvement.

From her experience as a process engineer specialising in water treatment, Sarah has a deep understanding of the Australian water industry with particular expertise in risk-based drinking water management systems. Her management consulting experience focussed on the provision of strategic risk advisory and internal audit services to public sector and water industry clients.