Management, compliance & auditing

What’s So Different with Audit?

Though aimed at InfoSec practitioners, I hope this piece will also be of interest to audit practitioners, whether InfoSec centered or not.

Audit, Assurance, Testing

There are fundamental differences of approach between security audit, assurance, and testing. For this piece, I'm going to omit InfoSec testing: though it can form part of an audit, testing is usually concerned with the verification of technical things, not of people and procedures.

A few years ago, I decided to seek out a professional audit certification, to improve my chances of getting some InfoSec audit-based work. This seemed straightforward. I had been an InfoSec professional for 18 years, most recently conducting assurance reviews. How, I reasoned, could audit be so very different for an experienced InfoSec assurance practitioner like me? In fact, some serious personal rewiring was needed for me to qualify as an InfoSec auditor. This was despite having contributed to numerous InfoSec audits.

This might seem strange. After all, the learning paths of InfoSec auditors, practitioners (and testers) should differ very little. After some self-analysis, I realized that the biggest difference between audit and assurance was in audits use of facts. As an information assurance person, I had always used fact to justify building, maintaining and improving security systems. But from my newly acquired audit practitioner viewpoint, I could see that establishing fact was the currency an InfoSec auditor must use to report accurately on the difference between agreed processes and procedures (e.g. security operating procedures, etc.) and their execution.

This is important to grasp. I have found that the different approaches used for audit and assurance, though complementary, can lead to misunderstandings, especially among non-auditor InfoSec practitioners.

The auditor's POV

There are numerous pathways to professional audit training that are not directly associated with InfoSec (for example, financial management audit). It is quite possible that a qualified auditor specializing in InfoSec may have got their professional experience in another, very different business area. But some overarching principles are to be found within common InfoSec audit standards (e.g. in ISO 19011
"…Requirements for Bodies Providing Audit and Certification of Information Security Management Systems").

Though there are variations between audit methods, all auditors will recognize a specification, set down in ISO 19011, that they should adopt an evidence-based approach. [1] Even this might not sound exceptional to an InfoSec practitioner: after all, none of us should be reaching conclusions not based on some sort of evidence. The real difference is that the auditor's priority is to check for any gaps between policy and practice and to report on what these are. It is a secondary function for them to judge the merits of processes in the overall operation of a system. Indeed, it may not be appropriate to advise a client at all about the standards they have chosen to operate. This narrowing of focus helps auditors to produce reliable reports for clients (often inside very limited timescales) about how well a system is performing against their chosen security standards.

Evidence vs. Opinion

The evidence-based approach is a cornerstone of audit. Given the need to stick to a time limit, that will include the drawing up and presentation of a report, there can be no room to enter debates about the validity of procedures. The auditee's managers will also rightly expect reasonable proof that their processes and procedures are not (or indeed are) being practiced in accordance with their security policies. A report that does not concentrate on evidence can still be useful (for example, in drawing attention to bad - and of course, good - practices). I have authored informal reviews both as a consultant and as an experienced member of staff. They have included a degree of professional opinion mixed in with fact-finding and analysis. Such an approach can still be appropriate, for instance, to assist an organization in deciding how to deploy its security resources, or agreeing levels of risk. Though such reports must be persuasive and based on professional competence, they will not usually be exclusively evidence-based. Nor is it a requirement to be a certified auditor, or to apply other audit principles, to produce them.

Fear and loathing

InfoSec assurance reviews can, therefore, be less structured and more opinion-oriented. This approach is certainly more palatable to organizations, by contrast to the formality of audit. Some of the audit approaches can be unnerving, too. For example, in the way during an audit that an auditor may ask staff to perform a process, then to silently observe and note their actions. This might appear to be at best a lofty critique of the way things are being done; at worst a prelude to a witch-hunt, where disciplinary action will surely follow revelations of wrongdoing. And in the popular mindset, the very word 'audit' is easily conflated with 'IRS,' perhaps conjuring images of government agents with special powers who are appointed to examine your shaky accounting processes - before penalizing you. Good auditors should do all they can to assuage these negative connotations, though such feelings will often remain under the surface. Yet ultimately the evidence-based approach of observed, factual assessments of processes by an experienced and impartial professional is a prerequisite for InfoSec assurance. It is also the only pathway towards successful certification to an external standard.

Internal/external/first, second, third party

These are the broad descriptions of the types of audit. Even though auditors may be trained to the same standards, internal (sometimes known as first party) audits are conducted by the organization itself, whereas external (usually called third party) audits may be conducted by another organization or by a statutory body. Audits done by an organization on one of its own suppliers take the description of second party.

From an InfoSec viewpoint, perhaps the best-known ISO standard is ISO 27001. [2] Where an organization wants its security management system (i.e. ISMS) to be certified against this standard, auditors from an external certification body will judge the application. ISO 27001 also requires that internal audits be conducted at regular intervals, [3] so organizations seeking to certify to the standard will get used to undergoing both internal and external audits

Stating the obvious

Occasionally, it might seem an auditor undertaking an audit (while having Infosec knowledge) is asking obvious questions of a limited number of staff. This goes back to the need for traceable evidence of compliance (or of non-compliance) with whatever standard they are auditing the organization against. Once enough evidence has been obtained to draw a reasonable conclusion, the auditor will usually move on to review another process. It is impossible for auditors within the time range of typically less than one week to interview all staff of a large organization about a procedure, so they must often select a sample of people to question more closely about specific processes.

The essential link to policies and standards

As I have said, it is not an auditor's primary duty to second-guess or even advise upon the efficacy of any policy or procedures they are reviewing. Nor should an auditor be first to tell an organization what policies it should apply, or how bad (or good) its internal security policies are. These are activities for the organization's security specialists, advisors, and risk managers. It is certainly possible for an organization to read between the lines of audit reporting: an audit that has turned up a lot of major non-compliances might just be an indicator that staff is not following procedures. But it could also point towards wrong or inappropriate security procedures being selected that are simply unworkable. Having identified any non-compliance, an auditor's recommendations will focus on how compliance with policy might be met in future. It is therefore still up to the organization to ensure its security policies are a good fit with their business needs as well as to any internal and external requirements. This is where audit leaves off, and information assurance begins.

Conclusion

I was glad of the opportunity to get certified to an audit point of view. Audit skills, by providing professional, even-handed and verifiable reports, are essential for the establishment of effective security controls. Other security professionals contribute to security systems through their own skills, e.g. by assessing and applying appropriate security policies and procedures. I hope this piece will improve their understanding of how an auditor can see things.

______

[1] Defined in ISO 19011:2011 para 4. f) "the rational method for reaching reliable and reproducible audit conclusions in a systematic audit process." For a comparison, see PCAOB Auditing Standard No. 15 "Sufficient Appropriate Audit Evidence - 4".

[2] ISO/IEC 27001:2013 - Information security management

[3] ISO/IEC 27001:2013 paragraph 9.2