Risky Business – Auditing, Auditors and Audit Outcomes
Auditing is risky business. The choice of becoming an auditor, choosing your field of expertise, gaining the understanding of the standards and applying to technical areas all relies on a professionals competence. What underlies this competence is the confidence to complete the task. This confidence is achieved through the ability to resolve the risk inherent in the process. Let’s talk risk.
Compliance auditing, surveillance and inspectorate audits, internal audits and reviews all operate on the same premise; review past performance against current practice to predict future compliance. Another way of expressing this is the professional is conducting a risk analysis. They are looking for risks, assessing their likelihood of occurrence and the severity of the risk in the event the risk is realized. If the risk analysis is high a corrective action is issued; if it is low an observation; if the risk is managed they are in compliance.
But risk is subjective and depends on an individual’s assessment of likelihood and severity which in turn is determined by an individual’s appetite for risk. Let me expand on this topic, risk is usually equated with finances or health and safety. Gambling or investment is something we attribute to risk. Would you leverage your life savings on an investment? Some of you would have immediately yelled out “Are you crazy?!!” whilst others may have said ”depends on the odds”. In health and safety if I had said to you “let’s go cave diving this weekend” some of you would have said ”Never in a million years!!” whilst others may have said “Sounds like fun!”.
This concept of risk is emerging in the auditing profession as an attribute that is to be considered. Currently the ISO /TC176 committee is developing a new ISO:19011 that integrates risk. In sections 5.2.3; 7.3.1(e); 7.3.3(b); 7.6.2; A.2 – A.4; B.2 – B.5 they all speak of knowing how to identify and assess risk as an audit outcome.
So the theory is sound that an auditor needs to perform risk assessments before, during and as part of making findings but where is the training coming from, and more importantly how do we understand the risk appetite for the auditor/team leader? From my experience risk management training is not a part of any auditor training courses. I also have not yet seen an agreed upon international standard for risk management. So how do we determine auditors’ competence in risk management knowledge? In this instance there is no ability to rely on industry experience as risk has not yet become part of the language of “doing business”. OK environmental impact studies and workplace safety assessments are prevalent but not common place and they do not translate into overarching risk assessment methodology.
Next there is the question of risk assessment methodology. Are we talking Monte Carlo Analysis, Pareto, Causal Analysis, or combinations? How should the audit professional determine when to deploy various tools or understand their appropriateness.
Now onto my subject of choice, the individual. Risk appetite of the individual is to be considered. Joining the profession of auditing is made with some understanding of the risks involved. In some fields audit/inspectorate outcomes carry a legal responsibility for the lifetime of the product or service inspected. i.e. electrical inspections. After an electrical installation is completed it falls upon an individual to inspect for compliance to rules and safety of use. Should the inspector sign off on the work they are as liable as the installer for the lifetime of the installment. Death, injury or damage caused by faulty installations is as much the responsibility of the inspector as the installer.
In the conformity assessment field (EMS, QMS, OHS, FS) an auditor makes claims that the outcomes are only as good as the evidence presented at the time of audit. Let’s take Food Safety (FS) for instance. An auditor can inspect an manufacturer of food product and make the assessment that based on the evidence presented they are following food safety practices and food is safe to eat from this manufacturer. In the following week the factory undergoes a major product recall based on food safety risks. So where is the auditor in this chain of events? Somewhat protected by the disclaimer of audit and yet the risk assessment they performed is an outcome of the auditors “satisfaction” with what they have observed, discussed, and reviewed. There is a disconnect.
Understanding the knowledge competence for risk is equally as important as understanding the personal risk attributes of the auditor. In the financial sector, assessments of risk attributes of financial planners and their clients is already occurring. In the United Kingdom the Financial Services Reform Act says that risk must be addressed before providing service. So it is possible to assess personal risk behaviors as a quotient of providing a competent, managed (risk) service.
Can the risk behavior assessment be adapted to the conformity assessment sector? Yes. We are in the process of “breaking ground” in this area. Behavior or personal inventory has been a consideration of auditor competence since 2004 and has been deployed through psychometric tools.
The term psychometrics was first used early in the 20th century and is defined by Merriam Webster as “the psychological theory or technique of mental measurement”. Although the term is somewhat new measuring the mind is an ages old technique, dating back to 210 B.C. with the Han Dynasty in China. Great strides have been made over the last century in the science of measuring mental processes. It is easiest to think of “mental measurement” in three primary areas: measurement of knowledge, measurement of skill (or performance) and measurement of psychological attributes.
The declaration that an individual is competent can be made based on measurements of these three areas. Using risk management as an example: the knowledge of risk and the means in which it is identified and assessed, the skills involved in responding to the risk, and lastly, the individual’s tendency towards (or aversion of) risk. If we begin our argument that all individuals engage in some sort of risk-taking, we must clarify in what environment that individual is taking risks and why. The individual who said “depends on the odds” when investing their life savings may say “never in a million years!” about cave diving because he is an investment banker and not a strong swimmer.
Why this difference and why would it vary among individuals? While it is relatively easy to measure one’s knowledge and skill, it becomes more difficult to measure how an individual may behave since to do so requires partly a measurement of personality. Measuring an individual’s knowledge of risk and the means to identify and assess it can be easily quantified by developing and administering a multiple choice test, from which we receive a specific range of scores that can be interpreted as pass/fail. Measuring an individual’s skills at responding to the risk can also be quantified either by developing and administering a writing assessment or using a scoring rubric and assessing the individual as they respond to a risk. These results can also result is a very specific range of scores that are interpreted against pass/fail standards.
Measuring an individual’s risk aversion or tendency becomes more difficult because one’s own risk type may depend on their knowledge level, their skills as well as aspects of their own psychological attributes, such as personality. One such example that many of us are familiar with is the linking of the type A personality with the increase in heart problems. Dr Meyer Friedman linked the highly competitive, high strung personality to a higher probability of heart risk based on his observations and later study. While many of us might conclude that we cannot change our personality, we can change our behaviors that are correlated with those personalities. One Friedman study[1] found that those who received counseling had a marked decrease in behaviors that are typical of type A personalities. We might conclude that as the individual became increasingly self-aware of their personality behaviors, they could decrease their heart risk.
The same could be true for an individual with their risk type. One might argue that if individuals have adequate knowledge and skills within an industry, their risk tendencies will mirror those of their peers.
With this information at hand, personality risk assessment can be used to gain a better understanding of risk-based audit outcomes. The theory of identifying auditor risk is to allow an examination of the likelihood of a mistake or an incorrect or incomplete audit. It’s likely that those working in audit roles may tend towards a specific personality profile and furthermore, that high performing auditors may have an even more distinct profile.
The Risk-Type Compass is a psychometric tool that can be used to measure an individual’s “predisposition to risk and their capacity to manage it”[2]. The two main personality scales that underpin risk predisposition are estimated to be Calm:Emotional, which concerns the more emotional side of risk taking, from fearful or anxious through to composed and optimistic; and Daring:Measured, which indicates an individual’s preference for a methodical approach or conversely, a spontaneous and adventurous approach to risk.
Auditors are likely to need to be prudent, thorough, organised and compliant. These are characteristics associated with a Measured rather than Daring disposition, or a lower score on the Daring:Measured scale. This leads to the hypothesis that auditors will have lower levels of the personality scale Daring:Measured compared to the general population.
Calm:Emotional concerns emotional stability. High scorers on this scale are likely to be cool headed, calm and optimistic, but at the extreme seem almost oblivious to risk. Those with lower scores are likely to be apprehensive and pessimistic about risk taking and alert to any threats in their environment. They will put security at the top of their agenda. This could be linked to an ability to spot the risks associated with products assessed by auditors. This leads to a second hypothesis that auditors will have lower levels of the personality scale Calm:Emotional compared to the general population.
These personality scales are used to place individuals into Risk Types, ranging from the most risk averse, the Wary type, to the most risk tolerant, the Adventurous type. It is likely that certain Risk Types will be more prevalent, specifically, there will be a greater prevalence of Risk Types associated with a more apprehensive, careful and cautious approach to risk taking, i.e. Wary, Intense and Prudent types.
Furthering the concepts of personal risk the theory can predict patterns of work. It may be the case that teams with a balanced distribution of Risk Types perform better than those with concentrations of certain Types. Conversely, it could also be the case that those teams with high numbers of certain types (such as those likely to be associated with higher Auditor performance) will perform better than those with a balanced mix.
There are those among us that are regarded as “good” auditors for one of many reasons. One reason is how we conduct ourselves on site using our attributes and talents. The professionals in our field may owe their performance to the way that they manage themselves, i.e. they are ‘strategically self aware’. Such people would be aware of their strengths and limitations and understand how they may impact on others. They may know how to compensate for weaknesses/ rein in excesses/ maximise their assets/ or to rack up their performance.
‘Strategic Self Awareness’ or ‘Political Awareness’ are ideas that crop up in discussion, or as a focus of coaching, but I am not aware that such an assessment exists as a formal psychometric. We are working on this now with Psychological Consultancy, Ltd. and will have a research project for risk-types among auditors operating shortly.
Using the theory we could predict if the auditor is placing themselves and the client at risk using psychometric analysis. This data would be used to influence choice of auditor for standard, technical area and risk level of process. It can even be used to determine certification type, length and CPD activities.
Risk management, when planning, undertaking or reporting on audit outcomes, is an emerging area which requires research and analysis. The results of assessing risk in personnel and the process will further advance the industry and the auditing professional. Whilst the TC176 committee has begun to frame the premise of risk and how it should be demonstrated the market must be given time to define, examine and record the risk of the audit professional.
The risk of managing risk is not yet managed and forms all good intent without direction. I am hoping to provide some direction by conducting this study of risk of personnel and apply it to the process of certification. I like to think that this is an exciting and important service being offered back to the industry. Should you be interested in becoming a part of the research work and survey group please contact me.
About the author
Peter Holtmann is the founder and Director of Holtmann Professional Services, a global provider of business risk management and transformation practices. Peter has more than 25 years of experience in executive roles and has been the President and CEO of a global Non-profit. Peter has written for many journals and blogs, is a keynote speaker and is a champion of prosperity through excellence of leadership.
Peter can be contacted at www.holtmann.com.au
[1] Friedman, M., et.al. (1986). Alteration of type A behavior and its effect on cardiac recurrences in post myocardial infarction patients: summary results of the recurrent coronary prevention project. American Heart Journal, 112(4), 653-665.
[2] (Trickey & Stewart, 2010) (insert ref for trickey and Stewart’s tech manual here)
