Evaluating Your Risk Management Framework
This article is the sixth of fourteen parts to our risk management series. The series will be taking a look at the risk management guidelines under the ISO 31000 Standard to help you better understand them and how they relate to your own risk management activities. In doing so, we’ll be walking through the core aspects of the Standard and giving you practical guidance on how to implement it.
In previous articles we’ve looked at the core elements of the risk management framework, as well as the role of leadership and commitment, integration, design and implementation more specifically. In this article, we’ll be looking at how to effectively evaluate your organisation’s risk management framework.
Introduction
At this point in your risk management journey, you have implemented your framework. The step following implementation is evaluating whether or not your framework is effective. Evaluation can be tricky to do; knowing exactly what to evaluate and when can be difficult to determine, as is the case for knowing when to accept or alter the framework as a result of your evaluation.
With this in mind, we’ll be focusing on three key aspects of the evaluation process. Firstly, we’ll be turning our minds to how often evaluations should be conducted, secondly, we’ll be considering what that evaluation should focus on, and thirdly, we’ll be looking at how to determine whether or not changes need to be made to your framework, if at all.
How often evaluations should be conducted
Part of effective risk management is the periodic review of how your risk management framework is performing. This raises the question of how often such an evaluation should be conducted.
The reality of the situation is that however often evaluations are or are not conducted is at the discretion of your organisation. With this in mind however, many organisations will find it easiest to pick a recurring period to conduct the evaluation. More often than not, this will occur on an annual basis. If your risk governance is a little more robust than this, it may also occur quarterly. We’ll call this a proactive approach and it should be used as the baseline for risk management evaluation. This is because it’s predictable and it integrates a degree of certainty and therefore encourages accountability for the person or team which ultimately conducts the evaluation.
In addition to a proactive approach, you may also choose to employ a reactive approach. This is the approach to be taken when, for example, your risk management team receives a complaint that warrants considerable investigation into whether or not your framework is truly operating as it should, such as when there is a significant workplace injury. This reactive contingency approach can be effective at immediately addressing and overcoming shortfalls in your framework. It’s much more efficient than its proactive counterpart, and somewhat eases the load when it comes to improvement at the end of proactive evaluation.
Beyond employing a proactive approach as a baseline and then a reactive approach as a contingency, you may also choose, in the early days of implementation, to conduct more regular evaluations than what you would where your framework is well established. This is a particularly useful approach when you are iterating your framework in its early days.
Ultimately, evaluations are conducted at the discretion of your organisation. With this in mind however, evaluations should occur in light of the wants and needs of your organisation together with thought given to the age of your framework. The most recommended approach is that which uses a combination of proactive and reactive attitudes.
What should the evaluation focus on?
There are approximately four key factors which your evaluation should primarily focus on. These include purpose, implementation plans, indicators, and expected behaviour.
- Purpose
When you evaluate the performance of your risk management framework, you need to consider whether or not it is achieving the purpose which you originally assigned to it. This is arguably the most fundamental aspect of any framework evaluation; is your risk management framework effectively managing risk? A good place to start with this factor is comparing risk events prior to the framework’s implementation against the occurrence of risk events after the framework’s implementation. You might find this data in your organisation’s risk register.
- Implementation plans
Evaluation allows us to uncover whether or not your implementation plans have been effective. In particular, it allows us to identify and adjust those aspects of the framework’s implementation which have not been effective, as well as recognise what has been effective. For example, the way in which your framework is communicated to your team may be an issue, such as having hundred-page long policies that ground staff don’t have the time to read. How can you implement a risk management framework if you can’t communicate it effectively?
- Indicators
Periodic review and evaluation allows you to determine whether key performance indicators are being satisfied. If not, evaluation allows you to recognise where things are falling short, of which allows you to design strategies to overcome them. As we touched on in the ‘purpose’ section above, indicators which you may choose to utilise can include the number of risk events prior to implementation against the number of risk events after implementation. When designing your framework, you should select a number of indicators which you would like to equate to its successful operation.
- Expected behaviour
If you’re attempting to control or influence certain behaviours through your framework, evaluation is an effective way to determine whether or not those behaviours are being followed. This can be a difficult matter to quantify, so it may be better to take advantage of more qualitative data such as conversations with your department managers to identify any behavioural trends pre-framework implementation and post-framework implementation.
Evaluated holistically, these factors are able to reveal whether or not your framework is operating as it should. If it isn’t, then this provides sufficient opportunity for you to address that factor specifically and design strategies to improve it.
Bear in mind that these factors you can consider are not limited to the ones we’ve listed here. If you have other priorities or focuses that you’d like to take beyond these four factors, you are at liberty to incorporate them into your evaluation approach. We’d say the more robust, the better.
Determining whether it remains suitable to support achieving the objectives of the organisation
Once you’ve completed the periodic evaluation of your framework, you need to determine whether or not it remains suitable for supporting your organisation’s goals. If the plan doesn’t align with your organisation’s wants, needs or expectations, be prepared to make adjustments to your framework to better accommodate for this. Ideally, this won’t involve an overhaul of the framework as a whole and then trigger a complete restart of the process. Rather, it will involve an iterative approach which specifically addresses those aspects that need the most attention. This tailored approach to solution design is often also the most efficient.
Conclusion
Evaluation is a critical aspect of ensuring that your risk management framework is operating to the best of its ability. Without evaluation, we can find ourselves with a framework that merely exists and offers no actual benefit to risk management.
For this reason, we need to conduct regular evaluations which are specifically focused on the core factors of purpose, implementation, indicators, and expected behaviour (but they can focus on more than this if you please) and then following the results from this evaluation, you need to determine whether or not the framework remains suitable for supporting and achieving your organisation’s risk management goals. If it doesn’t, you need to design and implement strategies to overcome these shortfalls.
If you have any stories – good or bad – about how you’ve evaluated your risk management framework, I would love to hear them.
If you’re looking to implement a risk management framework and would like some guidance or a conversation to help you on your journey, please contact me. I’m more than happy to guide you.
