Risk management: To quantify or not to quantify?

Patrick Parker, Director of CGR Ltd, explores whether a qualitative or quantitive risk assessment is optimal for risk management in your organisation

Those who are averse to a qualitative risk approach will complain that it is a ‘wet finger in the air’ exercise with no substance. Those opposed to a quantitative risk approach will complain that it removes human judgment from the equation. In reality, this is a false dichotomy.

Risk management, whether you consider it to be an art or science, requires an effective balance between analysis, treatment, and monitoring. Although there may be merit in a quantitative approach to risk analysis, there are also pitfalls – and a disproportionate focus on complex risk assessment at the expense of monitoring means you won’t know whether the assessment is correct (however you reached it).

Risk monitoring lends itself more to a quantitative approach: well-targeted risk indicators should guide your risk assessment, and their trends over time will support your understanding of the risk vector. This, in turn, will mean your risk treatment (controls and actions) can be more effective in keeping the risk within your appetite. If the focus is overwhelmingly on risk assessment, you will simply admire the risk rather than manage it.

Risk Analysis – What’s behind the number?

In a former public sector career, I undertook postgraduate studies in safety risk management. This included technical probability of failure assessment through failure mode analysis based on Boolean logic and algebra.

The message was that this would deliver much-needed objectivity into an overly subjective world of risk. The resulting probability figures did indeed appear more specific but were dependent on subjective assumptions feeding the algebra. The numbers just amplified the assumptions. Worst case, this meant that rubbish in led to ‘rubbish squared’ coming out.

I saw this dynamic in practice in an aircraft technical hazard assessment which sought compliance with a regulatory target of 10-7 for fatality likelihood. The input data involved historical accident rates, and the initial result didn’t meet the target – so the date period for accident data was adjusted until the ‘right’ answer was achieved.

The decision on accident data may actually have been appropriate in this case – the problem was that the 1accountable risk owner would have been briefed on the ‘right’ number without being aware (or have a chance to approve) the assumptions behind it. Quantitative assessment may have utility or even be a requirement – but any baseline assumptions must be transparent and understood.

Risk monitoring and the ‘ivory tower’

There is an inherent danger in the upper layers of business having an ivory tower view of risk which may be overly positive: either because of distance from the business ‘front line’ or subconscious cognitive bias driven by proximity to objectives and targets.

Risk monitoring should actively seek dissonance between any prevailing ivory tower view and reality. It needs to embrace an approach that looks for reds, not greens, giving you a chance to get in front of a risk event. This is a cultural mindset that needs to be driven from the top.

Risk intelligence – ‘The bringers of bad news’

There is a saying that to avoid failure, ‘the bringers of bad news must be celebrated’. Far better to learn the bad news in time to prevent an incident than to learn it during a subsequent inquiry.

An ivory tower view of risk might assume that a risk control is effective – for instance, an audit policy (for a business risk) or an alarm (for a safety risk). However, questioning those on the ground may yield challenging insights – the policy might not be followed correctly, or the alarm may not be functioning. This may be discovered through a qualitative regime of safety conversations and observations or through a more quantitative survey approach.

Risk intelligence can also be sought from other sources and monitored in a structured and quantitative way – for instance, the number of non-compliances, or near-miss events, or the number and severity of issues where the risk event has occurred. This could be taken from internal business data and/or external reporting within the industry sector. The monitoring is essentially a quantitative exercise, although the thresholds of severity will be a matter of judgement.

In any case, the aim is to detect dissonance between the risk assessment and reality on the ground. Analytics will tell you a lot, and a quantitative approach to monitoring indicators will pay dividends. You should be able to gain insights into indicator performance over time. This may allow you to correlate recurrence with specific time periods or events – and then act pre-emptively.

The importance of data integration

Effective and efficient monitoring requires an open approach to data integration. In my former career, flight data monitoring was a very tactical exercise involving huge data sets that could yield significant insights on aviation risks – but the risks were typically managed separately by people who operated above that level of detail.

For instance, the data could usefully show an increasing trend in aircraft landing long on a particular runway, thereby increasing the risk of accidents from insufficient stopping distance – but the data didn’t feed the risk unless the dots were joined manually. Automated monitoring can trigger alerts without you drowning in data, detail, and process. API connectors make this increasingly possible, and you should challenge your business and suppliers to explore this.

You need to take action

Without effective action management, your business won’t be able to respond to any of the above – and risk management doesn’t happen. Risks depend on the performance of the controls. You will need to place actions on people to put controls in place or improve control performance, and you must hold those people to account – whoever they are. Dealing with people is an inherently subjective exercise. Action performance and verification can and should be measured.

Risk assessment is a starting point, not an end in itself. It shouldn’t be disproportionately complex, whatever approach you use. If you want to manage your risks rather than admire them, you need to get beyond risk assessment: focus on the controls, and monitor indicator performance over time.

This will avoid an ivory tower view and detect changes in risk exposure in time to respond. An effective response requires a robust action management system that lets you hold people to account. However complex and sophisticated a risk assessment may appear, without action nothing happens – until it’s too late.