Simon Goldsmith, Head of Risk Solutions, SAS UK & Ireland

Imagine the situation. The Chief Risk Officer of a global bank is in front of a Parliamentary Review Committee facing questions about why the bank has just made significant losses. The reason: the computer had said “Yes” to a number of decisions when it should have been saying “No”.   Regulators have recently issued very clear guidelines on how they expect banks to manage this so- called “Model Risk” in critical decision models.

Back to our CRO.  He is probably struggling to demonstrate the bank has even identified all its critical models (which are likely to be in the hundreds), never mind showing that these models are being controlled and governed in accordance with the guidelines.

Complex models are being increasingly used to support very significant business decisions across the organisation. For example:

• Are we holding sufficient capital to support our business should we have a major economic downturn?
• Are we raising sufficient capital to support our aggressive five-year growth plans?
• Could our credit approval & pricing models be significantly underestimating the risks in new business?

Model risk is inevitable

The trouble is, models by definition are simplifications of reality.  As the British mathematician George E. P. Box pointed out “essentially all models are wrong, but some are useful”.

The risks posed by model supported decisioning are broken down into two areas:

• Is the model giving inaccurate outputs &
• Is the model being misused? This could be misunderstanding of outputs or using for things outside the scope of design.

This risk of an incorrect decision being taken is exacerbated by the fact that the analytic model developers/builders are often separate from the decision takers.  This results in there being a ‘Computer Said Yes’ approach – where the person using the results from the model does not necessarily understand it.

The consequences of “poor” model guidance on significant business decisions are:

1. The bank could have insufficient capital (or cash) and fail (or require a government bail-out).
2. The bank could suffer significant losses. Even if these not large enough to cause capital problems (point 1), these will damage the bank’s value and reputation.

A wake-up call for regulators & senior management

Regulators are now much more aware of model risk and are tackling the issue. The US Fed led the way in 2011 by issuing SR11/7 “Supervisory Guidance on model risk management”. The European Banking Authority has followed US regulators and in December 2014 incorporated specific Model Risk Management assessment directions in the latest Supervisory Review and Evaluation Process (SREP) Guidance. This document is issued to all EU Regulators to direct how they conduct their periodic SREP reviews of each bank (typically annually) and is effective 1 January 2016.

As well as the increased regulatory pressure, senior management are now much more appreciative of the risk there is in the models used by their bank. It means banks now building a new approach to enterprise wide governance and control of model risk – and, crucially, mechanisms to evidence to the regulator that this has been done.

Addressing model risk

The fundamental approach taken to managing model risk is to have regular, independent checks of models. A rigorous exercise need to be undertaken when the model is first created or significant change (model validation). Then there are periodic checks on the model to check it’s still giving results within tolerances (model reviews).

The challenge for a large bank (with perhaps 2,000 significant models over many teams and countries) is how do you ensure that the entire model portfolio is getting the appropriate level and quality of model validation and review? The central risk team can issue model risk policy setting out what should be done…but how do senior management enforce this policy?

SAS has developed a new SAS Model Management solution which is already live at Discover Financial Services in the US with some excellent results to date. Before using SAS it took four-to-five weeks to collect and prepare the data, documents and reports for a CCAR (US regulatory) review. With the new solution, it took less than a week, and model interdependency and linkage were readily available.

But the real value is in providing systematic help that comes from centralising model information management. Discover has six major units and a few dozen smaller groups heavily involved in model development, deployment and usage. With this approach model risk management can be centralised, but not the actual development, testing and implementation of the models, which are left to the business units.

When in the hot-seat Discover’s CRO should feel comfortable fielding questions from the regulator about what models it’s using, how the risks are monitored and controlled and how to provide evidence this is happening.

SAS was recently ranked as a category leader in Chartis’ RiskTech Quadrant® for Model Risk Management Systems 2014.