The independent validation of models is a key step of their life-cycle management. The objective is to undertake a comprehensive review of models in order to assess whether they are performing as expected and in line with their designed objective. While monitoring and validation are different processes run at different frequencies, the content of the monitoring process forms a subset of the broader scope covered by the validation process. Therefore, when available, the results of the monitoring process must be used as inputs into the validation process.
10.1.2
Institutions must put in place a rigorous process with defined responsibilities, metrics, limits and reporting in order to meet the requirements of independent model validation. Part of the metrics must be common between the monitoring process and the validation process. Independent validation must be applied to all models including statistical models, deterministic models and expert-based models whether they have been developed internally or acquired from a third party provider.
10.1.3
The validation scope must cover both a qualitative validation and a quantitative validation. Both validation approaches complement each other and must not be considered separately. A qualitative validation alone is not sufficient to be considered as a complete validation since it does not constitute an appropriate basis on which modelling decisions can be made. If insufficient data is available to perform the quantitative validation, the validation process should be flagged as incomplete to the Model Oversight Committee, which should then make a decision regarding the usage of the model in light of the uncertainty and the Model Risk associated with such partially validated model.
10.1.4
The scope of the validation must be comprehensive and clearly stated. The scope must include all relevant model features that are necessary to assess whether the model produces reliable outputs to meet its objectives. If a validation is performed by a third party, institutions must ensure that the validation scope is comprehensive. It may happen that an external validator cannot fully assess all relevant aspects of a model for valid reasons. In this case, institutions are responsible to perform the rest of the validation and to ensure that the scope is complete.
10.1.5
A validation exercise must result in an independent judgement with a clear conclusion regarding the suitability of the model. A mere description of the model features and performance does not constitute a validation. Observations must be graded according to an explicit scale including, but not limited to, ‘high severity’, ‘medium severity’ and ‘low severity’. The severity of model findings must reflect the degree of uncertainty surrounding the model outputs, independently of the model materiality, size or scope. As a second step, this degree of uncertainty should be used to estimate Model Risk, since the latter is defined as the combination of model uncertainty and materiality.
10.1.6
In addition to the finding severity, institutions must create internal rating scales to assess the overall performance of each model. This performance rating should be a key input in the decision process in each model step of the model life-cycle.