8 Model Usage
8.1.1
Model usage is an integral part of model management. Model usage must be defined, documented, monitored and managed according to the following principles.
8.2 Usage Definition and Control
8.2.1
As part of the definition of model strategy and objectives, institutions must articulate and document upfront the expected usage of each model. Model usage must cover, at a minimum, the following components:
(i) The users identified either as individuals or teams, (ii) The expected frequency of model utilisation, (iii) The specific source and nature of the inputs in the production environment, (iv) The destination of the outputs in terms of IT system and operational processes, (v)
The interpretation of the outputs, that is a clear guidance on how the outputs should be used, their meaning and the decisions that they can support, (vi)
The limits of the outputs, associated uncertainty and the decisions that can be supported by the model versus those that should be supported, and (vii)
The governance of output overrides.
8.2.2
Institutions must produce indicators to actively monitor the realisation of the components in 8.2.1 and compare them against initial expectations. These must be documented and reported as part of the monitoring and validation steps of the model life-cycle.
8.2.3
Any deviation between the real usage of a model and the expected usage of a model must be documented in the monitoring and validation phases and remedied promptly, by reverting to the intended usage of the model.
8.3 Responsibilities
8.3.1
The management of model usage is shared between several parties. The model owner is responsible to define the usage of his/her models. The usage of each model should then be approved by the Model Oversight Committee. If the model owner and model user are different parties, the owner is responsible to provide documentation and training to the user. The model user must therefore follow appropriately the guidance provided by the owner.
8.3.2
The monitoring of model usage can be performed by the model owner, by the validator, or both, depending on the institution’s circumstances. Irrespective of the party performing the monitoring process, the validator must conduct an independent assessment of the appropriate usage of models as part of the validation process. For this purpose, the validator should refer to the monitoring reports, when available.
8.3.3
It may happen that the model owner has limited control over the usage of a model by other parties. In this case, the model owner is responsible to report to the Model Oversight Committee any model misuse or usage without his consent.
8.4 Input and Output Overrides
8.4.1
This section refers to all model types including, but not limited to, rating models. Manual overrides of model inputs and outputs are possible and permitted but within limits. For this purpose, institutions must put in place robust governance to manage these overrides. Such governance must be reviewed by the internal audit function. Institutions must implement limits and controls on the frequency and magnitude of overrides. Models whose input and/or outputs that are frequently and materially overridden must not be considered fit for purpose and must be recalibrated or replaced.
8.4.2
During the execution phase, input and/or output overrides must be documented, justified and approved at the appropriate authority level. When necessary, an opinion from technical subject matter experts should be produced. Overrides used by the business lines must be subject of review by the risk management function before being implemented.
8.4.3
The development and validation teams must analyse and understand the reasons for input and/or output overrides and assess whether they are caused by model weaknesses. Overrides must be tracked and reported to the Model Oversight Committee, Senior Management and the Board as part of the monitoring and validation processes.
8.4.4
If a model has been approved and is deemed suitable for production, its outputs must not be ignored. This also applies when model outputs are not meeting commercial expectations. Model outputs must be considered objectively and independently from actual, potential or perceived business line interests.
8.5 User Feedback
8.5.1
Institutions must have a process in place to ensure that model functionalities are working as expected during ongoing utilisation. The objective is to ensure that models have been designed, calibrated and implemented successfully.
8.5.2
The user feedback must cover the model functionalities, stability and consistency of output against economic and business expectations. The user feedback must be documented and reported as part of the monitoring and validation steps of the model life-cycle. If model users are different from model developers, institutions must have a process in place to collect feedback from the identified model users.
8.6 Usage of Rating Models
8.6.1
Institutions must pay particular attention to the usage of rating models due to their material impacts on financial reporting, provisions, risk decisions and business decisions. Specific policies and procedures must be designed to govern overrides, including the scope of usage, the responsibilities and the conditions of output overrides.
8.6.2
At a minimum, a rating must be assigned to each obligor on a yearly cycle. In the case of exceptional circumstances related to the obligor, the industry or the wider economy, ratings may need to be assigned more frequently.
8.6.3
Consistently with Article 8.6.2, upon the roll-out of a new rating model and/or a newly recalibrated (optimised) rating model, institutions must update client ratings as soon as possible and within a period no longer than twelve (12) months. Further details are provided in the MMG on this matter.