Institutions must implement a process to and monitor the performance of their models on a regular basis, as part of their model life-cycle management. The monitoring frequency must depend on model types. The required minimum frequencies are set in Part II of the MMS.
3.6.2
Prior to engaging in performance monitoring, institutions must ensure that models are used appropriately. This means that the analysis of model usage must have been completed successfully.
3.6.3
The objective of performance monitoring is to assess whether exogenous changes in the economic and business environment have impacted the assumptions of the model and therefore its performance. The monitoring process must be organised with specific responsibilities, monitoring metrics, limits associated with these metrics and required reporting for each model and/or model type. The process must incorporate a clear decision-making and escalation mechanism.
3.6.4
The responsibility for the execution of model monitoring must be clearly defined. This can be assigned to the development team, the validation team or any independent third party. If model monitoring is not performed by the validation team, then the validation team must review the quality and relevance of the monitoring reports during the validation cycle. Monitoring reports must be presented to the Model Oversight Committee on a regular basis, at least every quarter.
3.6.5
Metrics and limits must be designed to appropriately track the performance of each model based on its specific characteristics and its implementation.
3.6.6
Monitoring reports must be comprehensive, transparent and contain explanations regarding the nature of metrics, their acceptable range and respective interpretation. These reports must be designed in such way that non-technical readers can understand the implications of the observations. Each monitoring report must contain an explicit conclusion on the model performance. The report should also include suggestions for defect remediation, when deemed appropriate.
3.6.7
Upon the production of monitoring reports, a clear process must be followed to decide whether to either continue using a model (with further monitoring) or suspend it and work on remediation. This decision must be made by the Model Oversight Committee.
3.6.8
The monitoring process is a key preceding step towards the validation process. The results of the monitoring process must be used as inputs to the validation process (when available), if the monitoring reports are deemed of sufficient quality and relevance by the validator.