4 Model Governance
4.1 Overview
4.1.1
Institutions must develop and maintain policies and procedures that support their model management framework. In addition, they must regularly ensure that these policies and procedures are correctly implemented.
4.1.2
In addition to the elements mentioned in Part I, institutions must include the following components in their model governance framework, at a minimum: (i) the definition of model objectives, (ii) steps of model life-cycle, (iii) model inventory, (iv) model ownership, (v) identification of key stakeholders involved in decision-making, (vi) relations with third parties, (vii) adequacy of internal skills, (viii) comprehensive model documentation, and (ix) reporting.
4.2 Model Objectives and Strategy
4.2.1
Institutions must assign a clearly defined objective to each model and include it in the model development documentation.
4.2.2
If stakeholders disagree on the objective of a model, the model must remain under development or be removed from production until the disagreement is resolved.
4.2.3
Institutions must have a defined strategy to meet the objectives of their models. Institutions must distinguish between short term tactical solutions from longer term solutions. Such strategies must be documented and approved by the stakeholders involved in model management, including Senior Management and the Board.
4.2.4
The modelling strategy must clearly articulate the potential contribution of third party consultants to the development, management and validation of models. The outsourcing strategy must be defined and justified, in particular regarding data, systems, calibration and methodology design. If a quantity of portion of modelling work is outsourced, institutions must implement mechanisms to retain controls control over the key elements of modelling.
4.3 Model Life-Cycle
4.3.1
Institutions must manage each model according to a cycle that includes, at a minimum, the following steps.
(i) Development, (ii) Pre-implementation validation, (iii) Implementation, (iv) Usage and monitoring, (v) Independent validation, and (vi)
Recalibration, redevelopment or retirement, if deemed necessary.
4.3.2
The duration and frequency of each step must be specified in advance for each model and documented accordingly.
4.3.3
Upon independent validation and the response from the development team, the following decisions must be considered by the Model Oversight Committee, which must all be thoroughly justified:
(i) Leave the model unchanged, (ii) Use a temporary adjustment while establishing a remediation plan, (iii) Recalibrate the model, (iv) Redevelop or acquire a new model, or (v)
Withdraw the model without further redevelopment.
4.4 Model Inventory and Grouping
4.4.1
Institutions must maintain a comprehensive inventory of all their models employed in production to support decision-making. The inventory must cover internal models and models provided by third parties. It must contain sufficient relevant information to support model management and mitigate Model Risk.
4.4.2
The inventory must cover models both currently in use and employed in the past for production (starting from the implementation of this MMS). Institutions must ensure that they can refer and/or roll back to previously employed models, if necessary. Consequently, institutions must have a model archiving mechanism in place supported by appropriate documentation and IT system infrastructure.
4.4.3
Each model must have a unique nomenclature and identification number that must be explicitly mentioned in any related model documentation. A model with a new calibration must carry a different identification number. Any variation of a model requiring a separate validation or approval should be identified as a separate model.
4.4.4
The model inventory must include, for each model, all the references and documents pertaining to each step of the life-cycle. Amongst others: (i) the dates of each step, including past and planned steps, (ii) the internal party responsible for each step, and (iii) previous validation exercises and audit reviews plus any reference to their respective outcome. Third-party consultants must not be considered as responsible for any step but only considered as supporting their execution. Where consultants have been involved, the details of the consultants must be recorded.
4.4.5
Models must be grouped based on their associated Model Risk.
(i)
At a minimum, institutions must create two groups referred to as Tier 1 and Tier 2 models, with Tier 1 models being more critical than Tier 2 models. If institutions already employ more than two groups, those can be retained for internal purpose. In the context of the MMS and for regulatory purpose, the models deemed less material than Tier 2, must be regarded Tier 2. (ii)
Whilst the grouping decisions are left to the discretion of each institution, they will be reviewed by the CBUAE as part of its supervisory duty. At a minimum, IFRS9 models for large portfolios (measured by exposure) and capital forecasting models must be classified as Tier 1. (iii)
Institutions may prioritise model management by tier once they have established a clear grouping framework based on Model Risk. In the MMS, in the absence of specific reference to model tiers, the requirements apply to all models irrespective of their materiality, as these requirements must be regarded as fundamental building blocks of model management. Where needed, the MMS explicitly refers to model Tier 1 and Tier 2.
4.5 Model Ownership
4.5.1
The concept of model ownership is fundamental to model management. Institutions must ensure that an internal owner with a sufficient level of seniority is assigned to each model at all times.
4.5.2
The owner of a model is accountable for all modelling decisions and for ensuring that the model goes through all the steps of its life-cycle in a timely fashion. In other words, a model owner is not responsible for executing all the steps; however, a model owner must ensure that the steps are performed.
4.5.3
Risk models involving statistical calibration must be owned by the risk department and must not be owned by the business lines to avoid conflicts of interest. Pricing and valuation models used for commercial decisions can be owned by the business lines. Other financial models with no statistical calibration can be owned by the finance department, at the discretion of each institution.
4.6 Stakeholders and Decision Process
4.6.1
A modelling decision is defined as a deliberate choice that relates to each step of the model life-cycle. In particular, key modelling decisions relate to (i) the model strategy, (ii) the choice of data, (iii) the analysis of data, (iv) the methodology and the development process, (v) the calibration, and (vi) the implementation of models. Such decision have material impacts on model outcomes and have financial implications. Consequently, institutions must implement a clear governance process around these decisions.
4.6.2
All parties involved in making decisions required at any step of the model life-cycle must be identified and recorded in the model documentation. Within an institution, individuals may hold several of these roles (i.e. several responsibilities), with the exception of model validation which must remain independent from the other roles. At a minimum, the following roles must be identified for each model:
(i) Model owner, (ii) Model developer, (iii) Model validator, (iv) Model user, (v) Modelling data owner, and (vi)
Model Oversight Committee members.
4.6.3
Institutions must establish a Model Oversight Committee, to whom the stakeholders mentioned at 4.6.2 are accountable. This committee must be established separately from existing risk management committees. Its scope must cover all models across the institution, with the view to manage Model Risk in its entirety. The committee must convene regularly and at a minimum every quarter.
4.6.4
The Model Oversight Committee must provide substantiated decisions related to each step of the model life-cycle and in particular, strategic modelling options. Consequently, the committee members must have a minimum level of technical understanding to be able to contribute to those decisions.
4.6.5
The Model Oversight Committee must be accountable to Senior Management and the Board. The committee must provide an impartial view of the best modelling approach for the institution. It must remain independent from actual, potential or perceived interests of business lines. Therefore, the majority of the Committee members must not be from the business lines. If business views and risk management views related to modelling choices are irreconcilable, Senior Management must make a decision, be accountable for it and provide a clear rationale for it. The final decision must be in compliance with the requirements outlined in the MMS.
4.6.6
At a minimum, the Model Oversight Committee must hold the following responsibilities.
(i) Design the institution’s appetite for Model Risk to be approved by the Board, (ii) Ensure that Model Risk is managed appropriately across the institution, (iii) Escalate modelling decisions when necessary, (iv) Oversee the objective and strategy of each model, (v) Approve the development of new models, (vi) Request the development of new models when necessary, (vii) Approve material modelling decisions throughout the model life-cycle, (viii)
At the end of each cycle, review the validation results and make a choice amongst the options presented in the section 4.3 on model life-cycle.
Whilst some technical aspects of these responsibilities can be delegated to subcommittees, working groups and/or individuals, the Model Oversight Committee must remain the centralised forum where modelling decisions for the whole institution are discussed, made or proposed for escalation. Material modelling decisions must be ultimately approved by the Board.
4.6.7
Other subject matter experts across the institution and third party experts can contribute to each step of the model life-cycle depending on their field of expertise. They can be involved in model design, development and testing. However, their involvement must be viewed as consultative only.
4.6.8
The CRO is responsible for ensuring that Model Risk is managed appropriately. Consequently, as part of his/her duty, the CRO must ensure that:
(i)
Model Risk is appropriately identified, understood, estimated, reported and mitigated across the institution. (ii)
The governance for model management is efficient and appropriate to the size and complexity of the institution. (iii)
The Model Oversight Committee is functioning appropriately and meets the responsibilities outlined in article 4.6.6. (iv)
Material modelling decisions are approved by the Board (or the Board Risk Committee). The Board is adequately informed of Model Risk, the status of model management and the performance of models. (v) A suitable escalation process is in place through the institution and up to the Board. (vi)
The institution employs adequate resources to meet the demands of model management and, where required, escalate identified gaps to Senior Management and/or the Board. (vii) He/she is fully familiar with the requirements articulated in the MMS. (viii)
He/she has sufficient technical understanding to form an opinion about the modelling decisions with material financial implications. (ix)
He/she is sufficiently informed of material modelling decisions, in such a way that he/she can articulate a view about the suitability of these decisions. (x)
Particular attention is given to the quality, completeness and accuracy of the data used to make decisions based on models.
4.7 Third Party Provider
4.7.1
Institutions must remain the owners of their models at all times, under all circumstances. They must remain accountable for all modelling choices, even in the case of support from a third party consultant for any of the steps in the life-cycle.
4.7.2
If modelling support is provided by a third party, institutions must take the necessary steps to transfer knowledge from that third party to internal employees within a given time frame. This requirement applies to any of the steps of the model life-cycle.
4.7.3
Third party providers may offer a range of modelling contributions covering, amongst others, methodological support, system infrastructure, validation services and ready-made calibrations based on external data. Institutions must take the necessary action to fully understand the contributions provided by third parties. This requirement applies to all models and to all risk types.
4.7.4
In the case of methodological support, whilst institutions must operate within the constraints of the acquired model, they must demonstrate that the method is adequate to their portfolios. If a methodology acquired from a third party is not fully understood by the institution, then it must not be considered fit for purpose. If a third party provides a methodology to an institution, any subsequent validation exercise must be performed by an internal or external party independent from the original provider.
4.7.5
If a third party provides a ready-made calibrated model based on external data, such a solution must be justified, based on the following specific circumstances:
(i)
For portfolios and metrics for which an institution is not able to collect sufficient internal data, then externally calibrated models are acceptable. For instance, this applies in the case of low default portfolios or small portfolios for which data collection may not lead to statistically representative samples. (ii)
For portfolios and metrics for which an institution is in a position to collect internal data, then externally calibrated models must not be used. Externally calibrated models are acceptable, only temporarily over the short term until sufficient data is collected. In this case, immediately after the model implementation, institutions must take the necessary actions to (i) collect historical internal data from internal systems and (ii) collect future internal data in order to develop a model internally.
4.8 Internal Skills
4.8.1
Institutions must ensure that they acquire and retain adequate internal knowledge and core competences about modelling techniques. Full model ownership requires that institutions must have an appropriate number of internal employees with technical skills to understand and own models, even with the contribution of third parties. The contribution of external consultants cannot justify a lack of internal technical employees.
4.8.2
All institutions must ensure that they have a minimum number of technical employees to manage models independently of third parties. The skills of these employees must sufficient to cope with the complexity of the models implemented at the institution. If an institution does not have the required internal skills to manage complex models, these models should be simplified or replaced.
4.8.3
For branches or subsidiaries of foreign institutions, the internal technical expertise may reside at the parent group level, which are responsible for the oversight of the local implementation and/or usage of models. The technical experts from the parent entity must also oversee any third parties employed to deliver models for the local entity. The local branches or subsidiaries must nonetheless have employees with sufficient skills to ensure that models are suitably calibrated to the UAE context and meet the CBUAE requirements in this regard.
4.8.4
Knowledge about a model must not be restricted to a single individual in the organisation. Instead, knowledge must be shared amongst several staff members. This is necessary for the purpose of sound decision-making related to modelling choices and to minimise the impact of staff departure on the smooth continuation of model life-cycle execution.
4.8.5
Institutions are expected to recognise the scarcity of technical staff able to genuinely understand and own models. Therefore, they must put in place development plans and initiatives to retain and manage their technical employees appropriately. The strategic management of technical resources must include full and adequate cooperation of the institutions’ human resources function.
4.9 Model Documentation
4.9.1
Dedicated and consistent documentation must be produced for each step of the model life-cycle. The documentation must be sufficiently comprehensive to ensure that an independent party has all the necessary information to assess the suitability of the modelling decisions. In particular, the documentation must make a clear distinction between theoretical considerations, calibration choices and practical implementation considerations.
4.9.2
All model documentation, model management policies and procedures must be an accurate reflection of the institution’s practice and usage. In other words, institutions must ensure that the model attributes described in a modelling document are actually implemented. Any gaps and partial implementation must be recorded, tracked and reported to Senior Management and the Board by the modelling stakeholders. Institutions must have a remediation plan in place to address each of these gaps within an appropriate timeframe.
4.9.3
Institutions must develop internal standards for model documentation across all model types, with rigorous document control. This requirement is particularly relevant for the development and the validation steps. The documentation must be adapted to the type of model, the business context and the step of the life-cycle. At a minimum, all model development documentation must include the following information:
(i)
Document control including the model reference, owners, contributors and key dates of each life-cycle step, (ii) Model materiality in relation to the institution’s risk profile, (iii) Overview of the model strategy, structure and expected usage, (iv) Data set description, when applicable, (v) Methodology and modelling choices related to all the key modelling decisions, (vi) Modelling assumptions, weaknesses and limitations, (vii) Expert judgement inputs if any, (viii) Impact analysis of the new modelling decisions, and (ix)
Implementation process and timing of the new modelling decisions.
4.10 Performance Reporting
4.10.1
Institutions must implement a comprehensive reporting framework to ensure that Model Risk is analysed and assessed for the purpose of implementing risk mitigating measures.
4.10.2
Reporting must be implemented at several levels of the organisation, including to the Model Oversight Committee, the institution’s Risk Committee and the Board. Reporting must be specific and adapted to the nature of the stakeholders. The status of model management and Model Risk across the entire organisation must be presented to the Model Oversight Committee and the institution’s Risk Committee at a minimum on a quarterly basis, and to the Board or a specialised sub-committee of the Board at least on a yearly basis.
4.10.3
Reporting must be designed to support Model Risk management covering the identification, measurement, monitoring and mitigation of these risks. In particular, reporting must cover (i) the status of the model lifecycle for each model, (ii) the results of model performance assessment, (iii) the risks arising from the uncertainty surrounding certain modelling decisions, and (iv) the status and estimation of Model Risk throughout the organisation.
4.10.4
Institutions must comply with model reporting requirements from the CBUAE, as they evolve through time.
4.11 Mergers, Acquisitions and Disposals
4.11.1
If an institution merges with or acquires another institution, it must re-visit all the elements of the model management framework, as part of the integration process. The modelling framework and all the principles of model life-cycle management must be applied consistently across the newly formed institution. In particular, model ownership must be clearly defined. The newly formed institution must have sufficient resources to fully manage the new scope of models.
4.11.2
The scope of models must be re-visited to assess whether there is a degree of overlap between models. Depending on circumstances, models may need to be recalibrated or redeveloped. Models must be representative of the risk profile of the newly formed institution. In the case of overlap between two similar models, a new single model must be developed based on a larger data sample. This new development must occur promptly after the completion of the merger or the acquisition.
4.11.3
Institutions must pay particular attention to the integration of historical data, and future data collection, subsequent to the merger or the acquisition. This requirement applies to all data fields used as inputs to the existing models and to the future models to be developed, in particular, default rates and recovery information. Historical data time series must be reconstructed to reflect the characteristics and risk profile of the newly formed institution. Upon the implementation of the MMS, this requirement applies retroactively to cover, at a minimum, a full economic cycle in the UAE, and where possible covering the 2008 global financial crisis. Future data collection must be performed for the entire scope of the newly formed institution.
4.11.4
In the case of the disposal of an entity, a subsidiary, a branch and/or a large portfolio, institutions must ensure that the modelling framework and all the principles of model life-cycle management are adjusted to fit the needs of the reduced scope of portfolios, products, obligors and/or exposures.