Institutions must ensure that a User Acceptance Testing (“UAT”) phase is performed as part of the system implementation plan. The objective of this phase is to ensure that the models are suitably implemented according to the agreed specifications.
7.5.2
The model implementation team must define a test plan and test cases to assess the full scope of the system functionalities, both from a technical perspective and modelling perspective. The test cases should be constructed with gradually increasing complexity. In particular, the test cases should be designed in order to assess each functionality, first independently and then jointly. The test cases should also capture extreme and erroneous inputs. Partial model replication must be used as much as possible.
7.5.3
There must be at least two (2) rounds of UAT to guarantee the correct implementation of the model. Generally, the first round is used to identify issues, while the second round is used to verify that the issues have been remediated.
7.5.4
The UAT test cases and results must be fully documented as part of the model implementation documentation. The test case inputs, results and computation replications must be stored and must be available for as long as the model is used in production.
7.5.5
Institutions must ensure that UAT tests and results are recorded and can be presented to the CBUAE, other regulators and/or auditors to assess whether a model has been implemented successfully. In particular, all rounds of UAT test cases and results must be available upon request from the CBUAE, as long as a model is used in production.
7.5.6
The UAT must be considered successful only upon the sign-off from all identified stakeholders on the UAT results. The UAT plan and results must be approved by the Model Oversight Committee.
7.5.7
Institutions must ensure that the model being implemented remains unchanged during the testing phase.