Context
The project is composed of two parts:
- Vetting internal and third-party models for compliance with Federal Reserve Board SR 11-7 (“Supervisory Guidance on Model Risk Management”). Performing tests to evaluate the model under normal and stressed market conditions.
- Producing up-to-date documentation on a proprietary model suitable for reference by users, and for review by internal audit.
The Project
Arrayo coordinated with model developers, market risk function, and head office oversight, and liaised with support personnel from vendors in order to produce:
- A report for each model consisting of assessment and documentation of assumptions, limitations, evaluation of conceptual soundness, ongoing performance monitoring/benchmarking, and outcomes analysis/back testing. Documentation followed internal model risk standards, expressed in templates.
- A professional report including the addition of a step-by-step explanation of important procedures, and definitions of key terms that were previously implicit. Added appendix with exhaustive listing of configuration files. Added screenshots showing details of configuration and functionalities.
Outcome
- Memorialization of any prior testing done by the client. Strategic testing to determine whether model risk is controlled. Discoveries of unknown limitations and bugs in models. Communication to head office. Final reports are assets to be deployed as and when the Fed asks.
- Enhanced text to explain an aspect of methodology of commonly traded instruments based on request of internal audit. Readability improved: added table of contents, enlarged figures enlarged to show key features, and so on. Changed aspirational language (“the model will do …”) to actual (“the model does …”), reflecting the current implementation. Communicated with head office.