Overview
A foreign bank’s Transactional Monitoring System (TMS) was producing up to 51% unproductive alerts. Alacer Matrix LLC was asked to do model validation using historical transaction, alert, and alert investigation data.
Using model validation to reduce unproductive alerts from an international bank's Transaction Monitoring System.
Challenges
The TMS application used by the bank was not fully optimized and the alerts generated resulted in a lot of false positives, but the bank needed to understand the historical data to determine areas of improvements. Through the model validation process, the Alacer Matrix LLC team analyzed alerts and transactional data for the various rules/scenarios being used, followed FFIEC guidelines on model validation, and used statistical and analytical techniques including the following: key parameter assessment; statistical data distribution analysis; descriptive statistical analysis using means, medians, standard deviation, kurtosis and Pareto; threshold testing for alerts quality analysis, above and below the line testing, and sensitivity analysis.
Results
Analysis proved that the best means of fine-tuning alerts is payment type/transaction code. Alacer Matrix LLC data scientists provided detailed analysis of alerts with underlying transactional data, identifying factors triggering false positives/unproductive alerts. Detailed breakdowns, rule by rule, showed how threshold modifications can be applied without risk of false negatives. “What If” analysis for various threshold scenarios and projected alert volumes with “sweet spot” recommendations provided capacity savings as a result of minimizing unproductive alerts. The recommendations were accepted by the bank’s independent quality assurance and audit teams. Alacer’s recommendations were adopted, reducing unproductive alert volumes by 67% and providing insight for further optimizing the bank’s TMS.
