ControlOne CEO argues explainable AI need not weaken crime detection

ControlOne founder and CEO Swagatam Sen has argued that banks have spent years framing financial crime technology around a mistaken trade-off between explainability and detection performance. In an external opinion published by Finextra, Sen said that premise has influenced billions of pounds of industry technology spending.

Sen, whose company is based in the UK, said financial institutions have often treated rules-based systems as easier to defend to regulators, while viewing machine-learning models as more effective at identifying suspicious activity but harder to explain. He argued that the choice is not between transparency and performance, but between models that are built to explain their decisions and systems that try to add explanations after a decision has been made.

Post-hoc explanations under scrutiny

According to Sen, the problem often starts when a bank deploys a complex model, such as deep learning or gradient boosting, that performs well in testing but cannot give a direct account of why it identified a specific account or transaction as suspicious. He said institutions then add separate tools, including SHAP values or LIME-style attribution methods, to infer how the model may have reached its conclusion.

Sen described those add-on methods as approximations rather than the model’s own reasoning. In his view, that weakens their usefulness for model risk committees and compliance teams that must justify decisions in regulated settings.

The alternative, he said, is often to fall back on rules engines that can be read and defended in plain language. Sen argued that this can reduce detection quality, while still being treated inside institutions as a safer compliance option.

Architecture as the audit trail

Sen said the industry should instead focus on model architectures in which the decision is built from identifiable components. In that design, the system’s conclusion would be tied to specific accounts, transactions and patterns, with each contributing element assigned a visible role in the outcome.

He said the mechanism would make the audit trail part of the model output, rather than a later translation layer. For example, when a cluster of accounts is flagged, the model should be able to identify the particular transactions that drove that decision from a wider pool of available data, according to Sen.

Sen argued that this approach can be implemented by making model attention inspectable and sparse, with a direct link between individual transactions and final verdicts. He contrasted that with systems in which explanatory signals are spread across large numbers of parameters and become difficult for compliance staff to interpret.

On performance, Sen said models constrained to rely on traceable evidence can match or outperform black-box baselines in detection quality. The Finextra opinion did not include underlying benchmark data or name specific client deployments.

The argument comes as banks and fintech providers continue to assess how to use artificial intelligence in anti-financial crime controls while satisfying internal model risk standards and regulatory expectations. Sen’s position is that explainability should be treated as a design requirement before any transaction is scored, rather than as a reporting function added after deployment.

This story draws on original reporting from Finextra Research.