GLEIF applies AI to LEI duplicate detection

GLEIF is using AI to strengthen LEI duplicate detection AI controls in the Global Legal Entity Identifier System, according to Zornitsa Manolova, head of data quality management and data science at the organisation. Manolova said potential duplicate records currently account for less than 0.2% of all records in the system, a level GLEIF attributes to preventive checks, issuer review and established remediation procedures.

A Legal Entity Identifier, or LEI, is a unique code used to identify a legal entity in financial and business records. Under the Global LEI System’s core rules, each LEI can represent only one entity, and each entity can hold only one LEI.

The data quality issue is therefore operational as well as technical. If duplicate records enter the Global LEI Index, users of the system could face lower confidence when matching companies, funds, branches or other legal entities across jurisdictions and data sets. GLEIF says avoiding that outcome is part of its proactive data quality programme.

How does AI improve LEI duplicate detection?

GLEIF is applying AI to its Check for Duplicates facility, a tool used by LEI issuers before a new record is published. The facility assesses whether a proposed LEI and its reference data may already exist in the Global LEI Index.

During issuance, GLEIF said the tool compares new submissions with the full LEI Index and with records that other issuers have not yet published. That design is intended to catch cases in which the same legal entity approaches more than one LEI issuer, allowing possible duplicates to be identified and addressed before they appear publicly.

Manolova said the enhanced system moves beyond an earlier approach based mainly on fuzzy name matching. The AI-supported process is designed to compare related records earlier, handle larger data volumes and support coordinated resolution before publication.

What the new workflow checks

GLEIF describes the upgraded duplicate detection process as a three-stage workflow: pre-processing, filtering and scoring.

• In pre-processing, a submitted record is cleaned and standardised. GLEIF said this includes removing punctuation, normalising spaces, parsing the record for relevant reference data and creating vector embeddings for later comparison.

• In filtering, an AI-enhanced backend searches for potential matches. The process checks the LEI code, then compares registration authorities and registration identifiers. Selected reference data is also converted into vector embeddings and compared with existing records in the Global LEI Index.

• In scoring, possible matches are evaluated further to reduce false positives. GLEIF said the process considers data points including legal name, legal form, address, jurisdiction and entity creation date, with additional treatment for categories such as funds and branches.

Vector embeddings are mathematical representations that allow text and other record elements to be compared by similarity rather than by exact spelling alone. In this use case, they help the system compare proposed records against large volumes of existing LEI reference data.

Governance remains part of the control framework

GLEIF said the AI-supported approach is intended to make duplicate detection faster, more scalable and more consistent across issuers and locations. The organisation also said checking multiple data elements strengthens data quality controls compared with relying chiefly on names.

Manolova emphasised that AI depends on complete and trustworthy data and clear governance. GLEIF said its approach combines AI recommendations with validation processes, continuous monitoring, model refinement, transparent decision-making and human oversight.

The changes extend an existing control framework rather than replacing it. GLEIF said issuer commitments, review procedures and operational processes remain part of how the Global LEI System seeks to preserve confidence in entity identification.

This story draws on original reporting from Finextra Research.