Duplicate Invoice Detection: Why Exact-Match Checks Miss Most Duplicate Payments
Every accounts payable team has a duplicate invoice check, and almost every one of them catches only the duplicates nobody needed help with: the same vendor, the same invoice number, typed the same way twice. The duplicates that actually get paid look slightly different from the original. A prefix was dropped, the date moved, the supplier sits under a second vendor record. This post looks at where duplicate invoices come from, why exact-match checks miss them, and how a near-match check on extracted fields stops the payment before it leaves the bank.
In brief
- Most duplicate invoices are not identical copies: the invoice number, date, vendor record or amount differs slightly, so an exact-match check on invoice number and vendor lets them through.
- Duplicates come from re-sent invoices, statement-driven resubmissions, the same invoice arriving by email and by portal, and the same supplier existing as two vendor records.
- Normalizing invoice numbers and comparing on several fields with tolerances catches near-duplicates that an ERP uniqueness check cannot see.
- A duplicate check should run before approval, on extracted fields, and hold the payment with the matching prior invoice attached as evidence.
Why duplicates rarely look like duplicates
A duplicate payment is almost never fraud and almost never a careless clerk. It is a real invoice that entered the process twice through two different doors, and each door recorded it a little differently. The supplier's billing system prints INV-0048213; the AP clerk keying from a statement types 48213. The original was dated when it was issued; the reminder copy carries the date it was re-sent. The first copy was posted to the vendor record created by purchasing; the second landed on an older record for the same company with a slightly different name.
The typical ERP control blocks a new invoice only when vendor ID and invoice number match an existing one exactly. Against the cases above, that control sees three new invoices. It is doing exactly what it was configured to do, and that is the problem: uniqueness on two raw fields is a data integrity rule, not a duplicate payment control.
2
fields most ERP checks compare: vendor ID and invoice number
5+
fields a near-match check should weigh together
0
duplicates you want discovered after the payment run
Where duplicate invoices come from
Reminders and re-sent copies
A supplier who has not been paid sends the invoice again, sometimes stamped "copy" or "second notice", sometimes with a new issue date, sometimes as a fresh PDF regenerated from their system. If the first copy is still waiting on approval, the second one starts its own approval path, and both can reach the payment run.
The same invoice through two channels
The invoice is emailed to the AP inbox and also uploaded to a supplier portal, or mailed on paper and emailed by the sales rep. Each channel feeds intake separately, and nothing connects the two documents unless something compares their content.
Invoices keyed from a statement
A month-end vendor statement reconciliation finds an invoice on the statement that is missing from the ledger. Someone keys it from the statement line, often with a shortened invoice number. If the original was in fact received and simply waiting for approval, there are now two.
Duplicate vendor records
The same supplier exists twice in the vendor master, created by different teams, after an acquisition or a change of remit-to address. A check keyed on vendor ID can never compare invoices across the two records.
Credits and re-bills
The supplier issues a credit note for a disputed invoice and re-bills a corrected one. If the credit is not applied, or the corrected invoice reuses the original number with a suffix, the old and new amounts can both be paid.
Re-sent as reminder
Email and portal copy
Keyed from statement
Duplicate vendor record
Credit then re-bill
Identical resubmission
What a near-match check compares
A useful duplicate check scores an incoming invoice against recent invoices on several fields at once, each normalized first, and flags the pair when enough of them agree. No single field is decisive; the combination is.
- Normalized invoice number. Strip prefixes, punctuation, spaces and leading zeros, and compare what remains. Also look for common keying errors, such as transposed digits or a dropped character.
- Vendor identity, not vendor ID. Compare tax ID, remit-to bank account and address so two records for the same supplier are treated as one.
- Amounts within a tolerance. Total, subtotal and tax, allowing for small differences from rounding, currency conversion or a re-bill that added a late fee.
- Dates within a window. Invoice date and, for recurring charges, the service period. A reminder copy may be dated weeks after the original.
- References and lines. The PO number, delivery or shipment reference, and line items when they are present. Two invoices billing the same PO lines for the same quantities deserve a look even if the numbers differ.
Recurring invoices need the service period
A workflow that stops the payment, not the auditor
Duplicate payment recovery after the fact is slow: ask the supplier for a refund or a credit, chase it, and hope they are still trading. The control belongs before approval, on every invoice, as part of intake.
1. Extract the fields that identify an invoice
Vendor name, tax ID, remit-to details, invoice number exactly as printed, invoice date, service period, PO number, currency, subtotal, tax, total and line items. These have to come from the document itself, not from whatever a person typed, because the typed version is precisely where the variation creeps in. Defining them once with types and required flags is covered in the post on designing extraction schemas.
2. Normalize, then compare against open and recently paid invoices
Compare each incoming invoice against everything not yet paid and everything paid in a lookback window, such as the last twelve to eighteen months. The open invoices matter most: a duplicate that is caught while the original is still waiting for approval costs nothing.
3. Hold with evidence, not with a flag
A suspected duplicate should go to a person with both documents side by side and the list of fields that matched, so the decision takes seconds. A bare "possible duplicate" flag with no context trains reviewers to clear it.
{ "incoming_invoice": { "vendor_name": "Example Packaging Co.", "vendor_tax_id": "12-3456789", "remit_to_account_last4": "4421", "invoice_number": "INV-0048213", "invoice_number_normalized": "48213", "invoice_date": "2026-09-30", "po_number": "PO-55102", "currency": "USD", "subtotal": 4180.00, "tax": 344.85, "total": 4524.85 }, "candidate_match": { "ap_document": "AP-901377", "vendor_id": "V-2210", "vendor_name": "Example Packaging Company LLC", "invoice_number": "48213", "invoice_date": "2026-09-28", "total": 4524.85, "status": "approved_scheduled_2026-10-09" }, "signals": [ { "field": "invoice_number_normalized", "result": "exact" }, { "field": "vendor", "result": "same_tax_id_different_vendor_id" }, { "field": "total", "result": "exact" }, { "field": "invoice_date", "result": "within_window", "days_apart": 2 }, { "field": "po_number", "result": "exact" } ], "score": 0.97, "decision": "hold_suspected_duplicate", "reason": "Same supplier tax ID, invoice number and total as AP-901377, already scheduled for payment"}In this example the incoming invoice and the earlier one are filed under different vendor records with different names, and the invoice numbers are printed differently. An exact-match check on vendor ID and invoice number passes it. Normalizing the number and comparing tax ID, total, date and PO shows it is the same invoice, already approved and scheduled, so the new copy is held before the payment run.
Exact matchNear match
Same number, same vendor
Number format differs
Different vendor record
Amount or date differs
Tuning the check without drowning reviewers
A near-match check that flags too much is ignored within a month. Start by scoring matches and reviewing the top band only, then widen it as you learn which signals matter for your suppliers. Track every flag a person resolves as duplicate or not duplicate, and use that record to adjust weights and tolerances. Look at the false positives by vendor: a handful of suppliers with recurring invoices or reused numbering usually accounts for most of them, and a supplier-specific rule fixes it.
Measure the input as well as the rule. A misread invoice number or total produces both false alarms and missed duplicates, so measure field-level accuracy on invoice number, date and amounts against a labeled set of your own invoices. When reminders arrive bundled with statements and remittance advice in one PDF, split and classify them first so a statement line is never read as a new invoice. And keep the duplicate check next to the three-way match: a PO line that is already fully invoiced is one of the strongest duplicate signals you have.
Finally, fix the causes upstream. Merge duplicate vendor records and block new ones that share a tax ID or bank account. Route every intake channel into one queue. Stop keying invoices from statements; request the original document instead. Each of these removes duplicates the check would otherwise have to catch.
Before you automate: a short checklist
- Run the duplicate check before approval, not only before the payment run.
- Normalize invoice numbers: remove prefixes, punctuation, spaces and leading zeros.
- Identify vendors by tax ID, bank account and address, not only vendor ID.
- Compare amounts and dates with tolerances, not exact equality.
- Include the service period for recurring invoices.
- Compare against open invoices and a twelve to eighteen month paid history.
- Show both documents and the matched fields to the reviewer.
- Record every reviewer decision and tune weights from it.
Where extraction fits
Duplicate detection is a comparison problem, and comparisons are only as good as the fields being compared. When invoice data is keyed by hand from many channels, the variation that hides duplicates is built in. Parser extracts each invoice into structured JSON against the schema you define, with the invoice number exactly as printed, vendor identifiers, dates, amounts and line items, so the near-match check runs on what the supplier actually sent and a second copy is caught before it is paid.