Deduplicate sales leads in a CSV

Step by step: dedupe your lead list

1. 1. Export your leads to CSV. Pull the leads out of your CRM, ad platform, or webform tool as one CSV file.
2. 2. Open the duplicate remover. Drop the CSV into the duplicate remover — the file loads into your browser without being uploaded.
3. 3. Match on the email column. Ignore case and spacing so Jane@Acme.com and jane@acme.com count as the same lead.
4. 4. Keep the latest copy and download. Keep the last copy of each duplicate group to retain the freshest record, preview the result, and download the clean list.

What actually counts as a duplicate

Most lead lists have three layers of duplicate, and only the first one is obvious. Exact repeats — the identical row imported twice — fall out of any dedupe. Then come the case and whitespace variants: Jane@Acme.com and jane@acme.com, or the same address carrying a trailing space from a bad export. They read as different strings and identical mailboxes, which is why matching with case and spacing ignored catches far more than a raw match does. If the list is still open in a workbook, the exact-repeat layer is also covered by how to remove duplicates in Excel, including the COUNTIF check that shows the repeats before anything is deleted.

The third layer is provider-specific and catches people out. Google documents that dots in a consumer Gmail address are ignored, so jane.doe@gmail.com and janedoe@gmail.com reach one inbox — but it also warns that on a work or school domain, dots do change the address. So normalize dots for @gmail.com rows only, and leave every other domain alone. Getting that backwards merges two real people into one record, which is worse than the duplicate you were trying to remove.

If the address column is messy enough that matching is unreliable, clean it before you dedupe: lowercase and trim the email column first, then match. For the wider cost of leaving these rows in place, see what duplicate records do to a CRM, and work through the pre-import checklist before the list goes anywhere near your pipeline.

What your CRM will and will not dedupe for you

Both major CRMs already dedupe on email at import, so it is worth knowing exactly where their coverage stops. HubSpot documents that a contact with an email that already exists updates the existing record rather than creating a new one, and only mapped properties are updated. Salesforce goes a step further on names: its Standard Contact Matching Rule matches email exactly, while first and last names are matched fuzzily using Jaro-Winkler and Metaphone algorithms.

Two things follow. The first is that exact email repeats are genuinely handled for you — importing a 1,000-row file with 260 exact repeats will not create 260 extra contacts. The second is that the CRM’s rules, not yours, then decide what the surviving record looks like: which copy overwrites which field, which blank values stay blank, and which automations fire on the update. Deduping the CSV first is what moves that decision back to you — you pick whether the first or the last copy wins while the data is still a file you can re-download.

What neither CRM catches is the same person arriving under two different addresses — a work address on one list, a personal one on another. No matching rule can merge those safely, and a fuzzy name-only match will happily collapse two real people who share a common surname. Treat that layer as a manual review, not an automated pass: sort the export by last name, look at the collisions, and decide by hand. Standardizing the phone column first gives you a second field to judge them on — one shape for every number, which is what the E.164 standard specifies and what makes two spellings of one line finally collide, and the wider set of in-browser cleaning tools covers the rest of the pre-import pass.

What duplicate leads cost you at the mailbox

The pipeline damage is the visible half. The quieter half is that a duplicated list sends the same person the same message twice in one campaign, and the mailbox providers grade you on how often recipients react badly to what lands. Google’s sender guidelines set the bar tightly: bulk senders should keep spam rates in Postmaster Tools below 0.10% and avoid ever reaching 0.30%. Yahoo publishes the same ceiling, telling senders to keep their spam rate below 0.3%.

Those numbers are worth converting into rows, because the percentages read as more forgiving than they are. At the 0.30% ceiling, 3 complaints per 1,000 messages is the edge; at the 0.10% target it is 1. A send to 20,000 addresses has roughly twenty complaints of headroom before the stricter line is crossed. You do not need many people receiving the same pitch twice in an afternoon to spend that.

This is why deduping belongs before the import rather than after the first send. The reputation cost is not charged to the duplicate row; it is charged to the sending domain, and it outlasts the campaign that caused it. Matching on a normalised email column — lowercased and trimmed before the comparison, so Jane@Acme.com and jane@acme.com collapse instead of surviving as two sends — is what makes the dedupe catch the rows that would otherwise become complaints. The same logic applies to any list headed for a mail merge, which the pre-import cleaning sequence walks through end to end.

When matching on email gets it wrong

Email is the best key a lead list has, and it still fails in two directions. It over-matches when several people share one inbox, and it under-matches when one person owns several spellings of the same address. Both failures are quiet: the row count drops, the file looks tidy, and the damage only shows up when a rep calls a contact who was folded into someone else's record.

Role addresses collapse real people

A form fill captured info@acme.com, finance was entered as accounts@acme.com, and a rep typed sales@acme.com after a call. Those are three humans behind shared inboxes, and an exact-match dedupe treats each address as one lead — correct for the string, wrong for the account. Sort the export by the email column before you dedupe and read the addresses that start with a department word. Dedupe the personal addresses on their own and give the role rows a composite key, such as company plus last name.

That key needs a last-name column, and many CRM exports carry only one full-name field. Run the column through the first and last name splitter as an earlier step in the same workspace, then tick company and last_name as the columns the duplicate remover matches on.

Tagged addresses survive as separate leads

The opposite problem is subaddressing — the detail after a plus sign that RFC 5233 defines as extra meaning attached to the local part of an address. Someone who signed up once as jane+webinar@acme.com and once as jane@acme.com reads mail in one mailbox but appears in your file as two leads, and no case or spacing option will merge them. Copy the email column, remove the plus and everything after it in the copy, and dedupe on that column instead. Keep whichever original address you intend to send to.

Both passes are easier on a column that has already been trimmed and checked for malformed addresses, because a trailing space or a stray capital is enough to split one lead into two before any of this starts. Run that first, then the duplicate remover on the result.

Frequently asked questions

How do duplicate leads inflate my pipeline?

Every duplicate row is counted as a separate opportunity, so your lead volume, conversion rate, and forecast all look better than reality. Reps also waste time emailing the same person twice, which damages trust and hurts deliverability. Removing duplicates before reporting gives you numbers you can actually plan against.

Should I deduplicate leads by email or by name?

Match on email. It is the most reliable unique identifier for a person — names are frequently misspelled, abbreviated, or shared. If a lead has no email, fall back to matching on company plus name. Sigmera lets you ignore case and spacing so near-identical emails still match.

Will my lead list be uploaded anywhere?

No. The duplicate remover runs entirely inside your web browser using client-side JavaScript. Your lead list is never sent to a server and never leaves your device, so handling prospect PII this way is GDPR-safe by design.

Which copy of a duplicate should I keep?

Usually the most recent one, because it has the freshest activity and contact details. Sigmera lets you keep either the first or the last copy of each duplicate group, so if your export is sorted by date you can keep the newest record in one click.

Are jane.doe@gmail.com and janedoe@gmail.com the same lead?

On a consumer @gmail.com address, yes — Google ignores dots, so mail sent to either spelling reaches the same inbox. A case-insensitive match alone will not catch it, because the two strings genuinely differ. The caveat matters: Google states that on a work or school domain, dots do change the address, so the same trick is not safe on Google Workspace or any other provider. Strip dots only for the @gmail.com rows, and treat everything else as a literal string.

Does HubSpot deduplicate leads automatically on import?

For exact email matches, yes. HubSpot documents that if a contact already exists with the same email address, the imported information updates that existing contact rather than creating a new one, and only mapped properties are updated. What it does not do is decide the way you would: the CRM's rules choose which values survive the update and which automations fire. Deduping the CSV first keeps that choice with you, because you pick whether the first or last copy wins while the data is still a file.

Why dedupe before the import if Salesforce has duplicate rules?

Because the rules cover a narrower case than people assume. Salesforce's Standard Contact Matching Rule matches email exactly and only fuzzy-matches first and last names, using Jaro-Winkler and Metaphone. That catches repeat rows well, but a merge inside a live CRM means reconciling ownership, activity history, and automations that have already run. Deleting a row from a CSV leaves no trace; merging a live record does.

Do duplicate leads hurt email deliverability?

Indirectly, and the margin is thinner than most people expect. A duplicated list sends the same person the same message more than once in a campaign, which is a reliable way to earn a spam complaint. Google's sender guidelines tell bulk senders to keep spam rates in Postmaster Tools below 0.10% and never to reach 0.30%; Yahoo publishes the same 0.3% ceiling. At 0.30% that is 3 complaints per 1,000 messages, so a 20,000-address send has around twenty complaints of headroom. The reputation hit lands on your sending domain, not on the duplicate row, and it outlasts the campaign.

How do I dedupe leads that have no email address?

Fall back to a composite key. Match on company plus last name, or on a normalized phone number, and review the matches by hand before deleting anything — those fields collide far more often than email does. Standardizing the fallback column first makes the match usable: put every phone number into one shape before you dedupe, and lowercase and trim the company names so trailing spaces do not split one account into two.

Will deduplicating on email merge two different people at the same company?

It can, whenever the row carries a role address rather than a personal one. A form fill that captured info@acme.com, an invoice contact at accounts@acme.com, and a rep who typed sales@acme.com into the CRM are three separate humans behind one shared inbox, and matching on email collapses them into a single row. Sort the export by email before the dedupe and look at the addresses that begin with a department word. Give those rows a composite key — company plus last name — or hold them aside and dedupe the personal addresses on their own.

Why do jane@acme.com and jane+event@acme.com survive as two leads?

Because they are two different strings. The part after the plus sign is a subaddress, defined in RFC 5233 as detail information added to the local part of an address, and most mail providers route it to the same mailbox while treating it as a distinct address. A lead who filled in one form with a tagged address and another without it therefore passes through an exact-match dedupe untouched. Strip the plus and everything after it in a copy of the email column, run the dedupe on that, and keep whichever original address you want to mail.