Prevent duplicate records with a search-first create flow

Insights

Duplicates can start at the product's own Add button. Search existing records first, attach to a match and keep create new as a deliberate fallback. To prevent duplicate records, fix the order of the create flow before you fix the data.

To prevent duplicate records, fix the order of the create flow before you fix the data. When "Add" opens a blank form, the product asks for a new record before asking whether one already exists. Reverse that: start a search from what the person already has in hand, show existing records in a form they can recognise, let them attach to a match, and keep "Create new" as a deliberate second choice. Then back the flow with a unique key where the data has one, and a merge path for whatever still gets through.

Matching rules, save-time warnings and clean-up tools all have a place, but each acts after the record is typed. What prevents the duplicate is when the check runs, what it shows and what it lets the person do next.

The search-first create flow, part by part

The pattern also goes by "find or create" or "search before create". It has four parts, and each holds a design decision.

Part	The decision	What good looks like
Where the search starts	What the user already has in hand	The search is pre-filled from context: the address on the listing, the sender of the email, the company domain. Nobody retypes what the screen already knows.
What a match shows	What lets a person recognise the same thing	The fields that tell one record from its neighbours (status, owner, location, last activity) and why it matched.
Attach	What changes when the user picks a match	The thing in hand is linked to the existing record, and the product says so. No fields are overwritten, and the link can be undone.
Create new	How the fallback stays available	A clearly labelled option ("None of these: create a new deal") that is always present, comes second, and carries the search terms into the new form.

Two details decide whether people accept the search step. It has to be faster than creating, or people route around it. And "Create new" must never be hidden or disabled: a genuinely new record should cost one extra, informed click.

What Tcules changed in Dealpath's pipeline flow

Dealpath is software for institutional real-estate teams. When someone chose Add to pipeline on a listing, the product went straight to Create deal, without asking whether the listing already belonged to a deal the team was tracking. We moved that question to the front. In the Dealpath redesign, the flow opens on Attach to a deal, where the user searches existing deals and picks the match; Create new deal stays available for listings with no match. The listing's other actions, marking it as a comparable or passing on it, were largely left as they were.

This was a design decision. Dealpath's own team carried out the build, and our record shows neither which version shipped nor any change in duplicate rates, so we claim no reduction. What it does show is how one default, "this listing is a new deal", made the duplicate the easier path.

The create flow is where a product's model of its records meets the person adding one. Settling which objects exist, how they relate, who can act and what each state means is what our information architecture and workflow design work covers.

Search first, warn or block: what each check costs

Searching first is not the only check, and it has a cost of its own. Each option puts the cost on someone different, at a different moment.

Check	When it runs	What it costs	Use it when
Search first, then attach or create	Before any form opens	A search step on every add, including genuinely new records	People arrive holding something that may already exist (a listing, an email, a lead), and a wrong attach can be undone
"Possible duplicates" shown while typing	As key fields are filled in	A little attention while typing	New records are common, and the matching fields come early in the form
Warning at save	When the user saves	The record is already typed, so "continue" is the easy answer	As a safety net behind one of the above, never alone
Blocking at save	When the user saves	Legitimate work is stopped by any false match	Only on an exact, trustworthy key: an email address, a domain, an order number
Merge later	Later, possibly by someone else	Conflicting fields, related records and history to reconcile	Always, as the backstop for whatever the rest misses

The built-in checks in the large CRM platforms sit in the save-time rows. In Microsoft Dataverse, the data platform under Dynamics 365, a "Duplicate records found" dialogue box appears when the user saves a record, offering "Ignore and save" or, for accounts, contacts and leads, a merge. A Salesforce duplicate rule set to allow with an alert prompts the user to continue or cancel. Both are useful nets, but by the time either appears the whole record is typed, so carrying on is the cheapest click on the screen (our inference, not a measured finding).

Blocking needs a key the data can be trusted on. HubSpot deduplicates contacts by email address and companies by domain. Records that represent work, such as deals and tickets, have nothing equivalent: two deals on the same building can both be legitimate. Where there is no key, the database cannot refuse the duplicate, so only the flow can ask the question.

A workable rule: block only on exact keys; warn, never block, on fuzzy matches such as names and addresses; search first wherever people arrive holding something that probably exists already; and keep merge for the rest. A product may need several rows, at different entry points.

A wrong attach is worse than a duplicate

Searching first moves the risk rather than removing it. A duplicate is two records for one thing. A wrong attach is one record for two things, and it is harder to undo, because work, notes and history pile onto the wrong record.

Record linkage has drawn this line since Fellegi and Sunter (Journal of the American Statistical Association, 1969), who class each pair of records as a link, a non-link or a possible link where the evidence supports neither; in practice, possible links are left for a person to review. Healthcare states the trade-off plainly. In a 2014 report for the US Office of the National Coordinator for Health IT, most of the health systems interviewed said their matching must be set to produce duplicates rather than overlays (wrong matches) as a patient safety policy, because the wrong care could follow from an incorrect match.

Most SaaS products are not hospitals, but the consequences carry over:

Never attach automatically on a fuzzy match. The search proposes; the person confirms.
Show enough to tell similar records apart. Two "Elm Street" deals in different cities must not look identical in the results.
Make attach reversible. A detach that restores the previous state turns a wrong attach into a minor error.
When the person is unsure, let them create. A duplicate can be merged later; a wrong attach is harder to untangle.

Edge cases the flow will not catch

A search-first flow governs one entry point, for one person, at one moment. Three cases fall outside it.

Two people creating the same record at once. Both search, both find nothing, both create. No interface fixes this race. Where a natural key exists, the database has to enforce it: in PostgreSQL, INSERT with ON CONFLICT DO UPDATE guarantees an atomic insert-or-update outcome even under high concurrency. Where there is no key, detection after the fact is the honest answer: a Salesforce duplicate rule, for example, can report the duplicate instead of interrupting anyone.

A match the user is not allowed to see. In Salesforce, a duplicate rule that enforces sharing rules lets the insert go ahead with no message when the user has no access to the existing record. A product has to choose: create silently and merge later, or say that a matching record exists and route the person to its owner. The second prevents the duplicate but reveals that the record exists, which is fine for a company name and may not be for a patient or a personnel file.

Imports and integrations that bypass the screen. In Microsoft Dataverse, duplicate detection is suppressed by default for records created or updated through the Web API or the SDK for .NET, and HubSpot does not deduplicate companies created through its API by domain. The same rule has to run on the server: an integration should search, then create, as Zapier's "Find or Create" step does by pairing a search with a create.

How to tell whether it worked

Set these up before the change ships, so there is a baseline to compare against.

Measure	How to read it
Duplicates created per records created	The core rate. Fix the numerator and denominator before measuring.
Attach versus create at the entry point	How often the search finds what the person needed. A very low attach rate may mean the search is failing, not that every record is new.
"Create new" chosen after a match was shown	Overrides. Sample them: some are genuinely new records, some are matches the results did not make recognisable.
Detaches and corrections after an attach	The wrong-attach rate. If it rises, the results need more distinguishing detail.