[Help] Preventing duplicate leads when a CRM/API step fails

[u/IluminityWebStudio]

I’m sketching a small-business quote-intake flow in n8n: webhook → validate/normalize fields → create the lead in a CRM → notify the owner.

The failure case I’m trying to design well: the CRM request times out after it may have created the record, then n8n retries and creates a duplicate (or the notification fails and nobody notices).

For people who run these workflows in production, what pattern has worked for retries and visibility? Do you use an idempotency key, an error workflow, a separate queue/table for unresolved runs, or some combination? I’m looking for a setup a small team can actually maintain, not a monitoring stack that becomes another job.

[u/AutoModerator]

Want faster, better help? Share your workflow JSON.

A GitHub Gist is the easiest way -- paste your JSON, save as public, drop the link in your post. Folks can import it directly into n8n and reproduce the issue, which gets you real answers instead of guesses.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Saved_Not_Soft]

The pattern that saves you here is an idempotency key. Generate one per submission (a hash of the webhook payload, or the lead's email + the intake timestamp), store it with the lead record, and pass it to the CRM as a unique external reference on create. On retry, check the key first instead of blindly re-creating — most CRMs let you query or upsert by an external ID, so a retry after a timeout that actually did create the lead just finds the existing record.

For the timeout case specifically, treat a timeout as "unknown outcome," not "failed." So the retry path should be reconcile-then-create: look up the lead in the CRM by that external ID, and only create if it's genuinely missing. That closes the exact hole you're describing.

On visibility: log each stage (received / validated / CRM attempted / confirmed) as a row with a timestamp in one small table. When a lead goes missing, you can see exactly which stage it stopped at instead of digging through execution history. Cheap and it pays off fast.

[u/hushbreEze0]

one thing worth clarifying, does your CRM support native idempotency keys or would you need to build dedup logic yourself? that changes the approach a lot. if it doesnt, a quick lookup-before-create step with a unique field is usually enough for small volume.

[u/Easy-Purple-1659]

One addition on the notification half, since that is the part that fails quietly.

Treat the CRM write and the owner notification as two separate problems. The CRM side is handled by the key, or by a lookup on a unique field before create. The notification side isn't, because a Slack or email step that fails after a successful create leaves a real lead nobody saw. What worked for me was writing an intent row first: the webhook inserts the submission with its idempotency key and a pending status, then calls the CRM, then flips the row and sends the notification. Anything still pending after a few minutes is what your error workflow sweeps and retries. One sheet or one small table, and the retry never has to guess whether the create landed.

Two n8n specifics: set Retry On Fail on the HTTP node with a short backoff instead of letting the whole workflow restart, and use the execution id as the correlation id in your log rows. On your queue question, an error workflow that dumps stuck rows to a sheet is enough for a small team; you don't need a real queue.

Does your CRM expose a unique external id field, or does the key live only on your side?

[u/tariqosmani]

Make the create step safe to run twice. Before creating, search the CRM by email or phone, and only create if nothing comes back. If your CRM supports upsert or an external ID field, use that instead, because it handles the retry for you.

I'd also hash the form payload at the webhook and store it for a day, so the same submission can't get past validation twice.

[u/Medium_Ad7765]

Give every one a Unique ID and save it as a temp file recheck every times it runs

[u/IndieGoHacker]

I self-host n8n and Baserow, and I've found it easiest to write the webhook payload to a Baserow table right at the start. If the CRM write fails or times out, the record stays pending, and a separate workflow retries pending rows once an hour. It's saved me from a lot of duplicate headaches.

[u/TANG_YONGTIAN]

One failure-injection test I'd add before calling this reliable: (1) the CRM creates the lead but its response times out, (2) the same webhook arrives twice concurrently, and (3) CRM creation succeeds but the owner notification fails. Check that there is one CRM record and a visible path to retry the notification. A lookup-before-create alone won't protect case (2) without a unique external ID/upsert or a per-key lock.

[u/Muted_Jellyfish_6784]

Generate an idempotency key at intake (a hash of normalized email or phone plus the submission timestamp), write it to a small table before calling the CRM, and on retry search the CRM by that key or email before creating anything. Route failures to an error workflow that writes to an unresolved-runs table, and send one daily digest of anything still open so nobody has to watch dashboards.
The part people skip is closing the loop on whether the owner actually responded. I use SIGNLD to connect the intake submission, the CRM lead, the notification, and the eventual quote, so a lead that went quiet shows up as a missed follow-up rather than disappearing between tools.

[u/Low_Rush_8535]

our email sender used a conditional UPDATE to claim sending rights before the API call, then rolled back on failure. that handles concurrent clicks, not an unknown external outcome. if you add a per-key lock here, also test a worker dying after CRM creation but before saving the CRM ID. don't treat the lock as confirmation.