Master Salesforce Duplicate Management & Data Quality

💬 In plain words: Keeping duplicate records out of Salesforce comes down to two partners working together: Matching Rules (the detective that spots when two records look like the same person or company) and Duplicate Rules (the security guard that decides whether to block the save, warn the user, or quietly log the twin for later). Still, clicking checkboxes in Setup won't magically fix messy data. Real data quality takes upstream discipline—cleaning files before bulk uploads, locking down field formats, and agreeing on which system owns the truth.

📌 Quick Everyday Example:
Matching Rule (The Detective): Uses a fuzzy match on the Account Name plus an exact match on the Website domain.
Duplicate Rule (The Bouncer): Set to BLOCK when a sales rep manually types a duplicate Account on their screen, forcing them to use the existing record. However, for the nightly ERP integration user, set a separate rule to ALLOW and REPORT. Why? Because you never want an automated 50,000-row midnight invoice sync to crash just because "Acme Corp" looks a lot like "ACME Corporation."

🔑 Key Points Summary

• Matching Rule vs. Duplicate Rule: A Matching Rule defines the math and logic for "sameness" (Exact vs. Fuzzy). A Duplicate Rule decides the penalty or action (Allow, Alert, or Block).
• Order of Execution Timing: Duplicate rules fire Step 6 in the Salesforce Save Order of Execution—right after Validation Rules and before triggers, but before the record is committed to the database and before after triggers or record-triggered Flows run.
• Platform Guardrails & Limits: Salesforce lets you activate up to 5 Matching Rules per object and up to 5 active Duplicate Rules per object (with up to 3 matching rules inside one duplicate rule). You can also match across objects, such as checking new Leads against existing Contacts.
• Large Data Volume (LDV) Bottlenecks: Native matching keys slows down or times out when you pump millions of records into an org. For heavy multi-system deduplication, use upstream ETL staging, an enterprise Master Data Management (MDM) tool, or Salesforce Data Cloud (Data 360) Identity Resolution.
• AI & Agentforce Dependency: Bad data poisons AI. If Agentforce finds three different records for the same customer—one showing a closed-won deal and another showing an angry unpaid billing dispute—the AI agent will hallucinate or give the wrong answer.

🧠 Easy Memory Trick: "Match Defines, Rule Decides"
The Matching Rule defines what "the same" looks like. The Duplicate Rule decides what to do about it (Allow, Alert, or Block).

⚙️ The Core Concept: How Salesforce Catches Twins

Salesforce splits its built-in deduplication engine into two distinct building blocks so admins can reuse matching logic across different teams and entry channels:

• 1. Matching Rules (The Formula): These tell Salesforce which fields to inspect and how strict to be. You can choose Exact matching (every letter must match) or Fuzzy matching (using algorithms like Jaro-Winkler, Edit Distance, or Soundex to realize "Robert" and "Bob" or "Intl Business Machines" and "IBM" are similar). Behind the scenes, Salesforce builds a normalized MatchKey index table so it can scan existing records in milliseconds.
• 2. Duplicate Rules (The Enforcer): These watch the front door during Insert and Update operations. When a Matching Rule raises its hand, the Duplicate Rule checks who is logged in (using Profiles, Roles, or custom criteria) and executes your chosen rule: Block the user cold, Alert them with a pop-up warning while letting them click "Save Anyway," or silently allow the save and write a row to the DuplicateRecordItem table for a stewardship report.

For cleanup after the fact, Performance and Unlimited Edition orgs also offer Duplicate Jobs, which scan your entire database on a schedule to find older twins that slipped in before your rules were turned on.

// Where Duplicate Rules Sit in the Salesforce Save Order of Execution
1. Load original record from database (or initialize new record)
2. System Validation Checks (layout required fields, field formats, max length)
3. Record-Triggered Flows (Before-Save / Fast Field Updates)
4. All "before" Apex Triggers execute
5. Custom Validation Rules run
6. ➡️ DUPLICATE RULES EXECUTE HERE! (If set to Block, execution stops immediately)
7. Record is saved to the database (Id is generated, but not yet committed)
8. All "after" Apex Triggers & After-Save Record-Triggered Flows execute
9. Commit to database

🎬 Real-Life Scenario: The Nightly ERP Sync vs. The Morning Sales Rep

The Problem: GlobalTech imports 100,000 customer updates every night at 2:00 AM from SAP and NetSuite. During the day, 300 sales reps manually type new prospects into Lightning Experience. When an admin turned on a strict "Block Duplicates" rule for everyone, the 2:00 AM SAP batch crashed with thousands of DUPLICATES_DETECTED errors.

The Architect Fix:

• Rule 1 (UI Users): Condition set to Current User: Profile NOT EQUAL TO Integration User. Action is set to Block on create and edit. Sales reps are forced to search and use existing Accounts.
• Rule 2 (Integration Batch): Condition set to Current User: Profile EQUALS Integration User. The batch uses an Upsert call keyed on an indexed SAP_External_ID__c field. If a brand-new record triggers a fuzzy name match, the rule is set to Allow & Report—keeping the nightly pipeline green while logging edge cases into a Data Steward dashboard for morning review.

☁️ Beyond Native Rules: MDM vs. Salesforce Data Cloud (Data 360)

Native Salesforce duplicate rules only protect a single Salesforce org. When your company has customer records scattered across Salesforce, SAP, Snowflake, Shopify, and legacy billing databases, native rules hit a wall. You have two modern architectural paths to build a unified Golden Record:

• Path A: Traditional Master Data Management (MDM): Platforms like Informatica, Reltio, or Riversand sit upstream of Salesforce. They ingest records from all systems, clean and standardize addresses, apply field-by-field survivorship rules, merge the duplicates physically, and push one clean "Golden Record" down into Salesforce with an MDM_Golden_ID__c.
• Path B: Salesforce Data Cloud (Data 360) Identity Resolution: Instead of physically overwriting your core CRM tables right away, Data Cloud maps source records into the Customer 360 Data Model (such as the Individual and Account Data Model Objects). You configure Identity Resolution Rulesets (combining deterministic exact email/phone matches with probabilistic fuzzy name/address matching) and Reconciliation Rules (such as Most Frequent, Last Updated, or Source Priority) to generate a virtual Unified Individual profile that powers Agentforce and Flow in real time.

🧭 360 Card — Duplicate Management & Data Quality

• Rule: The Matching Rule defines what "the same" means; the Duplicate Rule decides how to enforce it per channel.
• Gain: You keep one consistent definition of a duplicate while applying smart enforcement—blocking manual UI typos while letting automated API integrations flow safely.
• Price: Native rules only police records inside one Salesforce org; they are not a full cross-enterprise Master Data Management (MDM) engine.
• Limits: Maximum 5 active matching rules and 5 active duplicate rules per object. Rules run after Validation Rules and can slow down large API bulk loads if not tuned properly.
• Mirror (The Manual Merge Trap): Letting duplicates pour in freely and hoping sales reps will manually merge them on Fridays is a losing battle—duplicate debt compounds faster than humans can click.
• When to Upgrade: The moment three or more external systems argue over who owns the customer's true address or phone number, step up to an MDM tool or Salesforce Data Cloud Identity Resolution.
• At Volume (LDV): Always deduplicate and normalize large files in an external staging database or ETL tool before loading them into Salesforce. Use native Salesforce rules as your human UI safety net, not your bulk ETL engine.

⚠ ARCHITECT & INTERVIEW TRAP:
Wrong Answer: "We don't need an MDM tool or Data Cloud to merge records across our 4 ERP systems—we can just turn on standard Salesforce Duplicate Rules to build our Golden Record."

Correct Answer: "Never confuse native Salesforce Duplicate Management with Master Data Management (MDM). Native rules only flag or block duplicates inside a single Salesforce org and have zero field-level survivorship logic. To merge records from 4 external systems into a Golden Record, you must use upstream MDM or Salesforce Data Cloud Reconciliation Rules."

❓ Core Q&A

Q1: How would you design duplicate prevention for Accounts imported nightly from two separate ERP systems, plus manual entry by sales reps?

🎯 Say this first: Use one Matching Rule to define "same account," split Duplicate Rules by channel (Block for UI, Report for API), and deduplicate nightly loads upstream using External IDs before they touch Salesforce.

A: Build a three-layer defense:

• Layer 1 (Upstream Staging & Upsert): Before the two nightly ERP feeds hit Salesforce, combine and clean them in an ETL staging table. Run your heavy matching there, assign a master External ID, and use the Bulk API upsert call so known records update cleanly without triggering fuzzy matching overhead.
• Layer 2 (Channel-Specific Duplicate Rules): Create a Matching Rule using Fuzzy Account Name + Exact Website/Billing Zip. Pair it with two Duplicate Rules: one that strictly Blocks manual saves from human UI profiles, and a second for the Integration User profile set to Allow and Report so unexpected fuzzy matches don't crash the midnight job.
• Layer 3 (Stewardship Loop): Schedule a weekly exception report on DuplicateRecordSet items so a data steward can merge the edge cases that slipped through the nightly feed.

🚀 Scenario-Based Follow-Ups

Q2: Our Duplicate Rules are active, yet duplicates are still silently slipping in during data loads. Why is this happening?

A: Check these four technical culprits and one classic architecture blind spot:

• 1. Action Set to Allow: The Duplicate Rule action for create/edit is set to Allow instead of Block, so it creates the record and quietly logs a report item nobody is reading.
• 2. Rule Filter Bypass: The rule has filter conditions (like Profile != Integration User) or an Apex batch class is purposefully using Database.DMLOptions.DuplicateRuleHeader.allowSave = true to bypass enforcement.
• 3. Two-Step Field Population: Your matching rule relies on Website and Phone, but the integration inserts a bare-bones skeleton record first and populates those fields five minutes later in a separate update job (where edit enforcement is turned off).
• 4. MatchKey Index Skips on LDV: During massive bulk loads or when fuzzy matching hits complex characters, the matching engine can hit governor thresholds and skip evaluation to protect database stability.
• 5. The Architectural Trap (Split External IDs): Using upsert with an External ID only prevents duplicates from the same source system. If SAP sends "Acme Inc" with ID SAP-99 and NetSuite sends the exact same "Acme Inc" with ID NS-500, Salesforce treats them as two completely separate records unless upstream MDM or cross-field matching catches them!

Q3: Leadership wants a "Golden Record" that merges customer data from 4 different platforms. Is native Salesforce duplicate management enough?

A: No—not even close. Native Salesforce duplicate management is just a gatekeeper; it cannot do automated cross-system merging, bidirectional syncing, or field-level survivorship (deciding which system wins when two systems disagree on a phone number). To build a true Golden Record across four systems, you either need an external MDM platform or Salesforce Data Cloud (Data 360) using Identity Resolution and Reconciliation Rules. Once that Golden Record is built, native Salesforce rules simply act as the UI bouncer so sales reps don't ruin it manually.

Q4: The exact same customer exists in three systems (CRM, ERP, Support) with different field values. Which data wins?

🎯 Say this first: Never use a lazy "Newest Record Wins" rule across the whole record—define field-level survivorship rules based on which system is the authoritative source for each attribute.

A: First, align with business leaders on deterministic matching keys (such as Tax_ID__c, or Exact Email + Normalized Phone) and run the match in staging or Data Cloud. Next, assign survivorship per field:

• Billing Address & Credit Limit: The ERP / Billing system always wins.
• Mobile Phone & Email: The most recently verified entry across Support or CRM wins.
• Account Owner & Tier: Salesforce CRM always wins.

Assemble the winning values into the Golden Record, and keep the losing source values in a child lineage table (or Data Lakehouse object) so you never permanently destroy historical audit trails.

Q5: Two departments are fighting over survivorship rules for a key field and won't budge. How do you break the tie as an architect?

A: Stop arguing in theory and put 10 real, messy customer records up on the projector screen. Abstract debates drag on for weeks; looking at real data settles arguments in thirty minutes. Show both VPs ten actual customers where System A and System B disagree, and ask, "Which value would you bet your bonus on calling right now?" If it's still a dead tie, store the winning primary value on the main record for automation, display the secondary system's value in a read-only reference field right underneath it on the Lightning page, and review user click data in 60 days.

Q6: How do developers handle or bypass Duplicate Rules gracefully inside custom Apex code?

A: In Apex, you have full programmatic control using the Database.DMLOptions class and the Datacloud.FindDuplicates namespace. If a developer is running a trusted system process that needs to save a record even when an Alert duplicate rule fires, they can set dmlOpts.DuplicateRuleHeader.allowSave = true;. Conversely, if building a custom Lightning Web Component (LWC) intake screen, developers can call Datacloud.FindDuplicates.findDuplicates(recordList) in Apex before saving to display matching cards cleanly in the custom UI.

// Apex Example: Controlling Duplicate Rule Behavior Programmatically
Account newAcc = new Account(Name = 'Acme Corp', Website = 'www.acme.com');

Database.DMLOptions dmlOpts = new Database.DMLOptions();
// Allow save if the Duplicate Rule is set to "Alert" (cannot bypass "Block")
dmlOpts.DuplicateRuleHeader.allowSave = true;
dmlOpts.DuplicateRuleHeader.runAsCurrentUser = true;

Database.SaveResult sr = Database.insert(newAcc, dmlOpts);
if (!sr.isSuccess()) {
    for (Database.Error err : sr.getErrors()) {
        if (err instanceof Database.DuplicateError) {
            System.debug('Duplicate blocked save: ' + err.getMessage());
        }
    }
}

Q7: Why does poor duplicate management ruin Agentforce and AI grounding?

A: Autonomous AI agents like Agentforce rely on Retrieval-Augmented Generation (RAG) and deterministic tool calls (like Get Record by email or name) to ground their prompts. If a customer has three duplicate Contact records—one attached to an open VIP support case, one with an old mailing address, and one marked "Do Not Contact"—the AI agent can pull the wrong record context, violate compliance rules, or give the customer contradictory answers. Clean, deduplicated identity resolution is the #1 prerequisite for trustworthy enterprise AI.

🔗 Connecting the Dots: The Shield Encryption Catch

Q8: Follow-Up Architecture Question — What happens to your Matching Rules when your security team turns on Salesforce Shield Platform Encryption? (Hint: Watch out for fuzzy matching!)

A: This is a classic enterprise gotcha that breaks deduplication if you don't plan for it! When you encrypt a field using Shield Platform Encryption, the impact depends entirely on which encryption scheme you pick:

• Probabilistic Encryption (The Breaker): Probabilistic encryption adds random noise (initialization vectors) every time data is encrypted so the ciphertext never looks the same twice. Because the database cannot index or compare randomized ciphertext, fields encrypted with Probabilistic Encryption cannot be used in Matching Rules at all!
• Deterministic Encryption (Exact Match Only): Deterministic encryption always produces the exact same ciphertext for the exact same plain-text input (e.g., "john@acme.com" always encrypts to the same string). However, because encryption scrambles the underlying letters, Fuzzy Matching is completely broken and unsupported on encrypted fields!
• The Architect's Workaround: If you must use Shield Platform Encryption on sensitive fields like Email, Phone, or Name and still need native Salesforce Duplicate Management to work, you must configure those fields with Case-Sensitive or Case-Insensitive Deterministic Encryption and update all your Matching Rules to use Exact Matching only—or move your fuzzy identity resolution into an upstream secure enclave before the data is encrypted in Salesforce.