⚡ Key Points & Non-Negotiable Migration Rules
Large Data Volume (LDV) migrations never fail because of the loading tool you picked. They fail because of broken sequencing, unexpected automation, and missing reconciliation. Lock these core rules into your cutover runbook:
- Strict Dependency Order: Deploy metadata and configuration first. Next, load Users, Roles, and Queues, followed by Reference/Master data. Then load Parent objects (
Account) before Child objects (Contact,Opportunity,Case), and finally Activities (Task,Event). - External IDs Are Mandatory: Every migrated object needs a custom field marked as an
External ID(andUnique). Salesforce allows up to 25 External ID fields per object. UsingUpsertagainst an External ID lets you re-run failed files without creating duplicates and links children to parents automatically without ExcelVLOOKUPnightmares. - Silence the Org & Defer Sharing: Turn off triggers, flows, and validation rules via a Hierarchy Custom Setting or Custom Permission bypass. Enable Defer Sharing Calculations so the sharing engine doesn't recalculate access rules after every single batch.
- Preserve Historical Timestamps: Enable the
Set Audit Fields upon Record Creationpermission for your migration integration user so legacyCreatedDateandCreatedByIdvalues carry over cleanly. - Rehearse at Full Scale: Never run your first full-volume load on go-live weekend. Run at least two to three end-to-end dress rehearsals in a Full Copy Sandbox to catch data skew, dirty formatting, and real wall-clock timings.
- Keep Legacy Systems Read-Only for Rollback: Your safest rollback on Monday morning isn't deleting 40 million Salesforce records—it is flipping the legacy CRM back from read-only to read/write while you fix the issue.
Cutover weekend looks smooth on paper, but the real victory happens three weeks earlier during your Full Sandbox rehearsal. In one dry run, we discovered that our script tried loading
Contact records before Account records (an ordering bug) and a forgotten record-triggered flow added 6 hours to the load time (a missing bypass flag). Because we caught both in rehearsal, our actual production cutover was just a calm, timed replay.
On one of my enterprise consolidation projects, we had to merge three legacy CRMs into a single Salesforce org. We had one strict 48-hour weekend window to pull it off.
The Old / Bad Way: Initially, project management wanted to skip the Full Copy Sandbox dress rehearsal to save a week on the schedule. Their argument was, "We already tested 5,000 rows in a Partial Sandbox, and a sandbox isn't identically tuned to production anyway—let's just load straight into production on Friday night."
Why That Would Have Failed: A 5,000-row test never reveals parent lock contention, storage limits, or hidden legacy data formatting bugs. Discovering at 1:00 AM on Sunday that a batch job takes 19 hours instead of 3 hours means the weekend is lost, and you have to make a painful Monday morning phone call to three business VPs.
How I Solved It (The Good Way): I pushed back and insisted on a mandatory Full Sandbox rehearsal using 100% production-volume extracts (roughly 22 million total records). That single weekend rehearsal uncovered three showstoppers:
- One script pipeline attempted to load regional
Contactsbefore the parentAccountshad finished upserting. - An active phone-number validation rule rejected 140,000 European legacy records that included extensions in the phone field.
- An integration flow was still active and tried queueing 2 million outbound platform events.
🛠️ The Master Sequencing Blueprint for LDV Orgs
Enterprise migrations don't fail because Bulk API 2.0 is slow; they fail when teams load data in the wrong order or forget how to verify that the data arrived intact. Here is the exact, battle-tested sequence I use for every Large Data Volume cutover:
- Step 1 — Metadata & Configuration First: Deploy all custom objects, fields, picklist values, record types, page layouts, and your custom
Bypass_Automation__chierarchy setting before touching any data. - Step 2 — Users, Roles, Profiles & Queues: Load all active and inactive
Userrecords (you can assign inactive users as record owners ifUpdate Records with Inactive Ownersis enabled alongsideSet Audit Fields), plus your Role Hierarchy and Queues. - Step 3 — Reference & Master Data: Load custom look-up tables such as Country/State codes, Product2 catalogs, Pricebooks, and Territory trees.
- Step 4 — Quiet the Org: Turn on your custom bypass flags for the migration user, disable email deliverability (set to No Access or System Email Only), and turn on Defer Sharing Calculations.
- Step 5 — Core Business Data (Parents Before Children): Load
Accountsfirst (sorted to avoid hierarchy locks). Next, loadContacts,Opportunities, andCases. Finally, load child objects likeOpportunityLineItem,Task,Event, andContentVersion(Files). Every single load usesUpsertagainst a legacyExternal ID. - Step 6 — Delta Pass: Run a final delta extract from the source systems to pick up any records created or modified between your initial seed extract and the start of the cutover freeze window.
- Step 7 — Reconcile & Verify: Compare source vs. target record counts, validate financial checksums (such as total
Opportunity.Amountby stage), and run spot-checks logged in as actual sales and service users. - Step 8 — Reactivate & Recalculate: Turn automation back on, run deferred sharing recalculations, re-enable email deliverability, and open the org to users.
• The Rule: Load parents before children, turn automation off, rehearse at full production volume, and time every step of the cutover.
• The Gain: A rehearsed migration turns a stressful go-live weekend into a predictable checklist. Nothing gets discovered by surprise at 2:00 AM.
• The Price: Proper dress rehearsals require a Full Copy Sandbox license and dedicated calendar time. Teams that cut rehearsals to "save time" almost always pay for it during a failed go-live.
• Platform Mechanics: Deploy metadata first, then users/roles/queues, then Accounts, then child records. Use
Upsert on an External ID so every batch is safe to re-run, and enable Set Audit Fields upon Record Creation to preserve historical dates.• The Phased Alternative: Migrating wave-by-wave by business unit gives you smaller weekend windows and lower blast radius—but forces you to maintain bidirectional sync between two live systems until the final wave finishes.
• Scope Discipline: Anything that does not fit inside your weekend cutover window must become a cold archive or a post-go-live background load—never an excuse for a 4-day business outage.
• At Extreme Volume: Always defer sharing calculations during the data load, otherwise the Salesforce sharing engine will do ten times more database work than your actual data loader.
When you migrate records—or restore deleted records from a backup—Salesforce generates brand-new 18-character Salesforce IDs and brand-new Auto-Number values. If your business users or downstream integrations rely on an Auto-Number field (like
CaseNumber or Order_Number__c) as a permanent key, restoring from backup will break their references! Always preserve the original number in a custom Legacy_Record_Number__c External ID field, or temporarily convert custom Auto-Number fields to Text during the historic load and switch them back to Auto-Number before users log in.
💬 Core Architecture & Strategy Q&A
A: Designing a migration without External ID fields means choosing to do your error recovery by painful manual archaeology. Here is what External IDs actually solve under the hood:
- Safe Re-Runs (Idempotency): When you run an
Upsertusing anExternal ID, Salesforce checks the custom index automatically. If a 100,000-row batch fails halfway through due to a network drop, you don't have to spend hours figuring out which 43,210 rows succeeded. You simply fix the issue and re-run the exact same file—existing rows update cleanly and missing rows insert. - Zero VLOOKUP Relationship Mapping: When you load
Accounts, you populateLegacy_Account_ID__c(marked as an External ID). When you loadContactsan hour later, you don't need to export the new 18-character Salesforce Account IDs and match them in Excel. You simply map the Contact's parent relationship directly toAccount:Legacy_Account_ID__cin your CSV header:
// Sample Bulk API 2.0 CSV Header for Contact Upsert: FirstName,LastName,Email,Legacy_Contact_ID__c,Account.Legacy_Account_ID__c "Rahul","Sharma","r.sharma@example.com","CONT-99201","ACC-44012"
- Seamless Delta Loads & Live Integrations: When you run your final Friday-night delta pass for records modified during the week,
Upsertupdates those records effortlessly. And once you go live, your middleware can keep using that exact sameExternal IDfor daily ERP synchronization.
A: This is a classic architecture trade-off between cutover window risk and coexistence complexity:
- When to Pick Big-Bang: If your sales, service, and finance teams all touch the exact same Accounts and Opportunities, AND your Full Sandbox rehearsal proves you can extract, load, and reconcile everything within a 48-hour weekend window, go Big-Bang. Everyone logs into a single source of truth on Monday morning, and you write zero throwaway integration code.
- When to Pick Phased (Wave-by-Wave): If you have 150 million records that take 5 days to load, or three distinct regional business units that refuse to freeze operations on the same weekend, you must phase it. However, be upfront with leadership about the hidden price: for several months, your company will have "split-brain" truth across two CRMs, and you will have to build and maintain temporary bi-directional sync pipelines until the final wave lands.
A: Consolidating three CRMs into one is where most enterprise programs stumble because they treat data migration as a last-minute admin task instead of an engineering track. Here is how I structure it:
- 1. Lock the Target Data Model First: Never dump three different schemas into Salesforce and hope to clean them up later. Define the single canonical Salesforce data model first. Every legacy field from CRM A, B, and C must map into that model—anything that doesn't fit forces a business decision early, not an ugly custom field workaround.
- 2. Deduplicate and Transform in a Staging Database: Do not use Salesforce as a data-cleaning scratchpad. Pull extracts from all three CRMs into a staging database (like Snowflake, SQL Server, or an ETL staging area). Standardize picklists (e.g., mapping
USA,U.S., andUnited Statesto one value) and merge duplicate customer accounts across the three systems before generating your load files. - 3. Preserve Multi-Source Lineage: Create distinct External ID fields (such as
Legacy_CRM_A_Id__c,Legacy_CRM_B_Id__c, or a composite key likeCRM_A_10045) plus aSource_System__cfield. That way, every record remains 100% traceable back to where it came from, and each system's load remains idempotent. - 4. Run Iterative Dress Rehearsals: Execute 3 to 4 full cycles in a Full Copy Sandbox: Extract → Clean → Transform → Load → Reconcile → Fix → Repeat. By Rehearsal #3, your scripts, timings, and reconciliation queries should run like clockwork.
A: A professional cutover weekend is scripted down to 15-minute intervals so nobody has to improvise while sleep-deprived:
- 1. Freeze & Read-Only Switch (Friday 5:00 PM): Announce the business freeze window weeks in advance. At 5:00 PM Friday, flip the legacy CRMs to Read-Only mode for regular users and stop all legacy background jobs.
- 2. Execute the Delta Extract & Load: Since your heavy historical data (e.g., closed records older than 6 months) should already be pre-loaded prior to the weekend, Friday night focuses on extracting and upserting the delta—everything created or modified since your pre-load seed extract.
- 3. Automated & Human Reconciliation (Saturday): Run automated row-count and financial-sum scripts comparing the legacy databases against Salesforce. Then bring in a small group of business super-users for a 2-hour smoke test using "Log In As" to verify page layouts, sharing visibility, and reports.
- 4. The Pre-Written Go/No-Go Gate (Sunday Noon): Define your exact Go/No-Go criteria in writing two weeks before the cutover (e.g., "99.9% of Accounts/Opportunities loaded, zero P1 defects, ERP integration heartbeat green"). At Sunday noon, leadership checks those boxes. It is a calm checklist, never an emotional debate.
- 5. The Boring Rollback Plan: Never shut down or wipe the legacy CRM servers on Friday night. Because they sat untouched in Read-Only mode all weekend, if a catastrophic blocker forces a "No-Go" at Sunday noon, your rollback takes 10 minutes: you re-enable Read/Write access on the legacy CRMs for Monday morning and fix your Salesforce load for the following weekend.
- 6. Two Weeks of Hypercare: Start Monday morning with a dedicated live triage channel, daily defect burn-down reviews, and admins ready to fix missing permission sets or sharing edge cases on the spot.
🔍 Scenario-Based Follow-Up Questions (Troubleshooting & Backup)
A: Always cut scope before you chase raw technical speed. Trying to blindly crank up parallel threads at the last minute usually triggers database row locks (UNABLE_TO_LOCK_ROW) that make the load take longer, not shorter. Here is the exact playbook:
- 1. Defer the Cold Historical Tail: Your business users need active Accounts, open Opportunities, and open Cases on Monday morning at 8:00 AM—they do not need 5 years of closed Tasks, old EmailMessages, or historical PDF attachments in hour one. Move all cold history out of the 6-hour cutover window and trickle-load it in the background during the week (or archive it directly to Big Objects / Data Cloud).
- 2. Pre-Load Static Data Ahead of the Window: Load 95% of your data—Reference data, Accounts, Contacts, and historical records—3 to 5 days before the weekend while the old system is still live. Then, your 6-hour weekend window only has to process a tiny Delta Load of records modified in the last 96 hours.
- 3. Parallelize Across Independent Objects, Sort Within Objects: Once scope is trimmed, tune your speed using metrics from your sandbox rehearsal. Run independent objects (like
Product2andAccount, orContactandOpportunityonce Accounts are in) in parallel streams, while keeping child files strictly sorted byParentIdso batches never fight for locks.
A: When reconciliation counts match 100%, a "missing" record is almost always a visibility or filtering issue, not a data loss issue. Check these four culprits in order:
- 1. Ownership & Role Hierarchy Placement: The records were accidentally loaded with the Integration User still set as the
OwnerId, or the actual record owners were loaded without being assigned to their Role in the Role Hierarchy. In aPrivateOrganization-Wide Default (OWD) model, sharing rules won't open up access until owners and roles are properly linked. - 2. Deferred Sharing Has Not Finished Recalculating: You turned on Defer Sharing Calculations for the weekend load, and either forgot to kick off the recalculation batch on Sunday night, or the background sharing job is still running across tens of millions of rows.
- 3. Missing Record Type or App Assignments: The records were loaded with a valid
RecordTypeId, or live inside a custom object tab, that hasn't been assigned to the complaining user's Profile or Permission Set. - 4. List View & Report Filters ("My Accounts" vs "All Accounts"): The user is looking at a default list view filtered by "My Opportunities" or "Created This Week" (and because you enabled
Set Audit Fields, the migrated records accurately show their 2023 creation date!). - The 2-Minute Diagnostic Path: Query the record by
External IDas a System Administrator first to confirm it exists and inspect itsOwnerIdandSharingbutton hierarchy. Then use "Log In As" to paste the direct 18-character Salesforce ID into the browser URL as the complaining user—if they get an Insufficient Privileges error, you know instantly it is a sharing/ownership gap.
A: Let's clear up one of the biggest myths in the ecosystem: Salesforce protects the cloud platform infrastructure (disaster recovery if a data center loses power), but it does NOT protect you from human or integration errors. If an admin accidentally runs a bad Data Loader delete, a rogue flow overwrites 500,000 phone numbers, or a deployment wipes out metadata, that is 100% your responsibility.
- Why Free Native Tools Are Not a Backup Strategy: The Weekly Export Service only gives you raw CSVs once every 7 days (or 29 days), the Recycle Bin purges after 15 days and caps out by storage size, and Salesforce's retired last-resort "Data Recovery Service" was slow, expensive, and returned unlinked flat files weeks later.
- Use a Dedicated Enterprise Backup Solution: A real strategy uses purpose-built backup tooling—such as Salesforce Backup (native paid add-on) or Own (formerly OwnBackup, now a Salesforce company), Gearset, or Odaseva.
- Back Up Both Data AND Metadata: Backing up record rows is useless if someone deleted the custom fields or picklist values those rows lived in. Your tool must snapshot both data and metadata daily.
- Rehearse the Restore Twice a Year: Restoring data requires the exact same architectural discipline as a migration: you must bypass triggers/flows, restore parents before children using External IDs to rebuild relationships, and reconcile counts. As the classic architect rule goes: "A backup you have never practiced restoring is just a hope, not a strategy."
A: RTO and RPO are the two business metrics that dictate how you design—and how much you spend on—your backup and disaster recovery architecture:
- Recovery Point Objective (RPO — "How much data can we afford to lose?"): Measured backward in time from the moment disaster strikes. If your RPO is 4 hours, you cannot rely on Salesforce's Weekly Export (which loses up to 7 days of data). You must configure incremental backups or CDC/Platform Event replication every 4 hours.
- Recovery Time Objective (RTO — "How fast must we be back up and running?"): Measured forward in time from the moment of failure until business operations resume. If your RTO is 6 hours, you must know—from actual sandbox restore rehearsals—how many rows per hour your backup tool can push back into Salesforce via Bulk API 2.0 with automation bypassed.
- Why Architects Care: The business defines the RTO and RPO numbers based on regulatory and revenue risk; the architect designs the backup frequency, API capacity, and restore runbook to hit those exact numbers.
A: If you simply dump backed-up CSV files back into Salesforce without a smart restore tool or runbook, five things break immediately:
- 1. Broken Salesforce IDs & Lookups: Once a record is permanently deleted past the Recycle Bin, its original 18-character
Idis gone forever. Re-inserting the record creates a brand-newId, which orphans all child records unless you restore parents first and relink children usingExternal IDsor a smart backup tool's ID-remapping engine. - 2. Changed Auto-Number Fields: Re-inserted Cases, Orders, or custom objects receive brand-new incrementing Auto-Numbers, breaking external customer references unless you convert the field or store legacy numbers separately.
- 3. Lost Historical Audit Dates: Unless
Set Audit Fields upon Record Creationis enabled during the restore, every restored record will show today's date as itsCreatedDate, ruining historical pipeline reports. - 4. Automation & Integration Storms: If you forget to turn on your Trigger and Flow bypass flags before restoring 200,000 records, Salesforce will fire 200,000 "Welcome" emails to customers and send 200,000 duplicate order messages to your ERP.
- 5. polymorphic & Complex Relationships: Restoring
Task.WhoId/WhatId,ContentDocumentLink(Files), andFeedItem(Chatter posts) requires multi-pass relationship translation because their parent IDs have changed.
🔗 Connect the Dots: Follow-Up Questions to Test Yourself
To master enterprise data migrations and cutover governance, see how well you can answer these three real-world follow-up scenarios before your next architecture review:
- 1. The Sandbox Seeding & Privacy Question: "When we run our dress rehearsal in a Full Copy Sandbox using real production data from three legacy CRMs, how do we prevent accidentally emailing real customers or exposing sensitive PII (like SSNs or credit details) to contractors?" (Hint: Always mask sensitive PII fields during your staging transformation or with Salesforce Data Mask, turn Email Deliverability to No Access, and append
.invalidto all customer email addresses in non-production environments.) - 2. The Polymorphic Activity Question: "Why is migrating historical Tasks, Events, and Files (
ContentVersion/ContentDocumentLink) always the slowest part of a cutover weekend, and how should we sequence them?" (Hint: Activities and Files often outnumber core Accounts 20-to-1 and rely on polymorphic lookups likeWhatIdandLinkedEntityId. Always load active open Tasks for cutover, and schedule historical closed Tasks and large binary files as a post-go-live background wave.) - 3. The In-Flight Integration Question: "During the 48-hour weekend cutover freeze, our e-commerce website still needs to take customer orders 24/7. Where do those orders go while legacy CRM is read-only and Salesforce is mid-migration?" (Hint: Queue incoming orders in your middleware message broker—like MuleSoft Anypoint MQ or Kafka—during the cutover window, and replay the queue into Salesforce on Sunday afternoon right after your Go/No-Go sign-off!)
Comments
Questions, corrections and your own experience are all welcome.
Post a Comment