The Ultimate Guide to Salesforce Batch Apex: Processing Millions of Records Safely

💬 In plain words:
Batch Apex is the heavy-duty freight train for processing millions of Salesforce records. It grabs a massive query, chops it into smaller chunks (default is 200 records), and processes them piece by piece. Because each chunk gets a fresh set of governor limits, you can safely update massive datasets. When volume is the problem, Batch Apex is the solution.

📌 The 4 Million Customer Example:
Imagine you need to recalculate loyalty points for 4 million customers. Batch Apex queries them all at once (using a QueryLocator to stream up to 50 million rows safely) and processes them 200 at a time. That creates 20,000 mini-transactions, each with its own limits. If chunk #47 encounters an error and fails, the other 19,999 chunks will still commit successfully.

🎬 Real-Life Application: 3 Million Rows Before Breakfast

A sudden price change means you need to recalculate the fuel surcharge on 3 million Delivery records overnight.

The Old/Bad Way: Write one standard Apex method that queries everything, loops through the records, and updates them. You run this via Anonymous Apex at midnight.

Why it is bad: Standard queries die at the 50,000 row limit. Even if you try to chunk it manually, a single transaction cannot carry that much DML weight. It will crash at 2 AM—nothing gets committed, and you have no clean way to restart.

The New/Good Way:

• Write a Batch class. The start() method returns a Database.getQueryLocator to safely stream millions of rows.
• The execute() method receives 200 records at a time. Each batch gets brand new governor limits.
• The finish() method emails a summary to the admin.

The Payoff: 15,000 small, safe transactions. If one chunk fails, it logs the error, and you only have to re-run those 200 records—not 3 million.

🧠 Freight Train Memory Aid:
Millions of rows → chopped into chunks of 200 → each chunk gets fresh limits. High-volume nightly jobs belong on this train.

Core Concept: How Batch Apex Works

Batch Apex allows you to break down large jobs into manageable chunks. It operates in three distinct phases:

• start(): This method sets up the data. It returns either a Database.QueryLocator (capable of streaming up to 50 million rows) or an Iterable (for custom data sets).
• execute(): This runs once for every chunk of data. By default, the chunk (or "scope") size is 200 records, but you can configure it. Every time this method runs, Salesforce resets the governor limits.
• finish(): This runs exactly once after all chunks have been processed. Use it to send confirmation emails or chain another batch job.

By default, Batch Apex is stateless. If you need to retain data across chunks (like running a grand total), you must implement Database.Stateful. However, this comes at a performance cost because the platform must serialize and save the class instance state after every single chunk.

For concurrency, Salesforce allows up to 5 batch jobs to run simultaneously. Additional batch jobs are placed in the Apex Flex Queue, which can hold up to 100 pending jobs.

🧭 360 Card: Batch Apex Summary

• Rule: Use a batch when the requirement is "run this operation over millions of rows." The platform handles the chunking.
• Gain: Each chunk acts as its own transaction with fresh governor limits, making data volume irrelevant.
• Price: Increased complexity and latency. You need to write three separate methods, schedule the job, and wait for asynchronous execution. It is the wrong tool for instant UI updates.
• Limits: QueryLocator bypasses standard limits to stream up to 50M rows. 5 concurrent batches max; 100 in the Flex Queue.
• Mirror: Consider Queueable Apex if your steps involve different types of work rather than running the same operation over massive row counts. Queueable is simpler for smaller volumes (under 50k rows).
• At Volume: Database.Stateful forces the platform to serialize the instance between chunks. Keep your state variables tiny, or performance will tank over time.

⚠ INTERVIEW TRAP: Batch Scope Size
Batch scope size is NOT a setting you change in the Salesforce Setup menu. You define it dynamically in code when starting the job:
Database.executeBatch(new MyBatch(), 100);
Why change it? The heaviest action in your execute block dictates the scope. For example, if you make one API callout per record, you must lower the scope to 100 because Salesforce enforces a hard limit of 100 callouts per transaction.

Core Q&A

Q: Stateful vs stateless batch — how do you accumulate a total across 2 million records, and what is the cost?

🎯 Say this first: Implement Database.Stateful and accumulate the total in an instance variable. The cost is the serialized state passed between chunks, which slows the job down. Keep the stored state tiny.

A: Implement Database.Stateful and add to your instance variable inside the execute() block. The platform serializes the entire batch instance between chunks so the variables survive.

• The Catch: Everything non-transient in the class gets serialized. Keep the state incredibly lean. A single Decimal counter is great. A Map storing 2 million IDs will blow up your heap size and serialization limits.
• The Alternative: If state gets heavy, write partial chunk results to a custom staging object, then query and aggregate them in the finish() method. You trade DML operations for serialization safety.
• Error Semantics: One failed execute() chunk does not roll back the successful ones. A stateful total must account for partial failures, so you should always track and log failed scopes.

Follow-Ups (Scenario-Based)

Q1: QueryLocator vs Iterable in start() — when is each right, and what limit differs?

A1: A QueryLocator streams SOQL results and safely scales up to 50 million records. It should be your default choice when processing database rows.

• An Iterable is meant for non-query data: merging API responses, hardcoded lists, or highly custom ordering logic.
• The Limit Trap: If you build an Iterable by running a standard SOQL query inside start(), it falls back to the standard 50,000-row limit. The 50 million ceiling is an exclusive feature of QueryLocator.
• If you think you need an Iterable just to "post-process" query results, don't. Keep the QueryLocator and do that post-processing inside the execute() method instead.

Q2: A nightly batch overlaps with the next morning's run during month-end volume. What are the design fixes?

A2: First, ensure the runs cannot physically overlap.

• Inside start(), query the AsyncApexJob table. Look for instances of the same class in 'Processing' or 'Preparing' status. If found, safely abort or defer the new run.
• Alternatively, chain the jobs: schedule a controller class that only fires the batch if the queue is clear, and have the finish() method schedule the next run based on completion time, not the wall clock.
• To address the root cause (duration), ensure your start() query is highly selective and uses indexed filters. Adjust your scope size to match per-chunk limits perfectly.
• If volume allows, split the data by partitions and run parallel batches (keeping within the 5 concurrent job limit).
• Finally, monitor execution times. Trend the durations so month-end growth becomes obvious before it causes a collision.

Q3 (Compare): When do you pick an Iterable over a QueryLocator — and what is the trap?

A: Use an Iterable only when the data does not come from a single SOQL query (e.g., merging parsed JSON from an external API or generating custom objects in memory).

The trap: An Iterable built from SOQL falls back to the standard 50,000 row limit. Only QueryLocator gets the 50 million row exception. Saying "I need an Iterable to clean up the query results first" is a bad design pattern. Keep the huge limit by using a QueryLocator and do the data cleanup inside the execute() block.