💬 In plain words: Transformations reshape your data directly inside Salesforce Data Cloud. Streaming transforms evaluate records one by one as they arrive in real time for lightweight calculations. Batch transforms run on a schedule across entire tables for heavy multi-source joins, aggregations, and large-scale data cleansing.
📌 Real-World Example: When a customer clicks on your website, a streaming transform instantly extracts the
device_type from the incoming user-agent string. Later that night, a batch transform joins historical orders with product catalogs to compute each customer’s total spend by category across the entire dataset.
Understanding Data Cloud Transformations
Transformations clean, restructure, and enrich raw source data before mapping it to target Data Model Objects (DMO). Once processed through a Data Lake Object (DLO), the output fuels unified customer profiles and downstream activations.
- Streaming Transforms: Execute immediately in the live ingestion path. They evaluate only newly arriving data packets and calculate lightweight formula fields on the fly without reprocessing existing records.
- Batch Transforms: Run on data at rest on an automated schedule. They have access to the complete source dataset, making them ideal for heavy data cleansing, multi-table joins, and complex denormalization.
- Pipeline Target: Both transformation types typically write their cleaned output to a target
DLO, which is then mapped directly to a standard or customDMO.
Streaming vs. Batch: Quick Comparison
| Feature | Streaming Transform | Batch Transform |
|---|---|---|
| Execution Trigger | Immediately as data arrives (event-driven) | Scheduled intervals on data at rest |
| Processing Scope | New, incoming records only | Entire historical dataset in the source DLO |
| Primary Use Case | Low-latency, per-record calculations and simple formulas | Complex multi-source joins, aggregations, and full-table cleansing |
🧠 Core Architectural Rule: Use Streaming Transforms for immediate, single-record enrichment in flight. Use Batch Transforms for complete historical processing and heavy multi-object joins.
🧭 360 Card — Data Transformations
- Rule: Streaming transforms shape individual records upon arrival; batch transforms recalculate full datasets on a schedule.
- Gain: Real-time feeds remain fast and responsive, while resource-heavy processing runs at rest without blocking pipelines.
- Price: Streaming transforms do not backfill or modify historical data already stored in the system.
- Limits: Streaming operations support lightweight formula logic only; complex multi-table joins require batch pipelines.
- Upstream Alternative: Transforming data upstream in source systems centralizes logic, but tightly couples source systems to your Data Cloud data model.
- At Scale: Full-table batch runs consume significant compute. Schedule them during off-peak hours to avoid pipeline congestion.
⚠ INTERVIEW TRAP: A streaming transform will never update historical records that were ingested prior to its deployment. This is intentional architecture, not a platform bug. If historical data must be modified, run a batch backfill first, then rely on the streaming transform for continuous updates.
Core Interview Q&A
Q: When would you use a streaming transform versus a batch transform in Salesforce Data Cloud?
🎯 Say this first: Choose streaming transforms for instant, per-record calculations as data lands. Choose batch transforms for multi-table joins, aggregations, and full-dataset cleansing on a scheduled basis.
Detailed Answer: Selection comes down to latency, compute cost, and data scope:
- Streaming Transforms: Trigger instantly on each inbound record to calculate formula values, standardize string formats, or categorize events in the live ingestion stream.
- Batch Transforms: Query data at rest on defined schedules. They process historical records across one or more source
DLOs, allowing you to build denormalized structures and summarize business metrics. - Hybrid Approach: Real-world implementations often pair both: streaming handles live event derivations, while a recurring batch job performs periodic reconciliations across the broader dataset.
Scenario Follow-Up
Q: A newly created streaming transform is working for new incoming records but hasn't updated existing records in the DLO. How do you resolve this?
A: This behavior is by design. Streaming transforms only evaluate incoming data at the time of ingestion; they do not traverse historical data.
- Do not attempt to fix or recreate the streaming transform for past data.
- Create and execute a one-time Batch Transform across the source
DLOto backfill the historical records. - Keep the streaming transform active to continuously process new inbound records.