Data Processing Engine: A Guide to Nodes and Transformations
Introduction to Nodes
In Data Processing Engine (DPE), nodes are the fundamental building blocks that enable you to construct sophisticated data processing workflows. Each node performs a specific operation—such as filtering, joining, or aggregating—allowing you to manipulate and analyze data efficiently.
Availability & Permissions
✔ Editions: Professional, Enterprise, Unlimited, Developer
✔ Access: Lightning Experience
✔ Required Permissions:
Customize ApplicationModify All Data
Core Node Types
1. Data Source Node
- Defines the origin of your input data (e.g., CRM objects, Data Lake objects, external datasets).
2. Transformation Nodes
Apply logic to modify or enhance your data:
- Filter Node: Exclude records based on conditions (e.g.,
Status = "Closed"). - Formula Node: Calculate new fields (e.g.,
Revenue = Quantity * Unit Price). - Join Node: Merge datasets using key fields (inner, left, right, or full joins).
- Append Node: Stack rows from multiple sources (e.g., combining regional sales data).
- Aggregate Node: Roll up data (sum, average, count) by dimensions (e.g.,
Total Sales by Region).
3. Advanced Nodes
- Hierarchy Node: Analyze parent-child relationships (e.g., organizational structures).
- Forecast Node: Predict future trends using historical data (e.g., quarterly revenue projections).
- Slice Node: Select specific fields for downstream use.
4. Writeback Nodes
- Writeback Object Node: Save processed data to Salesforce objects, Data Lake, or Analytics datasets.
- Composite Writeback Node: Update interrelated records in a single operation.
Key Workflows
Batch Data Transforms
- Input Node: Pull data from sources (Data Lake or Data Model objects).
- Transform Node: Cleanse, enrich, or restructure data.
- Output Node: Write results to a target (e.g., CRM, analytics datasets).
Joining Data
- Use Join Nodes with fully qualified keys (FQKs) to ensure accuracy.
- Enable null-safe joins to handle missing values.
Appending Datasets
- Combine datasets (e.g., APAC + EMEA sales) while managing duplicates manually.
Pro Tips
🔹 Reference Nodes: Check dependencies before modifying a node to avoid downstream issues.
🔹 Node Cloning: Copy/paste nodes across workflows for efficiency.
🔹 Hierarchical Aggregation: Roll up multi-level data (e.g., sales team → region → global).
Example Use Cases
- Sales Forecasting:
- Input: Historical opportunity data
- Nodes: Forecast → Aggregate → Writeback
- Customer Segmentation:
- Input: Leads + Engagement data
- Nodes: Join → Filter → Formula (Score Calculation)
Permissions & Best Practices
- Admin Tasks: Require
Edit CRM Analytics DataflowsorEdit Dataset Recipes. - Optimization: Minimize processing time by filtering early in the workflow.
Next Steps
✔ Experiment: Build a simple transform (e.g., filter + append).
✔ Explore: Use forecast nodes for predictive analytics.
✔ Collaborate: Share reference node insights with your team.
DPE’s modular node system empowers you to streamline ETL, reporting, and AI-driven analytics—all within Salesforce.
🔔🔔 Follow us on LinkedIn 🔔🔔













