Data Processing Engine: A Guide to Nodes and Transformations

Introduction to Nodes

In Data Processing Engine (DPE), nodes are the fundamental building blocks that enable you to construct sophisticated data processing workflows. Each node performs a specific operation—such as filtering, joining, or aggregating—allowing you to manipulate and analyze data efficiently.

Availability & Permissions

Editions: Professional, Enterprise, Unlimited, Developer
Access: Lightning Experience
Required Permissions:

  • Customize Application
  • Modify All Data

Core Node Types

1. Data Source Node

  • Defines the origin of your input data (e.g., CRM objects, Data Lake objects, external datasets).

2. Transformation Nodes

Apply logic to modify or enhance your data:

  • Filter Node: Exclude records based on conditions (e.g., Status = "Closed").
  • Formula Node: Calculate new fields (e.g., Revenue = Quantity * Unit Price).
  • Join Node: Merge datasets using key fields (inner, left, right, or full joins).
  • Append Node: Stack rows from multiple sources (e.g., combining regional sales data).
  • Aggregate Node: Roll up data (sum, average, count) by dimensions (e.g., Total Sales by Region).

3. Advanced Nodes

  • Hierarchy Node: Analyze parent-child relationships (e.g., organizational structures).
  • Forecast Node: Predict future trends using historical data (e.g., quarterly revenue projections).
  • Slice Node: Select specific fields for downstream use.

4. Writeback Nodes

  • Writeback Object Node: Save processed data to Salesforce objects, Data Lake, or Analytics datasets.
  • Composite Writeback Node: Update interrelated records in a single operation.

Key Workflows

Batch Data Transforms

  1. Input Node: Pull data from sources (Data Lake or Data Model objects).
  2. Transform Node: Cleanse, enrich, or restructure data.
  3. Output Node: Write results to a target (e.g., CRM, analytics datasets).

Joining Data

  • Use Join Nodes with fully qualified keys (FQKs) to ensure accuracy.
  • Enable null-safe joins to handle missing values.

Appending Datasets

  • Combine datasets (e.g., APAC + EMEA sales) while managing duplicates manually.

Pro Tips

🔹 Reference Nodes: Check dependencies before modifying a node to avoid downstream issues.
🔹 Node Cloning: Copy/paste nodes across workflows for efficiency.
🔹 Hierarchical Aggregation: Roll up multi-level data (e.g., sales team → region → global).


Example Use Cases

  1. Sales Forecasting:
    • Input: Historical opportunity data
    • Nodes: Forecast → Aggregate → Writeback
  2. Customer Segmentation:
    • Input: Leads + Engagement data
    • Nodes: Join → Filter → Formula (Score Calculation)

Permissions & Best Practices

  • Admin Tasks: Require Edit CRM Analytics Dataflows or Edit Dataset Recipes.
  • Optimization: Minimize processing time by filtering early in the workflow.

Next Steps

Experiment: Build a simple transform (e.g., filter + append).
Explore: Use forecast nodes for predictive analytics.
Collaborate: Share reference node insights with your team.

DPE’s modular node system empowers you to streamline ETL, reporting, and AI-driven analytics—all within Salesforce.

🔔🔔  Follow us on LinkedIn  🔔🔔

Related Posts
Who is Salesforce?
Salesforce

Who is Salesforce? Here is their story in their own words. From our inception, we've proudly embraced the identity of Read more

Salesforce Marketing Cloud Transactional Emails
Salesforce Marketing Cloud

Salesforce Marketing Cloud Transactional Emails are immediate, automated, non-promotional messages crucial to business operations and customer satisfaction, such as order Read more

Salesforce Unites Einstein Analytics with Financial CRM
Financial Services Sector

Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more

AI-Driven Propensity Scores
AI-driven propensity scores

AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more