ORCFLO Documentation

Overview

The Transform component lets you clean, reshape, and organize data between workflow steps using a declarative pipeline of operations. Think of it as a series of spreadsheet steps applied automatically — filter out unwanted rows, sort by any field, rename messy column names, remove duplicates, and format the result for the next step or for export.

Unlike AI model steps, Transform runs deterministically. It is ideal for preparing data before sending it to a model step, or for post-processing model output before passing it downstream.

How It Works

Point to Input Data

Specify the output from a previous step using a reference like ${previous_step.output}. The input must be a list (array) of objects.

Build an Operations Pipeline

Add one or more operations in order. Each operation receives the output of the previous one, forming a pipeline.

Access the Result

The transformed data is available to downstream steps via ${step_name.output} or the custom key you configure under Save Result As.

Operations Reference

Operations are applied in the order they appear in your pipeline. Drag them to reorder. The Format Output operation, if used, must always be placed last.

Filter

Keep only rows that match a condition

Removes rows where the specified field does not match the condition. Supported operators: equals, not_equals, contains, not_contains, greater_than, less_than, is_empty, is_not_empty.

Example — keep only enterprise leads

op: filter
field: company_size
operator: equals
value: Enterprise

Pick Fields

Select only the fields you need

Removes all fields except those explicitly listed. Use this to reduce payload size before passing data to an AI step.

Example — keep only name, email, and score

op: pick
fields:
  - name
  - email
  - score

Rename Fields

Give fields friendlier names

Renames one or more fields without altering their values. Useful for cleaning up API response keys into readable labels.

Example — rename API keys to readable labels

op: rename
mapping:
  usr_nm: username
  acct_bal: account_balance

Sort

Order rows by a field

Sorts the list ascending or descending by any field. For numeric fields, numeric ordering is used; for string fields, lexicographic ordering applies.

Example — sort products by price descending

op: sort
field: price
order: desc

Limit

Keep only the first N rows

Truncates the list to at most N items. Combine with Sort to implement top-N selection patterns.

Example — keep top 10 results

op: limit
count: 10

Deduplicate

Remove duplicate rows

Removes duplicate rows based on the value of a specified field. The first occurrence of each unique value is kept.

Example — dedupe by email address

op: dedupe
field: email

Flatten

Expand nested arrays into rows

Expands a nested array field so each element in the array becomes its own top-level row. Useful when an API returns objects with embedded arrays.

Example — flatten tags array

op: flatten
field: tags

Add Computed Field

Derive a new field from existing ones

Adds a new field to every row using a template expression. Use the Insert Variable button to reference existing field values.

Example — combine first and last name

op: add_field
field: full_name
value_template: "first_name last_name"

Format Output

Convert to JSON, CSV, Markdown, or plain text

Serializes the final list into a string in the chosen format. This is useful when passing data to an AI model that expects a specific format, or when exporting results. Must always be the last operation in the pipeline.

Example — output as Markdown table

op: format
output_format: markdown

Format must be last

The Format operation converts your structured array into a string. Any operations placed after it will not work as expected. Always place Format at the end of your pipeline.

Configuration

Field	Description
Input Data	Reference to the data you want to transform, e.g. `${previous_step.output}`. Must be a list (array) of objects.
Operations	Ordered list of transform operations. Each operation is applied in sequence to the result of the previous one.
Save Result As	The variable key under which the result is stored. Defaults to `data`. Access it downstream as `${step_name.data}`.

Full example — top 5 enterprise leads formatted as CSV

- name: Filter Enterprise Leads
  step_type: transform
  transform_input: "${crm_fetch.output}"
  transform_output_key: leads
  transform_operations:
    - op: filter
      field: tier
      operator: equals
      value: Enterprise
    - op: pick
      fields: [name, email, arr]
    - op: sort
      field: arr
      order: desc
    - op: limit
      count: 5
    - op: format
      output_format: csv

Use Cases

Transform steps work well anywhere you need to shape data before or after an AI model step.

Reduce a large CRM export to only the fields an AI step needs
Sort and limit a product catalog to the top 20 before generating recommendations
Rename API response keys into human-readable labels before summarization
Deduplicate a mailing list by email address before sending outreach
Flatten nested tag arrays so each tag becomes a searchable row
Add a full_name field by combining first and last name
Format results as a Markdown table to include directly in an email step
Convert JSON output to CSV for download or export

Best Practices

Use Pick Fields early: Reducing the number of fields before later operations makes the pipeline faster and reduces token usage when passing to AI steps.
Chain Filter → Sort → Limit for top-N: Apply Filter first to remove irrelevant rows, then Sort by your ranking field, then Limit to keep only the top results.
Format Output last: Once you add a Format operation, the output becomes a string. Keep it as the final step so earlier operations can still work with structured objects.
Use a descriptive Save Result As key: Instead of the default data, use a name like filtered_leads to make downstream step references self-documenting.
Dedupe before sorting: Remove duplicates before sorting to avoid counting the same item multiple times in a top-N result.

Preview in execution history

After running your workflow, open the execution history and inspect the Transform step output to verify the pipeline produces the shape you expect before connecting it to downstream steps.

Troubleshooting

Issue	Solution
Output is empty after Filter	Check that the field name and value are exact — the filter is case-sensitive. Use the execution history to inspect what the input data looks like before the filter runs.
Format warning: must be last	Drag the Format operation to the bottom of the pipeline. Operations after Format receive a string instead of an array, which will cause unexpected results.
Downstream step can not read the result	Check the Save Result As key in the Advanced section. The downstream step must reference the exact key, e.g. `${transform_step.leads}` if the key is `leads`.
Sort order seems wrong for numbers	Ensure the field contains actual numbers, not numeric strings. If the source API returns numbers as strings, add an Add Computed Field operation first to cast the value.

Key Takeaways

Transform processes data deterministically with no AI token cost
Operations run in order — each receives the output of the previous one
Format Output must always be the last operation if used
Use Pick Fields early to reduce payload size before AI steps
Chain Filter → Sort → Limit for efficient top-N selection