# Conductor LLM context
This generated file is a curated technical context for Conductor. Source pages remain authoritative; regenerate this file with scripts/generate-llm-context.py after updating a listed page.
# Workflows
Conductor separates what a workflow is from each instance of when it runs. A **workflow definition** declares which tasks run, in what order, and how data passes between them. When you start a workflow, Conductor creates a **workflow execution**, which is a single run of that blueprint with its own ID, input, and history. Because the two are separate, editing a definition never rewrites the history of an execution that already ran. In practice, developers evolve definitions through versions, while operators inspect and recover executions.
Every workflow moves through the same lifecycle:
```mermaid
flowchart LR
define[Build a definition] --> register[Register a version]
register --> trigger[Start or trigger]
trigger --> execute[Durable execution]
execute --> observe[Inspect and operate]
observe --> evolve[Version and roll out]
evolve --> register
```
An execution is durable because Conductor saves progress after every task. That is why work can span services, wait on people or timers, and pick up where it left off after a restart. Since Conductor hands work from one task to the next, each task must spell out its own contract: where its inputs come from and what happens when it fails. Worker tasks add one more requirement. If no worker is polling for the task, the workflow simply waits and does not advance.
Day-to-day work with workflows typically falls into one of the following four activities.
- **Build**
Define the contract, select system tasks or workers, wire data, validate the schema, and register a version. Start with [Create or update workflows](../how-tos/Workflows/creating-workflows.md).
- **Run**
Start an execution, capture its workflow ID, and inspect task input, output, and status. Start with [Start workflows](../how-tos/Workflows/starting-workflows.md).
- **Trigger**
Choose whether an application, schedule, event, parent workflow, or external signal owns the next transition. Start with [Choose a trigger](../how-tos/Workflows/choosing-a-trigger.md).
- **Operate**
Add timeouts and retries, search executions, debug failures, recover safely, and roll out new versions. Follow the [best practices](../bestpractices.md).
## Choose how work runs
Most workflow steps should use a built-in system task. Use a `SIMPLE` task when code must execute in your service or no built-in task represents the operation.
| Requirement | Choose | What operates it |
|---|---|---|
| Call HTTP, wait, branch, fork, transform JSON, publish an event, or start another workflow | Built-in system task | Conductor server |
| Execute domain logic, access a private library, or call a proprietary system | `SIMPLE` task | Your worker process |
| Run a child and wait for its result | `SUB_WORKFLOW` | Conductor server |
| Start a child and continue immediately | `START_WORKFLOW` | Conductor server |
A `SIMPLE` task needs both a registered task definition and a worker polling the exact task type. Without them, the task remains queued and the workflow does not advance. The [task chooser](../how-tos/Tasks/choosing-tasks.md) covers the complete built-in catalog; the [first-worker quickstart](../../quickstart/first-worker.md) covers the external-worker path.
## Choose how execution starts or resumes
| Requirement | Mechanism | Use when |
|---|---|---|
| A service or user starts work now | Direct API, CLI, or SDK start | The caller already owns the request and input |
| Work starts at a time or cadence | Schedule | Cron and timezone define when to create a new execution |
| A message starts or advances work | Event handler | A broker or Conductor event is the source of truth |
| One workflow invokes another | `SUB_WORKFLOW` or `START_WORKFLOW` | The parent owns composition explicitly |
| Existing work pauses for an external decision | `WAIT`, `HUMAN`, or `asyncComplete` plus a task signal/event action | The same execution must resume rather than create a new one |
Do not use business correlation alone to complete waiting work through an event handler. The implemented OSS actions require a `taskId`, or a `workflowId` plus `taskRefName`. See [Event orchestration](../how-tos/event-bus.md) for delivery and idempotency rules.
## A practical lifecycle
During **Build**, define inputs and stable output parameters before task wiring. Prefer built-in tasks; register every task definition required by a `SIMPLE` step. Validate the definition, then use mocked workflow testing to exercise branches without invoking real dependencies. Finally run one real execution against test dependencies.
During **Run**, start a pinned version when repeatability matters, record the returned workflow ID, and inspect the execution rather than assuming submission means completion. Synchronous start is convenient for bounded tests; asynchronous start plus status lookup is safer for long-running work.
During **Trigger**, make ownership explicit. Schedules always create executions. Events can create executions or complete/fail an identified task. Workflow composition expresses a known dependency directly. Signals resume work that already exists.
During **Operate**, configure task retries and all relevant timeouts, define idempotent worker behavior, carry correlation data, monitor queues and execution state, and rehearse recovery. Roll out breaking input or output changes as a new workflow version, and keep callers pinned until they are ready.
## Pick your route
For a first success in a local environment, follow [Run your first workflow](../../quickstart/first-workflow.md). It uses only built-in tasks and ends with an observable completed execution.
For a production service, follow the [best practices](../bestpractices.md). They connect contract design, validation, real-boundary testing, worker deployment, reliability policy, observability, and recovery drills. Use the Recipes section when you already understand the lifecycle and want a compact runnable variant.
# Create or update workflows
A workflow definition is a versioned JSON document. It declares the workflow's name, its inputs and outputs, and the tasks it runs. This page covers writing that document, validating it, and registering it with the server.
## Prerequisites
- A reachable Conductor server and configured CLI.
- A task definition and polling worker for every `SIMPLE` task.
## 1. Write the definition
A minimal definition names the workflow, lists its tasks, and maps its inputs and outputs:
```json
{
"name": "order_flow",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["orderId"],
"tasks": [
{
"name": "process_order",
"taskReferenceName": "process_order_ref",
"type": "SIMPLE",
"inputParameters": {
"orderId": "${workflow.input.orderId}"
}
}
],
"outputParameters": {
"status": "${process_order_ref.output.status}"
}
}
```
Save it as `workflow.json`. A few rules to follow:
- Give every task a unique, descriptive `taskReferenceName`. Other tasks reference its output through that name.
- Prefer a [built-in task](../Tasks/choosing-tasks.md) when one covers the operation. A `SIMPLE` task needs a registered task definition and a polling worker, or it stays queued at runtime.
- Keep `outputParameters` stable across versions, because callers depend on them.
The [workflow definition reference](../../../documentation/configuration/workflowdef/index.md) documents every field.
## 2. Validate before registration
```bash
curl -i -X POST 'http://localhost:8080/api/metadata/workflow/validate' \
-H 'Content-Type: application/json' \
--data-binary @workflow.json
```
Success is an empty `200 OK` response. Validation checks the definition, not worker availability or external connectivity.
## 3. Register the definition
```bash
conductor workflow create workflow.json
```
Success is a registered name and version visible through:
```bash
conductor workflow get
```
The REST equivalents are `POST /api/metadata/workflow` for create and `PUT /api/metadata/workflow` for an update body containing an array of definitions. See the [Metadata API](../../../documentation/api/metadata.md) for both endpoints.
## 4. Verify SIMPLE task dependencies
List registered task definitions and compare them with every workflow task whose `type` is `SIMPLE`:
```bash
conductor taskDef list
```
Then verify that a worker polls each exact task type. Registration alone does not start a worker.
## 5. Test and run
Use [Validate and test workflows](testing-workflows.md) to mock branches through `/api/workflow/test`, then run one real execution against test dependencies.
## Update and version safely
Use a new version when inputs, outputs, task order, or failure semantics change in a way callers can observe. Register the new version, update callers deliberately, and leave the previous version available while existing callers or executions need it. See [Managing Workflow Versions](versioning-workflows.md).
## Create in the UI
1. In the left navigation, open **Definitions** and select **Workflow**.
2. Select **Define workflow** in the top right. The editor opens with an empty Start-to-End graph.
3. Under **Workflow Details**, enter a unique name and a description.
4. Add tasks either visually or as JSON:
- Select the **+** node on the canvas to insert a task, then configure it in the **Task** panel.
- Or open the **Code** tab and paste a complete JSON definition.
5. Select **Save**. Resolve any warnings the editor reports first.
To change an existing workflow, open it from **Definitions** and then **Workflow**, edit it, and save. Use the CLI/API flow in automation so the checked-in definition remains the source of truth.
## Limitations
- Definition validation does not verify task worker deployment, credentials, broker topics, or HTTP reachability.
- Updating the same version in place makes rollout and rollback harder to reason about.
- Large input/output payloads belong in external storage; carry references in the workflow.
Next, [start the workflow](starting-workflows.md) and inspect the returned execution.
# Workflows
A **workflow** is a sequence of tasks with a defined order and execution. Each workflow encapsulates a specific process, such as:
- Classifying documents
- Ordering from a self-checkout service
- Upgrading cloud infrastructure
- Transcoding videos
- Approving expenses
In Conductor, workflows can be defined and then executed. Learn more about the two distinct but related concepts, **workflow definition** and **workflow execution**, below.
## What makes Conductor workflows different
Conductor workflows stand apart from traditional orchestration approaches in several key ways:
- **Durable execution** — Workflows survive process failures, restarts, and infrastructure outages. Conductor persists state at every step, so a long-running workflow or async workflow picks up exactly where it left off — even after days or weeks.
- **JSON-native definitions** — Every workflow is a JSON workflow definition you can store in version control, diff across releases, and generate programmatically. No compiled DSL or proprietary format required.
- **Dynamic workflows** — Workflows can be created and modified at runtime as code-first or JSON definitions, enabling use cases where the task graph is not known ahead of time (for example, when the number of parallel branches depends on an API response).
- **Versioned** — Each workflow definition carries an explicit version number so you can roll out changes incrementally and run multiple versions side by side.
- **Language-agnostic** — Workers that execute tasks can be written in any language — Java, Python, Go, JavaScript, C#, or Clojure — and deployed anywhere. The workflow definition itself is decoupled from implementation.
## Workflow definition
The workflow definition describes the flow and behavior of your business logic. Think of it as a blueprint specifying how it should execute at runtime until it reaches a terminal state. The workflow definition includes:
- The workflow's input/output keys.
- A collection of [task configurations](tasks.md#task-configuration) that specify the task conditions, sequence, and data flow until the workflow is completed.
- The workflow's runtime behavior, such as the timeout policy and compensation flow.
### Example JSON workflow definition
Below is a realistic three-task workflow that fetches data from an API, transforms it with an inline script, and then delegates the result to a worker task for further processing.
```json
{
"name": "process_order",
"description": "Fetch order details, enrich them, and hand off to fulfillment",
"version": 1,
"schemaVersion": 2,
"ownerEmail": "team-platform@example.com",
"timeoutPolicy": "ALERT_ONLY",
"timeoutSeconds": 3600,
"restartable": true,
"failureWorkflow": "handle_order_failure",
"inputParameters": ["orderId"],
"outputParameters": {
"enrichedOrder": "${enrich_order.output.result}",
"fulfillmentStatus": "${fulfill_order.output.status}"
},
"tasks": [
{
"name": "fetch_order",
"taskReferenceName": "fetch_order",
"type": "HTTP",
"inputParameters": {
"http_request": {
"uri": "https://api.example.com/orders/${workflow.input.orderId}",
"method": "GET",
"connectionTimeOut": 5000,
"readTimeOut": 5000
}
}
},
{
"name": "enrich_order",
"taskReferenceName": "enrich_order",
"type": "INLINE",
"inputParameters": {
"order": "${fetch_order.output.response.body}",
"evaluatorType": "graaljs",
"expression": "(function() { var o = $.order; o.region = o.country === 'US' ? 'domestic' : 'international'; return o; })()"
}
},
{
"name": "fulfill_order",
"taskReferenceName": "fulfill_order",
"type": "SIMPLE",
"inputParameters": {
"enrichedOrder": "${enrich_order.output.result}"
}
}
]
}
```
### Workflow definition parameters
| Parameter | Type | Description |
|---|---|---|
| **name** | `string` | A unique name identifying the workflow. Used when starting executions. |
| **version** | `integer` | The version of the workflow definition. Allows multiple versions to coexist. |
| **tasks** | `array[object]` | An ordered list of [task configurations](tasks.md#task-configuration) that define the workflow's execution graph. |
| **inputParameters** | `array[string]` | List of input keys the workflow expects when triggered. |
| **outputParameters** | `object` | Mapping of output keys to expressions that extract values from task outputs. |
| **failureWorkflow** | `string` | Name of a workflow to trigger when this workflow transitions to FAILED. Useful for compensation or alerting. |
| **timeoutPolicy** | `string` | Policy to apply when the workflow exceeds `timeoutSeconds`. Supported values: `TIME_OUT_WF` (fail the workflow) or `ALERT_ONLY` (mark timed out but keep running). |
| **timeoutSeconds** | `integer` | Maximum time (in seconds) the workflow is allowed to run before the timeout policy is applied. Set to `0` for no timeout. |
| **restartable** | `boolean` | Whether the workflow can be restarted after completion or failure. Defaults to `true`. |
| **ownerEmail** | `string` | Email address of the workflow owner. Used for notifications and audit tracking. |
| **schemaVersion** | `integer` | Schema version of the workflow definition format. Current version is `2`. |
## Workflow execution
A workflow execution is the execution instance of a workflow definition.
Whenever a workflow definition is invoked with a given input, a new workflow execution with a unique ID is created. The workflow is governed by a defined state (like RUNNING or COMPLETED), which makes it intuitive to track the workflow.
### Workflow execution states
Each workflow execution transitions through a set of well-defined states:
| State | Description |
|---|---|
| **RUNNING** | The workflow is actively executing tasks. |
| **COMPLETED** | All tasks finished successfully and the workflow reached its terminal state. |
| **FAILED** | One or more tasks failed and the workflow could not recover. If a `failureWorkflow` is configured, it will be triggered. |
| **TIMED_OUT** | The workflow exceeded its configured `timeoutSeconds` and the `timeoutPolicy` was set to `TIME_OUT_WF`. |
| **TERMINATED** | The workflow was explicitly stopped by an API call or system action. |
| **PAUSED** | The workflow has been paused and will not schedule new tasks until resumed. |
The following diagram illustrates how a workflow transitions between states:
```mermaid
stateDiagram-v2
[*] --> RUNNING
RUNNING --> COMPLETED : all tasks succeed
RUNNING --> FAILED : task failure (unrecoverable)
RUNNING --> TIMED_OUT : timeout exceeded
RUNNING --> TERMINATED : API termination
RUNNING --> PAUSED : pause requested
PAUSED --> RUNNING : resume requested
PAUSED --> TERMINATED : API termination
FAILED --> RUNNING : retry
TIMED_OUT --> RUNNING : retry
TERMINATED --> RUNNING : restart (if restartable)
COMPLETED --> [*]
FAILED --> [*]
TIMED_OUT --> [*]
TERMINATED --> [*]
```
## Next steps
- [Tasks](tasks.md) — Learn about the building blocks that make up a workflow, including system tasks, worker tasks, and operators.
- [Workers](workers.md) — Understand how to implement task workers in any programming language.
- [Handling errors](../how-tos/Workflows/handling-errors.md) — Configure retries, failure workflows, and compensation strategies.
# Creating / Updating Task Definitions
A [task definition](../../../documentation/configuration/taskdef.md) specifies a task's general implementation details:
- Timeout policy
- Retry logic
- Rate limit and execution limit
- Input/output keys
- Input template
This definition applies to all instances of the task across workflows.
You can create task definitions using the Conductor UI, CLI, or APIs for the following scenarios:
- **Worker tasks**: all worker tasks (`SIMPLE`) must be registered to the Conductor server as a task definition before they can execute in a workflow.
- **System tasks**: system tasks don't require a task definition, but you can create one with the same name to customize retry, timeout, and rate limit behavior.
## Using Conductor UI
With the UI, you can create or update task definitions visually.
### Creating task definitions
**To create a task definition:**
1. In the left navigation, open **Definitions** and select **Task**.
2. Select **Define task**.
3. Configure the task in the **Task** form, or open the **Code** tab to edit the JSON directly. Refer to [Task Definitions](../../../documentation/configuration/taskdef.md) for the full parameters.
4. Select **Save**.
### Updating task definitions
**To update a task definition:**
1. In the left navigation, open **Definitions** and select **Task**, then select the task definition to be updated.
2. Modify the task in the **Task** form or the **Code** tab. Refer to [Task Definitions](../../../documentation/configuration/taskdef.md) for the full parameters.
3. Select **Save**.
## Using the CLI
Save your task definition to a JSON file and run:
```bash
conductor task create taskdef.json
```
The file can contain a single task definition object or an array of them. To update an existing definition, edit the file and run:
```bash
conductor task update taskdef.json
```
Refer to [Task Definitions](../../../documentation/configuration/taskdef.md) for a reference guide on the full parameters.
## Using APIs
Refer to [Task Definitions](../../../documentation/configuration/taskdef.md) for a reference guide on the full parameters.
### Creating task definitions
You can also create task definitions using the Create Task Definition API (`POST /api/metadata/taskdefs`). The API accepts an array of task definitions, allowing you to create them in bulk.
??? note "Example using cURL"
```shell
curl 'http://localhost:8080/api/metadata/taskdefs' \
-H 'accept: */*' \
-H 'content-type: application/json' \
--data-raw '[{"name":"sample_task_name_1","description":"This is a sample task for demo","responseTimeoutSeconds":10,"timeoutSeconds":30,"inputKeys":[],"outputKeys":[],"timeoutPolicy":"TIME_OUT_WF","retryCount":3,"retryLogic":"FIXED","retryDelaySeconds":5,"inputTemplate":{},"rateLimitPerFrequency":0,"rateLimitFrequencyInSeconds":1}]'
```
### Updating task definitions
You can update task definitions using the Update Task Definition API (`PUT /api/metadata/taskdefs`). This API can only be used to update a single task definition at a time.
??? note "Example using cURL"
```shell
curl 'http://localhost:8080/api/metadata/taskdefs' \
-X 'PUT' \
-H 'accept: */*' \
-H 'content-type: application/json' \
--data-raw '{"name":"sample_task_name_1","description":"This is a sample task for demo","responseTimeoutSeconds":10,"timeoutSeconds":30,"inputKeys":[],"outputKeys":[],"timeoutPolicy":"TIME_OUT_WF","retryCount":3,"retryLogic":"FIXED","retryDelaySeconds":5,"inputTemplate":{},"rateLimitPerFrequency":0,"rateLimitFrequencyInSeconds":1}'
```
## Using SDKs
Every [client SDK](../../../documentation/clientsdks/index.md) includes metadata-client methods that call the same create and update endpoints. Use them when task registration belongs in your application or deployment code rather than in a manual step.
## Reusing tasks
Once a task is defined in Conductor, it can be reused numerous times:
- **In the same workflow** — use the same task with different task reference names.
- **Across workflows** — any workflow can reference any registered task definition.
When reusing tasks in a multi-tenant system, all work assigned to a task goes into the same queue by default. If a noisy neighbor causes polling delays, you can scale up the number of workers or use [task-to-domain](../../../documentation/api/taskdomains.md) to route task load into separate queues.
# Wiring Task Inputs
In Conductor, task inputs can be provided in the workflow definition in multiple ways:
- As a hard-coded value –
```
"taskInputA": true
```
- As a dynamic reference to the workflow inputs, workflow variables, or the inputs/outputs of prior tasks –
```
"taskInputA": "${workflow.input.someValue}
```
## Syntax for dynamic references
All dynamic references are formatted as the following expression:
```
"${type.jsonpath}"
```
These dynamic references are formatted as dot-notation expressions, taking after [JSONPath syntax](https://goessner.net/articles/JsonPath/).
| Component | Description |
| -------------------- | ----------------------------------------------------------------------------------------------------- |
| `${...}` | The root notation indicating that the variable will be dynamically replaced at runtime. |
| type | The type of reference. Supported values:
**workflow**—Refers to the current workflow instance.
**workflow.input**—Refers to the workflow’s input parameters.
**workflow.output**—Refers to the workflow’s output parameters.
**workflow.variables**—Refers to the workflow variables set in the workflow using the [Set Variable](../../../documentation/configuration/workflowdef/operators/set-variable-task.md) task.
**_taskReferenceName_**—Refers to a task in the current workflow instance by its reference name. (For example, “http_ref”).
**_taskReferenceName_.input**—Refers to the task’s input parameters.
**_taskReferenceName_.output**—Refers to the task’s output parameters.
|
| jsonpath | The [JSONPath](https://goessner.net/articles/JsonPath/) expression in dot-notation. |
### Sample expressions
Here is a non-exhaustive list of dynamic references you can use:
- To reference a task’s input payload –
```
${.input}
```
- To reference a task’s output payload –
```
${.output}
```
- To reference a task’s input parameter –
```
${.input.}
```
- To reference a task’s output parameter –
```
${.output.}
```
- To reference the workflow's input payload –
```
${workflow.input}
```
- To reference the workflow's output payload –
```
${workflow.output}
```
- To reference the workflow's input parameter –
```
${workflow.input.}
```
- To reference the workflow's output parameter –
```
${workflow.output.}
```
- To reference the workflow's current status (RUNNING, PAUSED, TIMED_OUT, TERMINATED, FAILED, or COMPLETED) –
```
${workflow.status}
```
- To reference the workflow's (execution) ID –
```
${workflow.workflowId}
```
- (Used in sub-workflows) To reference the parent workflow (execution) ID –
```
${workflow.parentWorkflowId}
```
- (Used in sub-workflows) To reference the task execution ID for the Sub Workflow task in the parent workflow –
```
${workflow.parentWorkflowTaskId}
```
- To reference the workflow's name –
```
${workflow.workflowType}
```
- To reference the workflow's version –
```
${workflow.version}
```
- To reference the start time of the workflow execution –
```
${workflow.createTime}
```
- To reference the workflow's correlation ID –
```
${workflow.correlationId}
```
- To reference the workflow’s domain name that was invoked during its execution –
```
${workflow.taskToDomain.}
```
- To reference the workflow's variable created using the Set Variable task –
```
${workflow.variables.}
```
## Examples
Here are some examples for using dynamic references in workflows.
Referencing workflow inputs
For the given workflow input:
```json
{
"userID": 1,
"userName": "SAMPLE",
"userDetails": {
"country": "nestedValue",
"age": 50
}
}
```
You can reference these workflow inputs elsewhere using the following expressions:
```json
{
"user": "${workflow.input.userName}",
"userAge": "${workflow.input.userDetails.age}"
}
```
At runtime, the parameters will be:
```json
{
"user": "SAMPLE",
"userAge": 50
}
```
Referencing other task outputs
If a task previousTaskReference produced the following output:
```json
{
"taxZone": "A",
"productDetails": {
"nestedKey1": "outputValue-1",
"nestedKey2": "outputValue-2"
}
}
```
You can reference these task outputs elsewhere using the following expressions:
```json
{
"nextTaskInput1": "${previousTaskReference.output.taxZone}",
"nextTaskInput2": "${previousTaskReference.output.productDetails.nestedKey1}"
}
```
At runtime, the parameters will be:
```json
{
"nextTaskInput1": "A",
"nextTaskInput2": "outputValue-1"
}
```
Referencing workflow variables
If a workflow variable is set using the Set Variable task:
```json
{
"name": "Ipsum"
}
```
The variable can be referenced in the same workflow using the following expression:
```json
{
"user": "${workflow.variables.name}"
}
```
Note: Workflow variables cannot be re-referenced across workflows, even between a parent workflow and a sub-workflow.
Referencing data between parent workflow and sub-workflow
To pass parameters from a parent workflow into its sub-workflow, you must declare them as input parameters for the Sub Workflow task. If needed, these inputs can then be set as workflow variables within the sub-workflow definition itself using a Set Variable task.
```
// parent workflow definition with task configuration
{
"createTime": 1733980872607,
"updateTime": 0,
"name": "testParent",
"description": "workflow with subworkflow",
"version": 1,
"tasks": [
{
"name": "get_item",
"taskReferenceName": "get_item_ref",
"inputParameters": {
"uri": "https://example.com/api",
"method": "GET",
"accept": "application/json",
"contentType": "application/json",
"encode": true
},
"type": "HTTP",
},
{
"name": "sub_workflow",
"taskReferenceName": "sub_workflow_ref",
"inputParameters": {
"user": "${workflow.variables.name}",
"item": "${previous_task_ref.output.item[0]}"
},
"type": "SUB_WORKFLOW",
"subWorkflowParam": {
"name": "testSub",
"version": 1
}
}
],
"inputParameters": [],
"outputParameters": {}
}
```
To pass parameters from a sub-workflow back to its parent workflow, you must pass them as the sub-workflow’s output parameters in the sub-workflow definition.
```
// sub-workflow definition
{
"createTime": 1726651838873,
"updateTime": 1733983507294,
"name": "testSub",
"description": "subworkflow for parent workflow",
"version": 1,
"tasks": [
{
"name": "get-user",
"taskReferenceName": "get-user_ref",
"inputParameters": {
"uri": "https://example.com/api",
"method": "GET",
"accept": "application/json",
"contentType": "application/json",
"encode": true
},
"type": "HTTP",
},
{
"name": "send-notification",
"taskReferenceName": "send-notification_ref",
"inputParameters": {
"uri": "https://example.com/api",
"method": "GET",
"accept": "application/json",
"contentType": "application/json",
"encode": true
},
"type": "HTTP",
}
],
"inputParameters": [],
"outputParameters": {
"location": "${get-user_ref.output.response.body.results[0].location.country}",
"isNotif": "${send-notification_ref.output}"
}
}
```
In the parent workflow, these sub-workflow outputs can be referenced using the expression format `${.output.}`.
## Troubleshooting
You can verify if the data was passed correctly by checking the input/output values of the task execution in the UI. Common errors:
- If the reference expression is incorrectly formatted, the referencing parameter value may end up with the wrong data or a null value.
- If the referenced value (such as a task output) has not resolved at the point when it is referenced, the referencing parameter value will be null.
# Choosing Tasks
Tasks are the building blocks of Conductor workflows. In this guide, familiarise yourself with the tasks available in Conductor OSS and the differences between each of them.
## Built-in tasks
Built-in tasks allow you to easily run common tasks on the Conductor server without needing to build and deploy your own task workers. Here is an introduction of the built-in tasks available in Conductor:
* **[System tasks](../../../documentation/configuration/workflowdef/systemtasks/index.md)** common tasks that allow you to get started quickly without needing custom workers.
* **[Operators](../../../documentation/configuration/workflowdef/operators/index.md)** enable you to declaratively design the workflow's control flow and logic with minimal code required.
### System tasks
Here are the system tasks available in Conductor OSS for common use:
| System Task | Description |
| :-------------------- | :----------------------------------- |
| [Event](../../../documentation/configuration/workflowdef/systemtasks/event-task.md) | Publish events to an external eventing system (AMQP, SQS, Kafka, and so on). |
| [HTTP](../../../documentation/configuration/workflowdef/systemtasks/http-task.md) | Call an API or HTTP endpoint. |
| [Human](../../../documentation/configuration/workflowdef/systemtasks/human-task.md) | Wait for an external signal. |
| [Inline](../../../documentation/configuration/workflowdef/systemtasks/inline-task.md) | Execute lightweight JavaScript code inline. |
| [No Op](../../../documentation/configuration/workflowdef/systemtasks/noop-task.md) | Do nothing. |
| [JSON JQ Transform](../../../documentation/configuration/workflowdef/systemtasks/json-jq-transform-task.md) | Clean or transform JSON data using jq. |
| [Kafka Publish](../../../documentation/configuration/workflowdef/systemtasks/kafka-publish-task.md) | Publish messages to Kafka. |
| [Wait](../../../documentation/configuration/workflowdef/systemtasks/wait-task.md) | Wait until a set time or duration has passed. |
### Operators
Here are the operators available in Conductor OSS for managing the flow of execution:
| Operator | Description |
| -------------------------- | ----------------------------------------- |
| [Do While](../../../documentation/configuration/workflowdef/operators/do-while-task.md) | Execute tasks repeatedly, like a _do…while…_ statement. |
| [Dynamic](../../../documentation/configuration/workflowdef/operators/dynamic-task.md) | Execute a task dynamically, like a function pointer. |
| [Dynamic Fork](../../../documentation/configuration/workflowdef/operators/dynamic-fork-task.md) | Execute a dynamic number of tasks in parallel. |
| [Fork](../../../documentation/configuration/workflowdef/operators/fork-task.md) | Execute a static number of tasks in parallel. |
| [Join](../../../documentation/configuration/workflowdef/operators/join-task.md) | Join the forks after a Fork or Dynamic Fork before proceeding to the next task. |
| [Set Variable](../../../documentation/configuration/workflowdef/operators/set-variable-task.md) | Create or update workflow variables. |
| [Start Workflow](../../../documentation/configuration/workflowdef/operators/start-workflow-task.md) | Asynchronously start another workflow, like an entry point. |
| [Sub Workflow](../../../documentation/configuration/workflowdef/operators/sub-workflow-task.md) | Synchronously start another workflow, like a subroutine. |
| [Switch](../../../documentation/configuration/workflowdef/operators/switch-task.md) | Execute tasks conditionally, like an _if…else…_ statement. |
| [Terminate](../../../documentation/configuration/workflowdef/operators/terminate-task.md) | Terminate the current workflow, like a _return_ statement. |
## Custom tasks
If you need to implement custom logic beyond the scope of Conductor's system tasks, you can use Worker (`SIMPLE`) tasks instead. Unlike a built-in task, a Worker task requires setting up a worker outside the Conductor environment that polls for and executes the task.
## Task comparison
To help you decide on which tasks to use, here is a detailed comparison of similar tasks available in Conductor.
### Inline vs Worker tasks
The [Inline task](../../../documentation/configuration/workflowdef/systemtasks/inline-task.md) is used to execute custom JavaScript code directly within the workflow. It’s ideal for lightweight operations like **simple data transformations, conditional checks, or small calculations**. Because the code executes within the Conductor JVM, Inline tasks benefit from low latency, no network overhead, and easier debugging. However, it also has limitations on using other languages, custom libraries, frameworks, or stacks.
The Worker task is handled by external task workers that execute a custom function or service
is an external custom function or service that performs a specific task in a workflow. Written in any language of choice (Python, Java, etc), it can execute **complex business logic, custom algorithms, or long-running operations**. Worker tasks run outside the Conductor server, meaning they require additional infrastructure set-up and logging mechanisms.
### Event vs Kafka Publish tasks
If you only need to publish messages to a Kafka topic for external services to use, the [Kafka Publish](../../../documentation/configuration/workflowdef/systemtasks/kafka-publish-task.md) task is simpler to set up.
In contrast, the [Event](../../../documentation/configuration/workflowdef/systemtasks/event-task.md) task supports more involved set-ups, such as using events to start a Conductor workflow, or having Conductor consume messages. It also supports a wider range of event brokers across AMQP, NATS, SQS, Kafka, and Conductor's own internal queue.
### Wait vs Human tasks
The [Wait](../../../documentation/configuration/workflowdef/systemtasks/wait-task.md) task and [Human](../../../documentation/configuration/workflowdef/systemtasks/human-task.md) task both support waiting until a specific condition is met. Use the Wait task for cases when the workflow needs to wait for specific wait duration or timestamp, and use the Human task when the workflow needs to wait for an external trigger.
### Start Workflow vs Sub Workflow tasks
Both [Start Workflow](../../../documentation/configuration/workflowdef/operators/start-workflow-task.md) and [Sub Workflow](../../../documentation/configuration/workflowdef/operators/sub-workflow-task.md) tasks are useful for starting another workflow within a workflow. However, the Start Workflow task starts another workflow and proceeds to the next task without waiting for the started workflow to complete, while the Sub Workflow task will wait for the subworkflow to reach terminal state before proceeding to the next task.
The Sub Workflow task provides a tighter coupling between the parent workflow and the subworkflow. This is useful for cases when you need to associate workflow progress and states, or if you need to pass the output of the subworkflow back into the parent workflow.
### Fork vs Dynamic Fork tasks
Both [Fork](../../../documentation/configuration/workflowdef/operators/fork-task.md) and [Dynamic Fork](../../../documentation/configuration/workflowdef/operators/dynamic-fork-task.md) facilitate parallel execution of tasks. The Fork task executes a predetermined number of forks, while the Dynamic Fork executes a variable number of forks at runtime.
If each fork must run a different set of tasks, it is best to use the Fork task, because Dynamic Forks can only run the same task for all its forks.
### Dynamic vs Switch tasks
Both the [Switch](../../../documentation/configuration/workflowdef/operators/switch-task.md) task and the [Dynamic](../../../documentation/configuration/workflowdef/operators/dynamic-task.md) task are useful in situations when the specific task to run is determined only at runtime. Using the Switch task allows you to easily predefine and set the specific conditions for each switch case, while using the Dynamic task allows to to mark a dynamic point in the workflow without having to pre-set all the case options into the workflow definition beforehand.
In the workflow diagram, the Dynamic task will produce a more simplified view, as it will only display the selected task. Meanwhile, the Switch task will produce a more comprehensive view that shows all possible paths that the workflow could have taken.
Here are some scenarios for deciding between a Dynamic task and a Switch task:
| Scenario | Task to Use |
| -------------------------- | ----------------------------------------- |
| You have a huge number of case options or the specific case options are not yet determined. | Dynamic |
| You need a default case option. | Switch |
| Each case option involves multiple tasks. | Switch |
| The conditions for each switch case is relatively straightforward. | Switch |
| The conditions for each switch case is constantly changing, or requires more complicated logic. | Dynamic |
If you opt for the Dynamic task, you must set up the control flow for how the task to run will be determined at runtime. For example, using a preceding task that must pass the task name into the Dynamic task.
# Workers
A **worker** is responsible for executing a task in a workflow. Each type of worker implements the core functionality of each task, handling the logic as defined in its code.
System task workers are managed by Conductor within its JVM, while `SIMPLE` task workers are to be implemented by yourself. These workers can be implemented in any programming language of your choice (Python, Java, JavaScript, C#, Go, and Clojure) and hosted anywhere outside the Conductor environment.
!!! Note
Conductor provides a set of worker frameworks in its SDKs. These frameworks come with comes with features like polling threads, metrics, and server communication, making it easy to create custom workers.
These workers communicate with the Conductor server via REST/gRPC, allowing them to poll for tasks and update the task status. Learn more in [Architecture](../architecture/index.md).
## How workers work
1. **Poll** — The worker polls the Conductor server for tasks of a specific type.
2. **Execute** — The worker receives a task, executes the business logic, and produces an output.
3. **Report** — The worker reports the task result (COMPLETED or FAILED) back to the server.
Conductor handles scheduling, retries, and state persistence. Your worker just focuses on business logic.
## Worker configuration
Workers are configured through the task definition on the Conductor server. Key settings:
| Parameter | Description |
| :--- | :--- |
| `retryCount` | Number of times Conductor retries a failed task. |
| `retryDelaySeconds` | Delay between retries. |
| `responseTimeoutSeconds` | Max time for a worker to respond after polling. |
| `timeoutSeconds` | Overall SLA for task completion. |
| `pollTimeoutSeconds` | Max time for a worker to poll before timeout. |
| `rateLimitPerFrequency` | Max task executions per frequency window. |
| `concurrentExecLimit` | Max concurrent executions across all workers. |
See [Task Definitions](../../documentation/configuration/taskdef.md) for the full reference.
## Scaling task workers
Workers can be scaled independently of the Conductor server:
- **Horizontal scaling** — Run multiple instances of the same worker. Conductor distributes tasks across all polling workers automatically.
- **Rate limiting** — Use `rateLimitPerFrequency` to control throughput per task type.
- **Concurrency limits** — Use `concurrentExecLimit` to cap parallel executions.
- **Domain isolation** — Use [task domains](../../documentation/api/taskdomains.md) to route tasks to specific worker groups.
See [Scaling Workers](../how-tos/Workers/scaling-workers.md) for detailed guidance.
# Your First Workflow & Worker
**Outcome:** a `greetings` workflow that queues a `greet` task and returns `Hello Conductor` from a worker.
**Time:** about 5 minutes.
Complete [Connect to Conductor](connect.md) first. This guide uses the SDK connection variables configured there: `CONDUCTOR_SERVER_URL`, plus `CONDUCTOR_AUTH_KEY` and `CONDUCTOR_AUTH_SECRET` when your server requires them.
## How a worker runs
In this quickstart you build two things: a **workflow** named `greetings` — the durable definition that Conductor executes — and a **worker** — a function in your code that performs one task inside it.
The workflow has a single task of type `SIMPLE`, which means the work is done by your code rather than by one of Conductor's built-in tasks. Every `SIMPLE` task has a task type — here, `greet`. When a running workflow reaches that task, Conductor places it on a queue for that task type. Your worker polls the `greet` queue, runs your business logic, and reports back `COMPLETED` or `FAILED`. Conductor durably persists the result, then advances the workflow to its next task.
Two rules follow from this design:
- The task type must match exactly between the workflow definition and the worker — otherwise the task sits on a queue that nothing polls.
- Workers run as ordinary processes in your own infrastructure and deploy and scale independently of the Conductor server. Conductor guarantees at-least-once delivery, meaning the same task can be delivered again after a failure or timeout — so write workers to be idempotent, where running the same task twice produces the same result.
```mermaid
flowchart LR
subgraph server["Conductor server"]
wf["greetings workflow"] --> task["greet task (SIMPLE)"]
end
queue[["greet queue"]]
subgraph worker["Your worker"]
fn["greet(name) your business logic"]
end
task -- "queues by task type" --> queue
fn -- "polls" --> queue
fn -- "reports COMPLETED / FAILED Conductor persists result, advances workflow" --> task
```
## Language-specific quickstart
Choose a language to reveal one complete `greet` worker and the matching `greetings` workflow. The examples are adapted from the maintained SDK hello-world worker examples.
Choose a language to reveal its install, worker, workflow, and run steps.
1. Install Python support
```bash
pip install conductor-python
```
2. Save the worker and workflow app
Save as `quickstart.py`:
```python
from conductor.client.automator.task_handler import TaskHandler
from conductor.client.configuration.configuration import Configuration
from conductor.client.orkes_clients import OrkesClients
from conductor.client.workflow.conductor_workflow import ConductorWorkflow
from conductor.client.worker.worker_task import worker_task
@worker_task(task_definition_name="greet", register_task_def=True)
def greet(name: str) -> dict:
return {"result": f"Hello {name}"}
def main():
config = Configuration()
clients = OrkesClients(configuration=config)
executor = clients.get_workflow_executor()
workflow = ConductorWorkflow(name="greetings", version=1, executor=executor)
greet_task = greet(task_ref_name="greet_ref", name=workflow.input("name"))
workflow >> greet_task
workflow.output_parameters({"result": greet_task.output("result")})
workflow.register(overwrite=True)
with TaskHandler(configuration=config, scan_for_annotated_workers=True) as handler:
handler.start_processes()
run = executor.execute(name="greetings", version=1, workflow_input={"name": "Conductor"})
print(run.output["result"])
if __name__ == "__main__":
main()
```
3. Run and verify
```bash
python quickstart.py
# Hello Conductor
```
See the [Python SDK guide](../documentation/clientsdks/python-sdk.md) for worker configuration and production patterns.
1. Install Java support
Add the SDK dependency to your Gradle project:
```groovy
dependencies {
implementation 'org.conductoross:conductor-client:5.0.1'
}
```
2. Save the worker and workflow app
Save as `Main.java`:
```java
import com.netflix.conductor.client.automator.TaskRunnerConfigurer;
import com.netflix.conductor.client.http.ConductorClient;
import com.netflix.conductor.client.http.TaskClient;
import com.netflix.conductor.client.http.WorkflowClient;
import com.netflix.conductor.client.worker.Worker;
import com.netflix.conductor.common.metadata.tasks.Task;
import com.netflix.conductor.common.metadata.tasks.TaskResult;
import com.netflix.conductor.sdk.workflow.def.ConductorWorkflow;
import com.netflix.conductor.sdk.workflow.def.tasks.SimpleTask;
import com.netflix.conductor.sdk.workflow.executor.WorkflowExecutor;
import java.util.List;
import java.util.Map;
class GreetWorker implements Worker {
@Override
public String getTaskDefName() {
return "greet";
}
@Override
public TaskResult execute(Task task) {
String name = (String) task.getInputData().get("name");
TaskResult result = new TaskResult(task);
result.setStatus(TaskResult.Status.COMPLETED);
result.addOutputData("result", "Hello " + name);
return result;
}
}
public class Main {
public static void main(String[] args) {
String serverUrl = System.getenv().getOrDefault(
"CONDUCTOR_SERVER_URL", "http://localhost:8080/api");
ConductorClient client = ConductorClient.builder().basePath(serverUrl).build();
WorkflowExecutor executor = new WorkflowExecutor(client);
ConductorWorkflow workflow = new ConductorWorkflow<>(executor);
workflow.setName("greetings");
workflow.setVersion(1);
SimpleTask greetTask = new SimpleTask("greet", "greet_ref");
greetTask.input("name", "${workflow.input.name}");
workflow.add(greetTask);
workflow.registerWorkflow(true, true);
TaskClient taskClient = new TaskClient(client);
new TaskRunnerConfigurer.Builder(taskClient, List.of(new GreetWorker()))
.withThreadCount(10)
.build()
.init();
WorkflowClient workflowClient = new WorkflowClient(client);
String workflowId = workflowClient.startWorkflow(
"greetings", 1, "", Map.of("name", "Conductor"));
System.out.println("Started workflow: " + workflowId);
}
}
```
3. Run and verify
Run the class with your Gradle application task, then inspect the completed `greet_ref` task in the `greetings` execution. Its output is:
```text
Hello Conductor
```
See the [Java SDK guide](../documentation/clientsdks/java-sdk.md) for complete imports and worker configuration.
Save as `quickstart.ts`:
```typescript
import {
OrkesClients,
ConductorWorkflow,
TaskHandler,
worker,
simpleTask,
} from "@io-orkes/conductor-javascript";
import type { Task } from "@io-orkes/conductor-javascript";
@worker({ taskDefName: "greet" })
async function greet(task: Task) {
return {
status: "COMPLETED" as const,
outputData: { result: `Hello ${task.inputData.name}` },
};
}
async function main() {
const clients = await OrkesClients.from();
const executor = clients.getWorkflowClient();
const workflow = new ConductorWorkflow(executor, "greetings")
.add(simpleTask("greet_ref", "greet", { name: "${workflow.input.name}" }))
.outputParameters({ result: "${greet_ref.output.result}" });
await workflow.register();
const handler = new TaskHandler({ client: clients.getClient(), scanForDecorated: true });
await handler.startWorkers();
const run = await workflow.execute({ name: "Conductor" });
console.log(run.output?.result);
await handler.stopWorkers();
}
main();
```
3. Run and verify
```bash
npx ts-node quickstart.ts
# Hello Conductor
```
See the [JavaScript SDK guide](../documentation/clientsdks/js-sdk.md) for TypeScript 5 decorators, worker health, and production configuration.
1. Install C# support
```bash
dotnet add package conductor-csharp
```
2. Save and start the worker
Save as `GreetWorker.cs`:
```csharp
using Conductor.Client.Extensions;
using Conductor.Client.Interfaces;
using Conductor.Client.Models;
using Conductor.Client.Worker;
using Task = Conductor.Client.Models.Task;
public class GreetWorker : IWorkflowTask
{
public string TaskType => "greet";
public WorkflowTaskExecutorConfiguration WorkerSettings { get; } = new();
public async Task Execute(Task task, CancellationToken token)
{
var result = task.Completed();
result.OutputData = new Dictionary
{
["result"] = $"Hello {task.InputData["name"]}"
};
return await System.Threading.Tasks.Task.FromResult(result);
}
public TaskResult Execute(Task task) => throw new NotImplementedException();
}
```
Start the worker with the SDK's maintained worker-host pattern in `Program.cs`:
```csharp
using Conductor.Client;
using Conductor.Client.Authentication;
using Conductor.Client.Worker;
using Microsoft.Extensions.Logging;
var configuration = new Configuration
{
BasePath = Environment.GetEnvironmentVariable("CONDUCTOR_SERVER_URL"),
AuthenticationSettings = new OrkesAuthenticationSettings(
Environment.GetEnvironmentVariable("CONDUCTOR_AUTH_KEY"),
Environment.GetEnvironmentVariable("CONDUCTOR_AUTH_SECRET"))
};
var host = WorkflowTaskHost.CreateWorkerHost(
configuration, LogLevel.Information, new GreetWorker());
await host.StartAsync(CancellationToken.None);
await Task.Delay(Timeout.Infinite);
```
In a second terminal, save this as `greetings.json`, then register and run it:
```json
{
"name": "greetings",
"description": "Return a greeting from a C# worker.",
"version": 1,
"schemaVersion": 2,
"tasks": [{
"name": "greet",
"taskReferenceName": "greet_ref",
"type": "SIMPLE",
"inputParameters": { "name": "${workflow.input.name}" }
}],
"outputParameters": { "result": "${greet_ref.output.result}" }
}
```
3. Run and verify
```bash
dotnet run
# In the second terminal:
conductor workflow create greetings.json
conductor workflow start -w greetings -i '{"name":"Conductor"}' --sync
# result: Hello Conductor
```
See the [C# SDK guide](../documentation/clientsdks/csharp-sdk.md) for the maintained examples and SDK reference.
1. Create a Rust app and add the SDK
```bash
cargo new greetings-worker
cd greetings-worker
```
In `Cargo.toml`, add the SDK and async runtime under `[dependencies]`:
```toml
[dependencies]
conductor = { version = "0.1", package = "conductor-sdk", features = ["macros"] }
conductor-macros = "0.1"
tokio = { version = "1", features = ["full"] }
```
```bash
cargo run
# result: Some("Hello Conductor")
```
See the maintained [Rust SDK quickstart](https://github.com/conductor-oss/rust-sdk#60-second-quickstart) for worker configuration, metrics, and production patterns.
## Verify durable execution
1. Open the Conductor UI (`http://localhost:8080` for the local server) and go to **Executions → Workflow** in the left navigation. Click the newest `greetings` execution — the completed `greet_ref` task in the timeline shows `result: Hello Conductor`.
2. Now watch durability at work. Your quickstart app exited after printing, so no worker is running. Start another execution with the CLI alone:
```bash
conductor workflow start -w greetings -i '{"name":"Conductor"}'
```
3. Refresh the executions list: the new run is `RUNNING` and `greet_ref` is `SCHEDULED` — durably queued, waiting for a worker. Nothing is lost.
4. Run your quickstart app again. The worker polls, the waiting task completes, and the execution finishes with `result: Hello Conductor`.
**Troubleshooting**
- `greet_ref` stays `SCHEDULED` even with the app running: the worker is not polling the `greet` task type — confirm the worker is running and its task type is exactly `greet`.
- Registration says the definition already exists: bump the version or update the local test definition.
- `greet_ref` is `FAILED`: inspect the task's input, output, and failure reason in the UI, fix the worker, and start a new execution.
## Keep learning
**Next:** [Run your first agent](first-agent.md) — the same durable execution model, applied to an LLM-powered agent.
Prefer no code? [Run a workflow from JSON](first-workflow.md) registers a two-step workflow with the CLI alone. The [SDKs landing page](../documentation/clientsdks/index.md) links to Go, Ruby, Rust, and the language-specific reference material and production guidance for every supported SDK.
# Validate and test workflows
Use three layers. Schema validation catches an invalid definition, mocked workflow testing checks orchestration decisions, and a real execution verifies workers and integrations.
## Prerequisites
- A reachable Conductor server.
- A workflow definition saved as `workflow.json`.
- Real workers and external dependencies only for the final execution layer.
## 1. Validate the definition
Validation checks metadata and graph rules but does not prove that a worker is polling or an external endpoint is reachable.
```bash
curl -i -X POST 'http://localhost:8080/api/metadata/workflow/validate' \
-H 'Content-Type: application/json' \
--data-binary @workflow.json
```
Success is an empty `200 OK` response. Fix validation errors before registration.
## 2. Test orchestration with mocked tasks
`POST /api/workflow/test` executes the decision logic with task outputs supplied by reference name. Each reference maps to a list because loops or retries can consume multiple mocks.
```json
{
"name": "input_param_demo_workflow",
"version": 1,
"input": {
"_scheduledTime": 1760000000000,
"_executedTime": 1760000000100
},
"workflowDef": {
"name": "input_param_demo_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "compute_report_window",
"taskReferenceName": "compute_report_window",
"type": "INLINE",
"inputParameters": {
"scheduledTime": "${workflow.input._scheduledTime}",
"executionTime": "${workflow.input._executedTime}",
"evaluatorType": "javascript",
"expression": "({scheduledAt: $.scheduledTime, triggeredAt: $.executionTime})"
}
}
],
"outputParameters": {
"scheduledAt": "${compute_report_window.output.result.scheduledAt}"
}
},
"taskRefToMockOutput": {
"compute_report_window": [
{
"status": "COMPLETED",
"output": {
"result": {
"scheduledAt": 1760000000000,
"triggeredAt": 1760000000100
}
}
}
]
}
}
```
```bash
curl -sS -X POST 'http://localhost:8080/api/workflow/test' \
-H 'Content-Type: application/json' \
--data-binary @workflow-test.json
```
Success is a simulated execution whose task states and workflow output match the expected branch. `executionTime` and `queueWaitTime` on a mock can exercise timeout behavior. Nested `SUB_WORKFLOW` tests use `subWorkflowTestRequest`.
## 3. Run the real boundaries
Register the definition, start it, and inspect the returned workflow ID.
```bash
conductor workflow create workflow.json
conductor workflow start -w order_workflow -i '{"orderId":"order-123"}'
conductor workflow get-execution -c
```
Success is a terminal status you expect and verified task output. A `SIMPLE` task without a registered task definition and polling worker remains queued; mock testing cannot detect that deployment gap.
## Limitations
Mock testing does not call workers, brokers, databases, or HTTP endpoints and cannot establish their authentication, latency, or retry behavior. Keep a real integration or smoke test for each production boundary.
Next, add reliability policies with [Reliability and error handling](handling-errors.md) and rehearse recovery with [Debug and recover](debugging-workflows.md).
# Start workflows
Starting a workflow creates a durable execution and returns a workflow ID. Preserve that ID: it is the primary key for status, tasks, logs, and recovery.
## Prerequisites
- The workflow definition is registered.
- Every `SIMPLE` task has a task definition and a running worker.
- The CLI or selected SDK is configured for the same server.
## Start with the CLI
Use asynchronous start for long-running work:
```bash
conductor workflow start -w sample_workflow -i '{"service":"fedex"}'
```
Pin a version and attach a business correlation ID when repeatability and lookup matter:
```bash
conductor workflow start -w sample_workflow --version 2 \
--correlation order-123 -i '{"service":"fedex"}'
```
For a bounded test, `--sync` waits for the execution result:
```bash
conductor workflow start -w sample_workflow -i '{"service":"fedex"}' --sync
```
Success is a returned workflow ID for an asynchronous start, or a workflow result with the expected status for a synchronous start.
## Start with REST
`POST /api/workflow/{name}` accepts the workflow input map directly and returns the workflow ID as text.
```bash
curl -sS -X POST 'http://localhost:8080/api/workflow/sample_workflow' \
-H 'Content-Type: application/json' \
--data '{"service":"fedex"}'
```
Use `POST /api/workflow` with a `StartWorkflowRequest` when you need fields such as `version`, `correlationId`, `priority`, or `taskToDomain`. Use `POST /api/workflow/execute/{name}/{version}` only when the caller should wait synchronously. The [Start Workflow API](../../../documentation/api/startworkflow.md) owns the complete request and response reference.
## Start with an SDK
These examples show the start call after client configuration. Use the SDK reference linked below each tab for dependency and authentication setup.
=== "Java"
```java
StartWorkflowRequest request = new StartWorkflowRequest();
request.setName("sample_workflow");
request.setVersion(2);
request.setCorrelationId("order-123");
request.setInput(Map.of("service", "fedex"));
String workflowId = clients.getWorkflowClient().startWorkflow(request);
```
See the [Java SDK](../../../documentation/clientsdks/java-sdk.md).
=== "Python"
```python
from conductor.client.http.models import StartWorkflowRequest
request = StartWorkflowRequest(
name="sample_workflow",
version=2,
correlation_id="order-123",
input={"service": "fedex"},
)
workflow_id = executor.start_workflow(request)
```
See the [Python SDK](../../../documentation/clientsdks/python-sdk.md).
=== "TypeScript"
```typescript
const workflowId = await workflowClient.startWorkflow({
name: "sample_workflow",
version: 2,
correlationId: "order-123",
input: { service: "fedex" },
});
```
See the [JavaScript and TypeScript SDK](../../../documentation/clientsdks/js-sdk.md).
=== "Go"
```go
workflowID, err := workflowExecutor.StartWorkflow(&model.StartWorkflowRequest{
Name: "sample_workflow",
Version: 2,
CorrelationId: "order-123",
Input: map[string]string{
"service": "fedex",
},
})
if err != nil {
return err
}
```
See the [Go SDK](../../../documentation/clientsdks/go-sdk.md).
## Inspect the execution
```bash
conductor workflow get-execution -c
```
Confirm the workflow name and version, input, current status, and each task status. Submission alone is not proof that a worker or integration completed.
## Limitations
- Synchronous execution keeps the client waiting and is a poor fit for human tasks, timers, and long-running workers.
- Omitting `version` selects the server's latest registered version; pin it when callers require repeatable behavior.
- A correlation ID helps lookup but is not necessarily unique and is not a substitute for the workflow ID.
Next, learn how to [view executions](viewing-workflow-executions.md) or [choose an automatic trigger](choosing-a-trigger.md).
# Viewing Workflow Executions
Use the workflow ID returned at start time to inspect the exact execution.
## Inspect with the CLI
```bash
conductor workflow status
conductor workflow get-execution -c
```
The compact execution view should show the workflow name/version, current status, input/output, and every task attempt. For API automation, use `GET /api/workflow/{workflowId}?includeTasks=true`; the [Workflow API](../../../documentation/api/workflow.md) owns the response contract.
Success means the execution's identity, status, and task state match the run you intended to inspect. For failures, record the failed task's `reasonForIncompletion`, retry count, and worker ID before recovery.
## Inspect with the UI
The Conductor UI presents the same durable execution as a diagram and timeline. You can open it:
- In **[Executions](http://localhost:8080/executions)**, after [searching for workflows](searching-workflows.md).
- In **[Workbench](http://localhost:8080/workbench)** > **Execution History**
**To view a workflow execution:**
In **[Executions](http://localhost:8080/executions)** or **[Workbench](http://localhost:8080/workbench)**, select the Workflow ID hyperlink.
## Workflow execution details
The following tabs are available for each workflow execution:
| Tab Name | Description |
|----------------------------|-------------------------------------------------------------------------------------------------------------------|
| **Tasks** > **Diagram** | Visual diagram of the workflow and its tasks. |
| **Tasks** > **Task List** | List of the task executions in this workflow, including details like the task name, task ID, status, and so on. |
| **Tasks** > **Timeline** | Timeline showcasing the duration and sequence of each task in the workflow. |
| **Summary** | Summary view of the workflow execution, which includes the workflow ID, status, duration, and so on. |
| **Workflow Input/Output** | View of the JSON payload for the workflow inputs, outputs, and variables. |
| **JSON** | View of the full workflow execution JSON, including all tasks, inputs, outputs, and so on. |
### Workflow diagram view
In **Tasks** > **Diagram**, you can view the workflow's exact execution path. The executed paths are shown in green and while other alternative paths are greyed out.

Each task status will also be clearly marked, highlighting any task errors.

### Task execution details
You can also view a task's execution details by selecting a task from the following tabs:
- **Tasks** > **Diagram**
- **Tasks** > **Task List**
- **Tasks** > **Timeline**
This action opens a left-side panel that contains the following tabs:
| Tab Name | Description |
|------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
| **Summary** | Summary view of the task execution, which includes the task execution ID, status, duration, and so |
| **Input** | View of the JSON payload for the task inputs. |
| **Output** | View of the JSON payload for the task outputs. |
| **Logs** | View of the log messages logged by the task, if any. |
| **JSON** | View of the full task execution JSON, including retry count, start time, worker ID, and so on. |
| **Definition** | View of the task configuration used when executing the task. |
## Limitations and next step
The execution view reports what Conductor persisted; detailed application logs remain in the worker's logging system unless the worker added task logs. Continue with [Search executions](searching-workflows.md) when the workflow ID is unknown, or [Debug and recover](debugging-workflows.md) for a failed run.
# Choose a workflow trigger
Choose the mechanism whose owner can make the start or resume decision reliably.
| Need | Use | Result |
|---|---|---|
| A request should create work now | [Direct start](starting-workflows.md) | A new workflow execution |
| Time or cadence should create work | [Schedule](scheduling-workflows.md) | A new execution at each cron slot |
| A broker message should create work | [Event handler](../../../documentation/configuration/eventhandlers.md) | A new execution for a matching event |
| A parent workflow owns the dependency | `SUB_WORKFLOW` or `START_WORKFLOW` | A child execution, waited for or fire-and-forget |
| An external result should resume existing work | Task signal or event-handler `complete_task`/`fail_task` | The identified task changes state |
## Decision procedure
1. Decide whether the action creates a new execution or resumes one that already exists.
2. If it creates work, identify the owner: application request, clock, message, or parent workflow.
3. If it resumes work, retain the task ID or workflow ID and task reference name when the task begins waiting.
4. Define an idempotency key or stable message ID before enabling retries or broker redelivery.
5. Verify the observable result: a returned workflow ID for a start, or the expected task status and downstream transition for a resume.
## Limitations
- Schedules have no native overlap policy; executions can overlap.
- Event actions are concurrent and not atomic; one can succeed while another fails.
- An OSS event handler cannot resolve a business correlation key to a waiting task.
- A signal changes existing work; it does not create a new workflow.
Next, implement the selected route with [Start workflows](starting-workflows.md), [Schedule workflows](scheduling-workflows.md), or [Event orchestration](../event-bus.md).
# Schedule workflows
A schedule creates a new workflow execution at each matching cron slot. Use it when the clock owns the decision to run; use [event orchestration](../event-bus.md) when a message owns that decision.
## Prerequisites
- The target workflow definition is registered.
- The scheduler is enabled on the server and its persistence module is configured.
- Workers required by the target workflow are running.
- The Conductor CLI is configured for simple CRUD, or REST is available for the complete scheduler model.
## Create a simple schedule
The canonical fixture runs once per minute in UTC:
```json
{
"name": "every-minute-demo-schedule",
"cronExpression": "0 * * * * *",
"zoneId": "UTC",
"startWorkflowRequest": {
"name": "daily_report_workflow",
"version": 1,
"input": {}
},
"runCatchupScheduleInstances": false,
"paused": false
}
```
Create it with the CLI:
```bash
conductor schedule create scheduler/examples/every-minute-schedule.json
conductor schedule get every-minute-demo-schedule
```
Success is a saved schedule with a non-null `nextRunTime`, followed by a workflow execution after the next slot. Use REST for multi-expression cron schedules, bounds, catchup behavior, preview, and execution-history search; CLI releases do not expose every scheduler field or operation consistently.
## Use the complete REST interface
```bash
curl -sS -X POST 'http://localhost:8080/api/scheduler/schedules' \
-H 'Content-Type: application/json' \
--data-binary @scheduler/examples/every-minute-schedule.json
```
The same `POST` creates or updates by schedule name and returns `200 OK` with the stored schedule. See the [Scheduler API](../../../documentation/api/scheduler.md) for exact bodies, query parameters, and status codes.
## Cron and timezone behavior
Conductor uses Spring six-field cron expressions: second, minute, hour, day of month, month, and day of week. Macros such as `@daily` are also accepted by Spring's parser.
The legacy single-expression form uses `cronExpression` plus `zoneId` (default `UTC`). The multi-expression form uses `cronSchedules`; when that array is non-empty it takes precedence over the legacy fields, and each entry has its own `zoneId` defaulting to UTC.
```json
{
"name": "regional-report",
"cronSchedules": [
{"cronExpression": "0 0 9 * * MON-FRI", "zoneId": "America/New_York"},
{"cronExpression": "0 0 9 * * MON-FRI", "zoneId": "Europe/London"}
],
"startWorkflowRequest": {
"name": "daily_report_workflow",
"version": 1
}
}
```
Cron evaluation follows the selected IANA timezone, including daylight-saving transitions. A local time that does not exist during a spring-forward transition is skipped by the cron engine; repeated local times follow the engine's next-instant calculation. Test business-sensitive schedules around DST boundaries.
The preview endpoint accepts no timezone parameter. It evaluates in `conductor.scheduler.schedulerTimeZone` (UTC by default), not a schedule's `zoneId`, and returns at most five times even if `limit` is larger.
## Catch up and bound execution
`runCatchupScheduleInstances: true` advances through missed cron slots after downtime. It can create a burst, so the workflow and dependencies must be idempotent and capacity-aware. With the default `false`, the scheduler advances from current time rather than replaying every missed slot.
Use `scheduleStartTime` and `scheduleEndTime` as epoch-millisecond inclusive bounds. A schedule outside its window stops producing new runs; it is not deleted automatically.
## Inputs added by the scheduler
The scheduler copies `startWorkflowRequest.input`, then adds these values to every execution:
| Input | Meaning |
|---|---|
| `_startedByScheduler` | Schedule name |
| `_scheduledTime` | Intended cron slot, epoch milliseconds |
| `_executedTime` | Actual dispatch time, epoch milliseconds |
| `_executionId` | Unique scheduler execution-record ID |
| `_schedulerCron` | Cron expression and zone that produced this run |
Use `${workflow.input._executionId}` when a downstream system needs per-run identity. `startWorkflowRequest.correlationId` is copied literally; the scheduler does **not** interpolate `${scheduledTime}` or other templates in it. If every workflow execution needs a unique correlation ID, derive it in the workflow from injected input or start the workflow through code that constructs the ID.
## Operate schedules
```bash
conductor schedule list
conductor schedule pause every-minute-demo-schedule
conductor schedule resume every-minute-demo-schedule
conductor schedule delete every-minute-demo-schedule
```
REST also supports filtering, search, a pause reason, and scheduled-execution history:
```bash
curl 'http://localhost:8080/api/scheduler/schedules/search?paused=false&size=20'
curl 'http://localhost:8080/api/scheduler/search/executions?freeText=every-minute-demo-schedule&size=20'
```
After pausing, verify the stored `paused` state and confirm no new execution appears after a cron slot. After resuming, confirm a new scheduled execution and inspect all five injected fields.
## Limitations
- There is no native overlap policy. If a prior workflow is still running, the next slot can start another execution.
- There is no scheduler endpoint for "run now" or manual backfill. Start the target workflow directly for an ad hoc run and pass the intended window explicitly.
- Preview is single-cron, capped at five, and uses the server scheduler timezone.
- Java, Python, TypeScript, and Go SDKs can call the REST surface through generated or low-level clients, but this repository does not define a consistent high-level scheduler API across all SDKs. Treat REST as the portable complete interface.
- `correlationId` is literal, not a schedule template.
For runnable catchup, bounded, concurrency, input, retry, and multi-step variants, use the [scheduled workflow recipes](../../cookbook/workflow-scheduling.md), which reuse `scheduler/examples/`.
# Event-Driven Orchestration
Event-driven orchestration connects workflows to the messages around them. A workflow can publish to a broker, an incoming message or webhook can start or advance workflows, and a signal can resume one specific execution that is waiting. Each page in this section covers one of those directions, and the table below routes you to the right one.
| Need | Start here | Availability |
|---|---|---|
| Publish workflow data to a queue or broker | [Publish events](publish-events.md) | OSS and Orkes |
| Consume a broker message and start or update workflow work | [Consume and route events](consume-route-events.md) | OSS and Orkes |
| Receive an HTTP callback from an external service | [Incoming webhooks](incoming-webhooks.md) | Orkes only |
| Continue a workflow blocked on `WAIT` | [Send signals to workflows](../cookbook/sending-signals.md) | OSS and Orkes |
| Notify external systems when executions change state | [Workflow status events](workflow-status-events.md) | OSS and Orkes |
`EVENT` publishes messages; an event handler consumes and routes them. A webhook is HTTP ingress, not a general-purpose event handler. A signal changes an existing workflow and does not create a new execution.
## Broker provider matrix
Provider support depends on the Conductor distribution and enabled server integration. The destination after the first colon in an event name is provider-specific.
| Provider | OSS Conductor | Orkes |
|---|:---:|:---:|
| Conductor internal queue | Yes | — |
| Kafka | Yes | Yes |
| Amazon SQS | Yes | Yes |
| NATS | Yes | Yes |
| NATS JetStream | Yes | — |
| NATS Streaming | Yes | — |
| AMQP queue / exchange | Yes | Yes (including RabbitMQ) |
| Azure Service Bus | — | Yes |
| Google Cloud Pub/Sub | — | Yes |
| IBM MQ | — | Yes |
## Operate the whole path
Monitor broker queue depth (`event_queue_depth`), message processing (`event_queue_messages_processed`, `event_queue_messages_handled`, and `event_queue_messages_error`), and handler actions (`event_execution_success` and `event_execution_error`). Then check the resulting workflow or task: broker acknowledgement alone does not prove the downstream action reached its intended state.
## Next steps
- **[Publish events](publish-events.md)** — send workflow data to a broker.
- **[Consume and route events](consume-route-events.md)** — start or advance workflows from incoming messages.
- **[Incoming webhooks](incoming-webhooks.md)** — accept verified HTTP callbacks.
- **[Send signals](../cookbook/sending-signals.md)** — advance an execution that is waiting.
- **[Workflow status events](workflow-status-events.md)** — notify external systems as executions change state.
# Production path for durable workflows
Use this guide after [your first workflow](../../quickstart/first-workflow.md). It turns a successful local run into a service with an explicit contract, bounded failure behavior, repeatable deployment, and an operating model.
## Outcome
You will have a workflow whose callers know its input and output contract, whose tasks have deliberate reliability settings, and whose operators know how to inspect and recover an execution.
## 1. Define the contract
Treat a workflow definition and its `outputParameters` as an API. Document required inputs, validate or reject invalid requests at the boundary, and keep outputs stable for callers. When a change is not backward compatible, register a new workflow version instead of changing an active definition in place.
Read [workflow definitions](../concepts/workflows.md), [task inputs](../how-tos/Tasks/task-inputs.md), and [workflow versioning](../how-tos/Workflows/versioning-workflows.md) before publishing a caller-facing workflow.
## 2. Make the failure policy explicit
For every external side effect, decide whether it is safe to retry and how it is made idempotent. Set task retry behavior and timeouts deliberately; use a failure workflow or compensation when a later failure requires business rollback. Bound the workflow itself when the business operation has a maximum acceptable duration.
Verify the design by forcing one transient task failure and confirming that the expected retry, timeout, or compensation path is visible in the execution.
Continue with [task timeouts and retries](../cookbook/task-timeouts-and-retries.md), [error handling](../how-tos/Workflows/handling-errors.md), and [best practices](../bestpractices.md).
## 3. Test the real boundaries
Test the registered definition with representative input, not only worker functions in isolation. Cover success, retryable failure, terminal business failure, timeout, and the idempotency behavior of each side effect. Use real dependencies or Testcontainers where practical so queue, persistence, and concurrency behavior is exercised.
**Verification:** start the workflow with a test correlation ID, inspect its full execution, and assert its output contract and terminal status.
## 4. Deploy definitions and workers safely
Deploy worker code and task definitions before routing production traffic to a workflow that needs them. Keep workers idempotent because Conductor delivery is at least once. Roll out a new workflow version, update callers deliberately, and retain the old version until its executions are drained.
Use [creating workflows](../how-tos/Workflows/creating-workflows.md), [scaling workers](../how-tos/Workers/scaling-workers.md), and [deployment](../running/deploy.md) for the implementation details.
## 5. Operate the execution
Give operators a workflow name, version, correlation-ID convention, and owner. Monitor queue depth, task failures, timeouts, and execution status. During an incident, inspect the failed task before retrying; retry only failures that are safe to repeat, then pause, resume, rerun, or terminate according to the business policy.
**Recovery drill:** intentionally leave a workflow waiting or fail a retryable task, then find it through [searching workflows](../how-tos/Workflows/searching-workflows.md) and recover it with the documented [debugging](../how-tos/Workflows/debugging-workflows.md) controls.
## Next production step
For platform-level deployment and storage choices, continue to [Deploy Conductor](../running/deploy.md) and [Durable Execution](../../architecture/durable-execution.md). For an AI workflow or agent, add the controls in [Production Agent Architecture](../ai/production-agent-architecture.md) to this workflow baseline.
# Handling Workflow Errors
In production microservice architectures, failures are inevitable. Conductor provides multiple layers of error handling so you can build resilient, self-healing workflows:
* **Saga pattern** — run a compensation flow to undo completed steps when a workflow fails.
* **Retry strategies** — automatically retry failed tasks with configurable backoff.
* **Task-level error handling** — mark tasks as optional, fail immediately on terminal errors, or set per-task timeouts.
* **Timeout policies** — control what happens when a task or workflow exceeds its time limit.
* **Workflow status listener** — send notifications to external systems on workflow completion or failure.
## Saga pattern: compensation on failure
The saga pattern is a well-established approach for managing distributed transactions across microservices. Instead of a single atomic transaction that spans multiple services, a saga breaks the work into a sequence of local transactions. Each step has a corresponding **compensating action** that undoes its effect. When any step in the sequence fails, the previously completed steps are rolled back in reverse order by executing their compensating actions.
This pattern is essential in microservice architectures where two-phase commits are impractical. Because each service owns its own data, you cannot rely on a traditional database transaction to maintain consistency across services. The saga pattern gives you eventual consistency with explicit rollback logic, making failures predictable and recoverable.
### Configuring a failure workflow
You can configure a workflow to automatically run upon failure by adding the `failureWorkflow` parameter to your main workflow definition.
Additionally, you may also specify the _version_ of it by using the `failureWorkflowVersion` parameter.
```json
"failureWorkflow": "",
"failureWorkflowVersion": 2,
```
If your main workflow fails, Conductor will trigger this failure workflow. By default, the following parameters are passed to the failure workflow as input:
* **`reason`** — The reason for the workflow's failure.
* **`workflowId`** — The failed workflow's execution ID.
* **`failureStatus`** — The failed workflow's status.
* **`failureTaskId`** — The execution ID for the task that failed in the workflow.
* **`failedWorkflow`** — The full workflow execution JSON for the failed workflow.
You can use these parameters to implement compensation actions in the failure workflow, such as notification alerts, resource clean-up, or reversing completed transactions.
### Example: Slack notification on failure
Here is a failure workflow that sends a Slack message when the main workflow fails. It posts the `reason` and `workflowId` so the team can debug the failure:
```json
{
"name": "shipping_failure",
"description": "Notification workflow for shipping workflow failures",
"version": 1,
"tasks": [
{
"name": "slack_message",
"taskReferenceName": "send_slack_message",
"inputParameters": {
"http_request": {
"headers": {
"Content-type": "application/json"
},
"uri": "https://hooks.slack.com/services/<_unique_Slack_generated_key_>",
"method": "POST",
"body": {
"text": "workflow: ${workflow.input.workflowId} failed. ${workflow.input.reason}"
},
"connectionTimeOut": 5000,
"readTimeOut": 5000
}
},
"type": "HTTP",
"retryCount": 3
}
],
"restartable": true,
"workflowStatusListenerEnabled": false,
"ownerEmail": "conductor@example.com",
"timeoutPolicy": "ALERT_ONLY"
}
```
### Example: saga compensation for order processing
A realistic saga implementation involves a main workflow that processes an order through multiple services and a compensation workflow that reverses each completed step if any step fails.
**Main workflow** — `order_processing` processes a customer order through three stages: charge the payment, reserve inventory, and arrange shipping.
```json
{
"name": "order_processing",
"description": "Process a customer order through payment, inventory, and shipping",
"version": 1,
"failureWorkflow": "order_compensation",
"tasks": [
{
"name": "charge_payment",
"taskReferenceName": "charge_payment_ref",
"inputParameters": {
"orderId": "${workflow.input.orderId}",
"customerId": "${workflow.input.customerId}",
"amount": "${workflow.input.totalAmount}"
},
"type": "SIMPLE",
"retryCount": 2,
"retryLogic": "EXPONENTIAL_BACKOFF",
"retryDelaySeconds": 5
},
{
"name": "reserve_inventory",
"taskReferenceName": "reserve_inventory_ref",
"inputParameters": {
"orderId": "${workflow.input.orderId}",
"items": "${workflow.input.items}",
"paymentTransactionId": "${charge_payment_ref.output.transactionId}"
},
"type": "SIMPLE",
"retryCount": 2,
"retryLogic": "FIXED",
"retryDelaySeconds": 3
},
{
"name": "arrange_shipping",
"taskReferenceName": "arrange_shipping_ref",
"inputParameters": {
"orderId": "${workflow.input.orderId}",
"shippingAddress": "${workflow.input.shippingAddress}",
"items": "${workflow.input.items}",
"inventoryReservationId": "${reserve_inventory_ref.output.reservationId}"
},
"type": "SIMPLE",
"retryCount": 1,
"retryLogic": "FIXED",
"retryDelaySeconds": 10
}
],
"restartable": true,
"workflowStatusListenerEnabled": true,
"ownerEmail": "order-team@example.com",
"timeoutPolicy": "TIME_OUT_WF",
"timeoutSeconds": 600
}
```
**Compensation workflow** — `order_compensation` reverses each completed step in reverse order: cancel the shipment, restore inventory, and refund the payment.
```json
{
"name": "order_compensation",
"description": "Undo completed order steps when order_processing fails",
"version": 1,
"tasks": [
{
"name": "cancel_shipment",
"taskReferenceName": "cancel_shipment_ref",
"inputParameters": {
"orderId": "${workflow.input.failedWorkflow.input.orderId}",
"shipmentId": "${workflow.input.failedWorkflow.tasks[arrange_shipping_ref].output.shipmentId}"
},
"type": "SIMPLE",
"optional": true,
"retryCount": 3,
"retryLogic": "FIXED",
"retryDelaySeconds": 5
},
{
"name": "restore_inventory",
"taskReferenceName": "restore_inventory_ref",
"inputParameters": {
"orderId": "${workflow.input.failedWorkflow.input.orderId}",
"reservationId": "${workflow.input.failedWorkflow.tasks[reserve_inventory_ref].output.reservationId}"
},
"type": "SIMPLE",
"optional": true,
"retryCount": 3,
"retryLogic": "FIXED",
"retryDelaySeconds": 5
},
{
"name": "refund_payment",
"taskReferenceName": "refund_payment_ref",
"inputParameters": {
"orderId": "${workflow.input.failedWorkflow.input.orderId}",
"transactionId": "${workflow.input.failedWorkflow.tasks[charge_payment_ref].output.transactionId}",
"amount": "${workflow.input.failedWorkflow.input.totalAmount}"
},
"type": "SIMPLE",
"retryCount": 5,
"retryLogic": "EXPONENTIAL_BACKOFF",
"retryDelaySeconds": 10
}
],
"restartable": true,
"workflowStatusListenerEnabled": false,
"ownerEmail": "order-team@example.com",
"timeoutPolicy": "ALERT_ONLY",
"timeoutSeconds": 1200
}
```
Notice that compensation tasks are marked `optional: true` for steps that may not have completed before the failure occurred. The refund task uses aggressive retries with exponential backoff because it is critical that the customer receives their money back.
## Retry strategies
When a task fails, Conductor can automatically retry it according to the retry logic configured on the task definition. You control the retry behavior with three parameters:
* **`retryCount`** — Maximum number of retry attempts.
* **`retryLogic`** — The backoff strategy between retries.
* **`retryDelaySeconds`** — The base delay between retries, in seconds.
### FIXED
Retries at a constant interval. Every retry waits the same amount of time.
```json
{
"retryCount": 3,
"retryLogic": "FIXED",
"retryDelaySeconds": 5
}
```
This retries up to 3 times, waiting exactly 5 seconds between each attempt.
### EXPONENTIAL_BACKOFF
Each retry waits exponentially longer than the previous one. The delay is calculated as `retryDelaySeconds * 2^(attemptNumber)`. This reduces load on downstream services that may be experiencing pressure.
```json
{
"retryCount": 4,
"retryLogic": "EXPONENTIAL_BACKOFF",
"retryDelaySeconds": 2
}
```
This retries up to 4 times with delays of approximately 2, 4, 8, and 16 seconds.
### LINEAR_BACKOFF
Each retry waits incrementally longer by a fixed amount. The delay is calculated as `retryDelaySeconds * attemptNumber`. This provides a gentler ramp-up than exponential backoff.
```json
{
"retryCount": 4,
"retryLogic": "LINEAR_BACKOFF",
"retryDelaySeconds": 5
}
```
This retries up to 4 times with delays of approximately 5, 10, 15, and 20 seconds.
### Choosing a retry strategy
| Strategy | Delay pattern | Best for |
|---|---|---|
| `FIXED` | Constant (e.g., 5s, 5s, 5s) | Predictable transient failures like brief network blips or short-lived lock contention. |
| `EXPONENTIAL_BACKOFF` | Doubling (e.g., 2s, 4s, 8s, 16s) | Rate-limited APIs, overloaded services, or any case where you want to reduce pressure on a struggling dependency. |
| `LINEAR_BACKOFF` | Incremental (e.g., 5s, 10s, 15s, 20s) | Moderate recovery scenarios where you need longer waits over time but exponential growth would be too aggressive. |
## Task-level error handling
Beyond retries, Conductor provides several task-level controls for managing failures within a running workflow.
### Optional tasks
Setting `optional` to `true` on a task tells Conductor to continue the workflow even if that task fails after exhausting all retries. The workflow will proceed to the next task rather than failing entirely.
```json
{
"name": "send_analytics_event",
"taskReferenceName": "send_analytics_ref",
"type": "SIMPLE",
"optional": true,
"retryCount": 2,
"retryLogic": "FIXED",
"retryDelaySeconds": 3
}
```
Use optional tasks for non-critical side effects like logging, analytics, or notifications where a failure should not block the primary business logic.
### Failing immediately with terminal errors
When a worker encounters an error that no amount of retrying will fix, such as invalid input data or a business rule violation, it should return a `FAILED_WITH_TERMINAL_ERROR` status. This tells Conductor to skip all remaining retries and fail the task immediately.
Workers signal this by setting the task status to `FAILED_WITH_TERMINAL_ERROR` in the task result. This avoids wasting time on retries when the failure is deterministic. For example, if a payment is declined due to insufficient funds, retrying the same charge will never succeed.
### Per-task timeout configuration
You can set timeouts on individual tasks to prevent them from blocking the workflow indefinitely:
```json
{
"name": "call_external_api",
"taskReferenceName": "call_api_ref",
"type": "SIMPLE",
"timeoutSeconds": 120,
"responseTimeoutSeconds": 60,
"timeoutPolicy": "RETRY"
}
```
* **`timeoutSeconds`** — Maximum total time for the task, including all retries.
* **`responseTimeoutSeconds`** — Maximum time to wait for a worker to pick up and respond to the task. If a worker does not update the task within this window, Conductor marks it as timed out.
## Timeout policies
Timeout policies determine what Conductor does when a task exceeds its `timeoutSeconds` or `responseTimeoutSeconds` limit.
### RETRY
Re-queue the task for another attempt. The retry counts against the task's `retryCount`.
```json
{
"timeoutPolicy": "RETRY",
"timeoutSeconds": 60,
"retryCount": 3
}
```
### TIME_OUT_WF
Fail the entire workflow immediately when the task times out. Use this for tasks where a timeout indicates a critical problem that makes continuing the workflow pointless.
```json
{
"timeoutPolicy": "TIME_OUT_WF",
"timeoutSeconds": 300
}
```
### ALERT_ONLY
Log an alert but allow the task to continue running. The task is not terminated or retried. This is useful for long-running tasks where you want visibility into slow execution without interrupting work.
```json
{
"timeoutPolicy": "ALERT_ONLY",
"timeoutSeconds": 600
}
```
### Choosing a timeout policy
| Policy | Behavior on timeout | Best for |
|---|---|---|
| `RETRY` | Retries the task (counts against `retryCount`) | Tasks that may hang due to transient issues like network timeouts or unresponsive workers. |
| `TIME_OUT_WF` | Fails the entire workflow | Critical tasks where a timeout means the workflow cannot produce a valid result. |
| `ALERT_ONLY` | Logs an alert, task keeps running | Long-running or best-effort tasks where you want monitoring without enforcement. |
## Implement a Workflow Status Listener
Using a Workflow Status Listener, you can send a notification to an external system or an event to Conductor's internal queue upon failure. Here is the high-level overview for using a Workflow Status Listener:
1. Set the `workflowStatusListenerEnabled` parameter to true in your main workflow definition:
```json
"workflowStatusListenerEnabled": true,
```
2. Implement the [WorkflowStatusListener interface](https://github.com/conductor-oss/conductor/blob/1be02a711dc20682718c6111c09d2b02ce7edde2/core/src/main/java/com/netflix/conductor/core/listener/WorkflowStatusListener.java#L20) to plug into a custom notification or eventing system upon workflow failure.
# Search executions
Search when you know attributes such as workflow name, status, correlation ID, or time range but not the workflow ID.
## Search with the CLI
```bash
conductor workflow search -w order_processing -s FAILED -c 20
conductor workflow search -s COMPLETED \
--start-time-after "2026-07-01" --start-time-before "2026-07-31"
```
| CLI option | Filters or controls | Example |
|---|---|---|
| `-w`, `--workflow` | Workflow name | `--workflow order_processing` |
| `-s`, `--status` | Execution status | `--status FAILED` |
| `-c`, `--count` | Number of executions returned (maximum 1000) | `--count 20` |
| `--start-time-after` | Executions started after a timestamp | `--start-time-after "2026-07-01"` |
| `--start-time-before` | Executions started before a timestamp | `--start-time-before "2026-07-31"` |
| `--json` | JSON output instead of the table view | `--json` |
| `--csv` | CSV output instead of the table view | `--csv` |
Results should include `workflowId`, name, status, and start time. Use the returned ID with `conductor workflow get-execution -c` before taking a recovery action.
For structured/free-text or task-based searches beyond CLI flags, use `GET /api/workflow/search` or `GET /api/workflow/search-by-tasks`; the [Workflow API](../../../documentation/api/workflow.md#search-workflows) owns the query syntax and pagination contract.
### REST query parameters
`GET /api/workflow/search` accepts the following query parameters:
| Parameter | Meaning | Default |
|---|---|---|
| `start` | Page offset | `0` |
| `size` | Number of results | `100` |
| `sort` | Sort order as `:ASC` or `:DESC` | None |
| `freeText` | Full-text search query | `*` |
| `query` | SQL-like filter expression | None |
| `classifier` | Filter or group agent workflow executions by classifier | None |
| `topLevelOnly` | Limit results to top-level workflow executions | `false` |
## Search with the UI
The UI has two modes:
* **Workflows** tab — Search using workflow parameters.
* **Tasks** tab — Search workflows by tasks.
**To search workflow executions:**
1. Go to **[Executions](http://localhost:8080/executions)** in the Conductor UI.
2. Configure the [search parameters](#search-parameters).
3. Select **Search**.
Once the search results are displayed, you can sort the results by different column values and select additional columns to display.
## Search parameters
Here are the search parameters for each search mode.
### Search by workflows
The following fields are available for searching workflows in the **Workflows** tab.
| Search Field Name | Description |
|-------------------|---------------------------------------------------------------------------------------------------------|
| Workflow Name | Filters workflow executions by its name. |
| Workflow ID | Filters to a specific workflow execution by its execution ID. |
| Status | Filters workflow executions by its status (RUNNING, COMPLETED, FAILED, TIMED_OUT, TERMINATED, PAUSED). |
| Start Time - From | Filters workflow executions that started on or after the specified time. |
| Start Time - To | Filters workflow executions that started on or before the specified time. |
| Lookback (days) | Filters workflow executions that ran in the last given number of days. |
| Lucene-syntax Query (Double-quote strings for Free Text) | (If indexing is enabled) Filters workflow executions by querying workflow input and output values. |
### Search workflows by tasks
The following fields are available for searching workflows by its tasks in the **Tasks** tab.
| Search Field Name | Description |
|--------------------|--------------------------------------------------------------------------------------------------------------|
| Task Name | Filters workflow executions by its task name. |
| Task ID | Filters to a specific workflow execution that contains this task execution ID. |
| Task Status | Filters workflow executions by its task status (IN_PROGRESS, CANCELED, FAILED, FAILED_WITH_TERMINAL_ERROR, COMPLETED, COMPLETED_WITH_ERRORS, SCHEDULED, TIMED_OUT, SKIPPED). |
| Task Type | Filters workflow executions by its task type. |
| Workflow Name | Filters workflow executions by its workflow name. |
| Update Time - From | Filters workflow executions by tasks that started on or after the specified time. |
| Update Time - To | Filters workflow executions by tasks that started on or before the specified time. |
| Lookback (days) | Filters workflow executions by tasks that ran in the last given number of days. |
| Lucene-syntax Query (Double-quote strings for Free Text) | (If indexing is enabled) Filters workflow executions by querying task input and output values. |
## Limitations and next step
Free-text and task searches depend on the configured index backend and its indexing latency. Search results identify candidates; always inspect the execution before retrying, restarting, or terminating it. Continue with [View executions](viewing-workflow-executions.md) or [Debug and recover](debugging-workflows.md).
# Debugging Workflows
The [workflow execution views](viewing-workflow-executions.md) in the Conductor UI are useful for debugging workflow issues. Learn how to debug failed executions and rerun them.
## Debug procedure
Start with the persisted execution:
```bash
conductor workflow get-execution -c
```
Identify the `FAILED`, `TIMED_OUT`, or terminal task and record its `reasonForIncompletion`, input, output, worker ID, and retry count. Fix the underlying worker, dependency, credentials, or definition before changing execution state.
When you view the workflow execution details, the cause of the workflow failure will be stated at the top. Go to the **Tasks > Diagram** tab to quickly identify the failed task, which is marked in red. You can select the failed task to investigate the details of the failure.
The following tab views or fields in the task details are useful for debugging:
| Field or Tab Name | Description |
|-------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------|
| _Reason for Incompletion_ in **Task Detail** > **Summary** | Contains the exception message thrown by the task worker. |
| _Worker_ in **Task Detail** > **Summary** | Contains the worker instance ID where the failure occurred. Useful for digging up detailed logs, if it has not already captured by Conductor. |
| **Task Detail** > **Input** | Useful for verifying if the task inputs were correctly computed and provided to the task. |
| **Task Detail** > **Output** | Useful for verifying what the task produced as output. |
| **Task Detail** > **Logs** | Contains the task logs, if supplied by the task worker. |
| **Task Detail** > **Retried Task - Select an instance** | (If the task has been retried multiple times) Contains all retry attempts in a dropdown list. Each list item contains the task details for a particular attempt. |

## Recovering from failure
Once you have resolved the underlying issue for the execution failure, you can manually restart or retry the failed workflow execution using the Conductor UI or APIs.
Here are the recovery options:
| Recovery Action | Description |
|---------------------|----------------------------|
| Restart with Current Definitions | Restart the workflow from the beginning using the same workflow definition that was used in the original execution. This option is useful if the workflow definition has changed and you want to run the execution instance using the original definition. |
| Restart with Latest Definitions | Restart the workflow from the beginning using the latest workflow definition. This option is useful if changes were made to the workflow definition and you want to run the execution instance with the latest definition. |
| Rerun from a specific task | Re-execute the workflow from a specific task, reusing the outputs of all prior tasks. This option is useful when a task in the middle of the workflow failed and you want to fix and re-run it without re-executing everything before it. |
| Retry - From failed task | Retry the workflow from the last failed task. |
CLI equivalents:
```bash
conductor workflow retry
conductor workflow restart
conductor workflow rerun --task-id
```
After recovery, run `conductor workflow status ` and verify that the expected task is running or the workflow reached the intended terminal status.
!!! Note
You can set tasks to be retried automatically in case of transient failures. Refer to [Task Definition](../../../documentation/configuration/taskdef.md) for more information.
### Using Conductor UI
**To recover from failure**:
1. In the workflow execution details page, select **Actions** in the top right corner.
2. Select one of the following options:
- Restart with Current Definitions
- Restart with Latest Definitions
- Rerun from a specific task
- Retry - From failed task
### Using APIs
You can restart workflow executions using the Restart Workflow API (`POST api/workflow/{workflowId}/restart`) or the Bulk Restart Workflow API (`POST api/workflow/bulk/restart`).
You can rerun a workflow from a specific task using the Rerun Workflow API (`POST api/workflow/{workflowId}/rerun`) with a request body specifying the `reRunFromTaskId`.
Likewise, you can retry workflow executions from the last failed task using the Retry Workflow API (`POST api/workflow/{workflowId}/retry`) or the Bulk Retry Workflow API (`POST api/workflow/bulk/retry`).
All three recovery operations — restart, rerun, and retry — work on workflows in any terminal state (COMPLETED, FAILED, TIMED_OUT, TERMINATED) and are available indefinitely. Conductor preserves the full execution history, so you can replay any workflow even months after the original run.
## Limitations and next step
Recovery can repeat side effects. Retry or rerun only when completed external operations are idempotent or have an explicit compensation policy. Continue with [Reliability and error handling](handling-errors.md) to make transient recovery automatic.
# Task Lifecycle
During a workflow execution, each task transitions through a series of states. Understanding these transitions is key to configuring retries, timeouts, and error handling correctly.
## State diagram
Every task starts in `SCHEDULED` when it enters its queue. A worker poll moves it to `IN_PROGRESS`, and a successful result moves it to `COMPLETED`. The other transitions cover failure: `FAILED` and `TIMED_OUT` tasks return to `SCHEDULED` for retry until their retries are exhausted, and every other state is terminal.
```mermaid
stateDiagram-v2
[*] --> SCHEDULED
SCHEDULED --> IN_PROGRESS : Worker polls task
SCHEDULED --> TIMED_OUT : Poll timeout exceeded
SCHEDULED --> CANCELED : Workflow terminated
IN_PROGRESS --> COMPLETED : Worker reports success
IN_PROGRESS --> FAILED : Worker reports failure
IN_PROGRESS --> FAILED_WITH_TERMINAL_ERROR : Non-retryable failure
IN_PROGRESS --> TIMED_OUT : Response/task timeout exceeded
IN_PROGRESS --> COMPLETED_WITH_ERRORS : Optional task fails
SCHEDULED --> SKIPPED : Skip Task API called
FAILED --> SCHEDULED : Retry (after delay)
TIMED_OUT --> SCHEDULED : Retry (after delay)
COMPLETED --> [*]
FAILED --> [*] : Retries exhausted or totalTimeoutSeconds exceeded
FAILED_WITH_TERMINAL_ERROR --> [*]
TIMED_OUT --> [*] : Retries exhausted or totalTimeoutSeconds exceeded
CANCELED --> [*]
SKIPPED --> [*]
COMPLETED_WITH_ERRORS --> [*]
```
## Task statuses
| Status | Description |
| :--- | :--- |
| `SCHEDULED` | Task is queued and waiting for a worker to poll it. |
| `IN_PROGRESS` | A worker has picked up the task and is executing it. |
| `COMPLETED` | Task completed successfully. |
| `FAILED` | Task failed due to an error. Conductor will retry based on the task definition's retry configuration. |
| `FAILED_WITH_TERMINAL_ERROR` | Task failed with a non-retryable error. No retries will be attempted. |
| `TIMED_OUT` | Task exceeded its configured timeout. Conductor will retry based on the retry configuration. |
| `CANCELED` | Task was canceled because the workflow was terminated. |
| `SKIPPED` | Task was skipped via the Skip Task API. The workflow continues to the next task. |
| `COMPLETED_WITH_ERRORS` | Task failed but is marked as optional in the workflow definition. The workflow continues. |
## Retry behavior
When a task fails with a retryable error, Conductor automatically reschedules it after the configured delay.
```mermaid
sequenceDiagram
participant W as Worker
participant C as Conductor Server
C->>W: Task T1 available for polling
W->>C: Poll task T1
C-->>W: Return T1 (IN_PROGRESS)
W->>W: Process task...
W->>C: Report FAILED (after 10s)
C->>C: Persist failed execution
Note over C: Wait retryDelaySeconds (5s)
C->>C: Schedule new T1 execution
C->>W: T1 available for polling again
W->>C: Poll task T1
C-->>W: Return T1 (IN_PROGRESS)
W->>W: Process task...
W->>C: Report COMPLETED
```
Retry behavior is controlled by the task definition:
| Parameter | Description |
| :--- | :--- |
| `retryCount` | Maximum number of retry attempts. |
| `retryLogic` | `FIXED`, `EXPONENTIAL_BACKOFF`, or `LINEAR_BACKOFF`. See [Retry Logic](../../documentation/configuration/taskdef.md#retry-logic). |
| `retryDelaySeconds` | Base delay between retries. |
| `maxRetryDelaySeconds` | Caps the computed delay. Prevents exponential growth from becoming arbitrarily large. |
| `backoffJitterMs` | Adds random milliseconds to each delay to spread concurrent retries over time. |
| `totalTimeoutSeconds` | Hard wall-clock budget across all attempts. See [Total timeout](#total-timeout). |
## Timeout scenarios
### Poll timeout
If no worker polls the task within `pollTimeoutSeconds`, it is marked as `TIMED_OUT`.
```mermaid
sequenceDiagram
participant W as Worker
participant C as Conductor Server
C->>C: Schedule task T1
Note over C,W: No worker polls within 60s
C->>C: Mark T1 as TIMED_OUT
C->>C: Schedule retry (if retries remain)
```
This typically indicates a backlogged task queue or insufficient workers.
### Response timeout
If a worker polls a task but doesn't report back within `responseTimeoutSeconds`, the task is marked as `TIMED_OUT`. This handles cases where a worker crashes mid-execution.
```mermaid
sequenceDiagram
participant W as Worker
participant C as Conductor Server
C->>W: Task T1 available
W->>C: Poll T1
C-->>W: Return T1 (IN_PROGRESS)
W->>W: Processing...
Note over W: Worker crashes
Note over C: responseTimeoutSeconds (20s) elapsed
C->>C: Mark T1 as TIMED_OUT
Note over C: Wait retryDelaySeconds (5s)
C->>C: Schedule new T1 execution
```
Workers can extend the response timeout by sending `IN_PROGRESS` status updates with a `callbackAfterSeconds` value.
### Task timeout
`timeoutSeconds` is the overall SLA for task completion. Even if a worker keeps sending `IN_PROGRESS` updates, the task is marked as `TIMED_OUT` once this duration is exceeded.
```mermaid
sequenceDiagram
participant W as Worker
participant C as Conductor Server
C->>W: Task T1 available
W->>C: Poll T1
C-->>W: Return T1 (IN_PROGRESS)
W->>W: Processing...
W->>C: IN_PROGRESS (callback: 9s)
Note over C: Task back in queue, invisible 9s
W->>C: Poll T1 again
W->>C: IN_PROGRESS (callback: 9s)
Note over C: Cycle repeats...
Note over C: timeoutSeconds (30s) elapsed
C->>C: Mark T1 as TIMED_OUT
C->>C: Schedule retry (if retries remain)
W->>C: Report COMPLETED (at 32s)
Note over C: Ignored — T1 already terminal
```
### Total timeout
`totalTimeoutSeconds` limits the total wall-clock time across **all** retry attempts. Once this budget is consumed, no further retries are scheduled regardless of how many remain in `retryCount`.
```mermaid
sequenceDiagram
participant W as Worker
participant C as Conductor Server
Note over C: totalTimeoutSeconds = 30s
C->>W: Task T1 (attempt 1)
W->>C: FAILED (at t=5s)
Note over C: Retry delay 5s
C->>W: Task T1 (attempt 2, at t=10s)
W->>C: FAILED (at t=20s)
Note over C: Retry delay 5s
C->>W: Task T1 (attempt 3, at t=25s)
W->>C: FAILED (at t=28s)
Note over C: t=28s ≥ 30s → total budget exhausted
C->>C: Mark workflow FAILED — no more retries
```
This is useful when you need a hard SLA on how long a task can run across all its attempts, independent of how many retries are configured.
## Timeout configuration summary
| Parameter | Description | Default |
| :--- | :--- | :--- |
| `pollTimeoutSeconds` | Max time for a worker to poll the task. | No timeout |
| `responseTimeoutSeconds` | Max time for a worker to respond after polling. | 600s |
| `timeoutSeconds` | SLA per individual attempt (from first `IN_PROGRESS` to terminal). | No timeout |
| `totalTimeoutSeconds` | Hard budget across all attempts combined. Overrides `retryCount`. | No timeout |
| `timeoutPolicy` | Action on timeout: `RETRY`, `TIME_OUT_WF` (fail workflow), or `ALERT_ONLY`. | `TIME_OUT_WF` |
# Managing Workflow Versions
Every workflow definition carries a `version` number, and Conductor can run multiple versions of the same workflow side by side. This page covers when to create a new version, how versions behave at runtime, and how to roll one out without disrupting ongoing executions.
## When to version workflows
Create a new version when inputs, outputs, task order, or failure behavior change in a way callers can observe. See [Update and version safely](creating-workflows.md#updating-workflows) for the registration mechanics.
Versioning is also useful for gradual rollouts. For example, suppose a new version of your core workflow adds a capability that _customerA_ requires, but _customerB_ will not be ready to adopt for another 6 months. With versioning, you can move _customerA_'s traffic to version 2 now while _customerB_ stays on version 1, and migrate _customerB_ later.
## Runtime behavior with multiple workflow versions
At runtime, every execution references a snapshot of the workflow definition taken when it started. Changes to a definition never affect executions that are already running.
Here is an illustration of workflow versions at runtime, when you run workflows based on the latest version, versus when you run workflows based on a specific version.

In the illustration above:
- At T1, an execution starts on version V1, so it uses the V1 definition as it exists at T1.
- At T2, version V2 is registered. New executions that start on the latest version now use V2.
- At T3, the V1 definition itself is updated in place. The execution from T1 keeps running on its T1 snapshot, while any new execution pinned to V1 uses the updated T3 definition.
### Runtime behavior during restarts
By default, restarts, retries, and task reruns also use the snapshot from the start of the first execution attempt. If required, you can instead restart a workflow with the latest definitions.
Here is an illustration of workflow versions at runtime, when you restart workflows using the current definitions versus using the latest definitions.

In the illustration above:
- Restarting the V1 execution with **current definitions** re-runs it on its original T1 snapshot, even after V2 exists and even after V1 is updated at T3.
- Restarting the V1 execution with **latest definitions** re-runs it on the newest registered version, V2.
## Rollout procedure
1. Register the new version instead of overwriting the version production callers use: increment the `version` field in the definition and register it.
```bash
conductor workflow create workflow.json
```
2. Validate and mock-test it, then run a real canary execution with the version pinned:
```bash
conductor workflow start -w --version 2 -i '{"orderId": "test-1"}'
```
3. Move callers, schedules, and parent-workflow references deliberately to the new version.
4. Compare completion, failure, latency, and outputs between the two versions.
5. Keep the previous version registered until callers have migrated and its executions no longer need restart or replay support.
Success means new callers start the intended version while existing executions continue against their recorded definition snapshot.
## Upgrading running workflows
Since definition changes never affect ongoing executions, a running workflow must be explicitly upgraded if required. The upgrade is a terminate followed by a restart on the latest definitions.
!!! warning
Terminating and restarting can repeat side effects. Prefer allowing running executions to finish on their snapshot unless the workflow is idempotent or compensation is defined.
### Using Conductor UI
**To upgrade a running workflow:**
1. In the left navigation, open **Executions** and select **Workflow**, then select the ongoing execution to upgrade.
2. In the top right, select **Actions** and then **Terminate**.
3. Once terminated, select **Actions** and then **Restart with latest definitions**.
### Using Conductor APIs
The API approach upgrades running workflows in bulk. Terminate the executions with the Bulk Terminate API, then restart them with the Bulk Restart API, passing `useLatestDefinitions=true`:
```bash
curl -X POST 'http://localhost:8080/api/workflow/bulk/terminate' \
-H 'Content-Type: application/json' \
-d '["", ""]'
curl -X POST 'http://localhost:8080/api/workflow/bulk/restart?useLatestDefinitions=true' \
-H 'Content-Type: application/json' \
-d '["", ""]'
```
Without `useLatestDefinitions=true`, a restart uses each execution's original definition snapshot and no upgrade happens.
## Limitations and next step
Omitting a version at start time selects the latest registered version, which trades rollout control for convenience. Pin versions in schedules and parent workflows when deterministic deployment matters. Next, rehearse [debugging and recovery](debugging-workflows.md) for both the current and previous version.
# Scaling Task Workers
Workers execute business logic outside the Conductor server. Keeping them healthy requires two things: **monitoring** queue and worker state, and **scaling** based on what the data tells you.
## Monitoring task queues
Conductor tracks queue size and worker poll activity for every task type. Use this data to detect backlogs, stalled workers, and capacity issues.
### Using the UI
Navigate to **Home > Task Queues** (or `/taskQueue`). For each task, the UI shows:
- **Queue Size** — tasks waiting to be picked up.
- **Workers** — count and instance details of workers polling this task.
### Using the CLI
```bash
# List all tasks with queue info
conductor task list
# Get details for a specific task
conductor task get
```
### Using APIs
Get the number of tasks waiting in a queue:
```shell
curl '{{ server_host }}{{ api_prefix }}/tasks/queue/sizes?taskType=' \
-H 'accept: */*'
```
Get worker poll data (which workers are polling, last poll time):
```shell
curl '{{ server_host }}{{ api_prefix }}/tasks/queue/polldata?taskType=' \
-H 'accept: */*'
```
!!! note
Replace `` with your task name.
## Prometheus metrics
Conductor publishes metrics that feed dashboards, alerts, and autoscaling policies. All metrics include `taskType` as a tag so you can monitor per-task.
### Queue depth (Gauge)
```promql
max(task_queue_depth{taskType="my_task"})
```
- Keep queue depth stable. It doesn't need to be zero (especially for long-running tasks), but sustained growth means workers can't keep up.
- Alert on queue depth increasing over a sustained period and use it to trigger autoscaling.
### Task completion rate (Counter)
```promql
rate(task_completed_seconds_count{taskType="my_task"}[$__rate_interval])
```
- Measures throughput — tasks completed per second.
- A sudden drop indicates workers are struggling, failing, or have stopped polling.
- Set a minimum throughput threshold and alert when it drops below.
### Queue wait time
```promql
max(task_queue_wait_time_seconds{quantile="0.99", taskType="my_task"})
```
How long tasks sit in the queue before a worker picks them up. If this is more than a few seconds:
1. **Check worker count** — if all workers are busy, add more instances.
2. **Check polling interval** — reduce it if workers aren't polling frequently enough.
!!! warning
Reducing the polling interval increases API requests to the server. Balance responsiveness against server load.
## Scaling strategies
### When to scale
| Signal | Action |
|---|---|
| Queue depth growing steadily | Add worker instances |
| Queue wait time > 5s at p99 | Add worker instances or reduce polling interval |
| Throughput dropping while queue grows | Investigate worker health (CPU, memory, downstream dependencies) |
| Queue consistently empty, workers idle | Scale down to save resources |
### Horizontal scaling
Add more worker instances. Conductor distributes tasks automatically — every worker polling the same task type competes for work from the same queue. No configuration changes needed on the Conductor server.
### Polling interval tuning
The polling interval controls how frequently workers check for new tasks. Shorter intervals mean lower latency but higher server load.
| Scenario | Recommended interval |
|---|---|
| Latency-sensitive tasks | 100–500ms |
| Standard processing | 1–5s |
| Batch / background work | 5–30s |
### Thread pool sizing
Each worker instance can run multiple polling threads. A good starting point:
```
threads = (task_throughput × avg_task_duration) / num_worker_instances
```
For I/O-bound tasks (HTTP calls, database queries), use more threads than CPU cores. For CPU-bound tasks, match thread count to available cores.
### Rate limiting
If downstream services have rate limits, configure task-level rate limits to prevent workers from overwhelming them:
```json
{
"name": "call_external_api",
"rateLimitPerFrequency": 100,
"rateLimitFrequencyInSeconds": 60
}
```
This limits the task to 100 executions per 60-second window across all workers.
### Domain isolation
Use [task-to-domain](../../../documentation/api/taskdomains.md) to route tasks to specific worker pools. This prevents noisy neighbors — a high-volume workflow won't starve workers serving a latency-sensitive one.
# Architecture Overview
This diagram showcases an overview of Conductor's system architecture:

In Conductor, workflows are executed on a worker-task queue architecture, where each task type (HTTP, Event, Wait, *example_simple_task* and so on) has its own dedicated task queue. The key components of Conductor’s core orchestration engine include:
* **State machine evaluator**—Orchestrates workflows by scheduling tasks to their relevant queues and assigning them to active workers when polled. Monitors each task's state and ensures it is completed, retried, or failed as required.
* **Task queues**—Distributed queues for each task type, where tasks are completed on a first-in-first-out basis.
* **Task workers**—Poll the Conductor server via HTTP or gRPC for tasks, execute tasks, and update the server on the task status. Each worker is responsible for carrying out a specific task type.
* **Data stores** (Redis by default)—High-availability persistence stores that maintain workflow and task metadata, task queues, and execution history
* **APIs**—REST APIs for programmatic access to the Conductor server.
By default, Conductor uses Redis as its data store, with Elasticsearch used for its indexing backend. These [storage layers are pluggable](../../documentation/advanced/extend.md), allowing you to work with alternative backends and queue service providers.
## Task execution
With a worker-task queue architecture, Conductor schedules and assigns tasks to its designated task queues based on its task type. Conductor follows an RPC-based communication model where task workers run on a separate machine from the server and communicate over HTTP-based endpoints with the server.
The workers employ a polling model for managing their designated queues, and update Conductor with the task status.

### Worker-server polling mechanism
Each worker declares beforehand what task(s) it can execute. At runtime, task workers poll its designated task queue(s) to receive and execute scheduled work. Conductor passes task inputs to the worker for execution and collects the task outputs, continuing the process according to the workflow definition.
By default, workers infinitely poll Conductor every 100ms. The polling interval value for each type of worker can be adjusted accordingly based on factors like workload. Here is the polling mechanism in detail:
1. The application starts a workflow execution by interacting with Conductor, which returns a workflow (execution) ID. It can be used to track the workflow's progress and manage its execution.
2. Conductor schedules the first task in the workflow to its task queue.
3. The workers responsible for executing the first task within the workflow are polling Conductor for tasks to execute via HTTP or gRPC. When a task is scheduled, Conductor sends it to the next available worker, which then performs the required work.
4. Periodically, the worker returns the task status to Conductor (e.g. IN PROGRESS, FAILED, COMPLETED, etc).
5. Once the first task in the workflow instance is completed, the worker returns the task output to the server, and Conductor schedules the next set of tasks to be performed.
Conductor manages and maintains the workflow state, keeping track of which tasks have been completed and which are still pending. This ensures that the workflow is executed correctly, with each task triggered precisely at the right time.
Using the workflow ID, the application can check the Conductor server for the workflow status at any time. This is particularly useful for asynchronous or long-running workflows, as it allows the application to monitor the workflow's progress and take appropriate action, such as pausing or terminating the workflow if needed.
# Design Patterns
Design patterns are complete, runnable workflow definitions for common orchestration problems. Each page takes one problem, such as parallel fan-out, sagas, timers, or human approval, and gives you a working definition to register, run, and adapt to your own tasks. This section covers workflow patterns. Agentic patterns and agent recipes live in AI Cookbook.
# Microservice orchestration
### HTTP service chain
A common pattern: call a series of HTTP endpoints where each step uses output from the previous one. No custom workers needed — Conductor handles it with built-in HTTP tasks.
```json
{
"name": "order_processing",
"description": "Validate order, charge payment, reserve inventory, send confirmation",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["orderId", "customerId", "amount", "items"],
"tasks": [
{
"name": "validate_order",
"taskReferenceName": "validate",
"type": "HTTP",
"inputParameters": {
"http_request": {
"uri": "https://api.example.com/orders/${workflow.input.orderId}/validate",
"method": "POST",
"body": {
"customerId": "${workflow.input.customerId}",
"items": "${workflow.input.items}"
},
"connectionTimeOut": 5000,
"readTimeOut": 5000
}
}
},
{
"name": "charge_payment",
"taskReferenceName": "payment",
"type": "HTTP",
"inputParameters": {
"http_request": {
"uri": "https://api.example.com/payments/charge",
"method": "POST",
"body": {
"orderId": "${workflow.input.orderId}",
"amount": "${workflow.input.amount}",
"customerId": "${workflow.input.customerId}"
},
"connectionTimeOut": 10000,
"readTimeOut": 10000
}
}
},
{
"name": "reserve_inventory",
"taskReferenceName": "inventory",
"type": "HTTP",
"inputParameters": {
"http_request": {
"uri": "https://api.example.com/inventory/reserve",
"method": "POST",
"body": {
"orderId": "${workflow.input.orderId}",
"items": "${workflow.input.items}",
"paymentId": "${payment.output.response.body.paymentId}"
},
"connectionTimeOut": 5000,
"readTimeOut": 5000
}
}
},
{
"name": "send_confirmation",
"taskReferenceName": "notify",
"type": "HTTP",
"inputParameters": {
"http_request": {
"uri": "https://api.example.com/notifications/send",
"method": "POST",
"body": {
"customerId": "${workflow.input.customerId}",
"orderId": "${workflow.input.orderId}",
"paymentId": "${payment.output.response.body.paymentId}",
"reservationId": "${inventory.output.response.body.reservationId}"
}
}
}
}
],
"outputParameters": {
"paymentId": "${payment.output.response.body.paymentId}",
"reservationId": "${inventory.output.response.body.reservationId}"
},
"failureWorkflow": "order_compensation",
"timeoutPolicy": "TIME_OUT_WF",
"timeoutSeconds": 120
}
```
Each task passes data forward using `${taskReferenceName.output.response.body.field}` expressions. If any step fails, Conductor retries it (configurable) and can trigger the `failureWorkflow` for compensation.
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @order_processing.json
curl -X POST 'http://localhost:8080/api/workflow/order_processing' \
-H 'Content-Type: application/json' \
-d '{"orderId": "ORD-123", "customerId": "CUST-456", "amount": 99.99, "items": ["SKU-A", "SKU-B"]}'
```
---
### HTTP with conditional branching
Use a SWITCH operator to route workflow execution based on a previous task's output.
```json
{
"name": "user_onboarding",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["userId"],
"tasks": [
{
"name": "get_user_profile",
"taskReferenceName": "profile",
"type": "HTTP",
"inputParameters": {
"http_request": {
"uri": "https://api.example.com/users/${workflow.input.userId}",
"method": "GET"
}
}
},
{
"name": "route_by_tier",
"taskReferenceName": "tier_switch",
"type": "SWITCH",
"evaluatorType": "javascript",
"expression": "$.tier == 'enterprise' ? 'enterprise' : 'standard'",
"inputParameters": {
"tier": "${profile.output.response.body.tier}"
},
"decisionCases": {
"enterprise": [
{
"name": "assign_account_manager",
"taskReferenceName": "assign_am",
"type": "HTTP",
"inputParameters": {
"http_request": {
"uri": "https://api.example.com/account-managers/assign",
"method": "POST",
"body": {"userId": "${workflow.input.userId}"}
}
}
}
],
"standard": [
{
"name": "send_welcome_email",
"taskReferenceName": "welcome",
"type": "HTTP",
"inputParameters": {
"http_request": {
"uri": "https://api.example.com/emails/welcome",
"method": "POST",
"body": {"userId": "${workflow.input.userId}"}
}
}
}
]
}
}
]
}
```
---
### Parallel HTTP calls with Fork/Join
When tasks are independent, run them in parallel with a static fork.
```json
{
"name": "enrich_customer_data",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["customerId"],
"tasks": [
{
"name": "parallel_enrichment",
"taskReferenceName": "fork",
"type": "FORK_JOIN",
"forkTasks": [
[
{
"name": "get_credit_score",
"taskReferenceName": "credit",
"type": "HTTP",
"inputParameters": {
"http_request": {
"uri": "https://api.example.com/credit/${workflow.input.customerId}",
"method": "GET"
}
}
}
],
[
{
"name": "get_purchase_history",
"taskReferenceName": "purchases",
"type": "HTTP",
"inputParameters": {
"http_request": {
"uri": "https://api.example.com/purchases/${workflow.input.customerId}",
"method": "GET"
}
}
}
],
[
{
"name": "get_support_tickets",
"taskReferenceName": "tickets",
"type": "HTTP",
"inputParameters": {
"http_request": {
"uri": "https://api.example.com/support/${workflow.input.customerId}",
"method": "GET"
}
}
}
]
]
},
{
"name": "join_results",
"taskReferenceName": "join",
"type": "JOIN",
"joinOn": ["credit", "purchases", "tickets"]
}
],
"outputParameters": {
"creditScore": "${credit.output.response.body}",
"purchases": "${purchases.output.response.body}",
"tickets": "${tickets.output.response.body}"
}
}
```
All three HTTP calls execute simultaneously. The JOIN waits for all to complete before the workflow continues.
# Dynamic parallelism
### Run different tasks in parallel (Dynamic Fork)
Use `dynamicForkTasksParam` + `dynamicForkTasksInputParamName` when each parallel branch runs a **different** task. The task list is determined at runtime by a preceding step.
```json
{
"name": "dynamic_fork_different_tasks",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "prepare_tasks",
"taskReferenceName": "prepare",
"type": "INLINE",
"inputParameters": {
"evaluatorType": "graaljs",
"expression": "(function() { return { dynamicTasks: [{name: 'HTTP', taskReferenceName: 'fetch_weather', type: 'HTTP'}, {name: 'HTTP', taskReferenceName: 'fetch_news', type: 'HTTP'}], dynamicTasksInput: { fetch_weather: { http_request: {uri: 'https://api.weather.gov/points/39.7456,-104.9994', method: 'GET'}}, fetch_news: { http_request: {uri: 'https://hacker-news.firebaseio.com/v0/topstories.json', method: 'GET'}}}}; })()"
}
},
{
"name": "fork_join_dynamic",
"taskReferenceName": "dynamic_fork",
"type": "FORK_JOIN_DYNAMIC",
"inputParameters": {
"dynamicTasks": "${prepare.output.result.dynamicTasks}",
"dynamicTasksInput": "${prepare.output.result.dynamicTasksInput}"
},
"dynamicForkTasksParam": "dynamicTasks",
"dynamicForkTasksInputParamName": "dynamicTasksInput"
},
{
"name": "join",
"taskReferenceName": "join_ref",
"type": "JOIN"
}
]
}
```
`dynamicTasks` is an array of task definitions (each with `name`, `taskReferenceName`, and `type`). `dynamicTasksInput` is a map keyed by each task's `taskReferenceName` containing its input payload.
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @dynamic_fork_different_tasks.json
curl -X POST 'http://localhost:8080/api/workflow/dynamic_fork_different_tasks' \
-H 'Content-Type: application/json' \
-d '{}'
```
---
### Run same task in parallel (fan-out)
Use `forkTaskName` + `forkTaskInputs` when running the **same** task type across multiple inputs.
```json
{
"name": "fan_out_http_calls",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "fork_join_dynamic",
"taskReferenceName": "parallel_fetch",
"type": "FORK_JOIN_DYNAMIC",
"inputParameters": {
"forkTaskName": "HTTP",
"forkTaskInputs": [
{"http_request": {"uri": "https://jsonplaceholder.typicode.com/posts/1", "method": "GET"}},
{"http_request": {"uri": "https://jsonplaceholder.typicode.com/posts/2", "method": "GET"}},
{"http_request": {"uri": "https://jsonplaceholder.typicode.com/posts/3", "method": "GET"}}
]
}
},
{
"name": "join",
"taskReferenceName": "join_ref",
"type": "JOIN"
}
]
}
```
!!! tip
Conductor injects `__index` into each fork's input so you can track the position of each parallel branch in the results.
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @fan_out_http_calls.json
curl -X POST 'http://localhost:8080/api/workflow/fan_out_http_calls' \
-H 'Content-Type: application/json' \
-d '{}'
```
---
### Run sub-workflows in parallel
Use `forkTaskWorkflow` + `forkTaskInputs` to fan out across instances of another workflow.
```json
{
"name": "parallel_sub_workflows",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "fork_join_dynamic",
"taskReferenceName": "parallel_regions",
"type": "FORK_JOIN_DYNAMIC",
"inputParameters": {
"forkTaskWorkflow": "process_region",
"forkTaskWorkflowVersion": 1,
"forkTaskInputs": [
{"region": "us-east-1", "data": "batch_a"},
{"region": "eu-west-1", "data": "batch_b"},
{"region": "ap-southeast-1", "data": "batch_c"}
]
}
},
{
"name": "join",
"taskReferenceName": "join_ref",
"type": "JOIN"
}
]
}
```
Each element in `forkTaskInputs` spawns one instance of the `process_region` workflow. All results are collected at the JOIN task.
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @parallel_sub_workflows.json
curl -X POST 'http://localhost:8080/api/workflow/parallel_sub_workflows' \
-H 'Content-Type: application/json' \
-d '{}'
```
# Wait and timer patterns
### Wait for a fixed delay
Introduce a delay between workflow steps — useful for rate limiting, cool-down periods, or retry backoff.
```json
{
"name": "delayed_notification",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "process_event",
"taskReferenceName": "process",
"type": "SIMPLE"
},
{
"name": "wait_before_retry",
"taskReferenceName": "cooldown",
"type": "WAIT",
"inputParameters": {
"duration": "5 minutes"
}
},
{
"name": "send_notification",
"taskReferenceName": "notify",
"type": "HTTP",
"inputParameters": {
"uri": "https://api.example.com/notify",
"method": "POST",
"body": {"eventId": "${process.output.eventId}"}
}
}
]
}
```
The `duration` field supports human-readable formats: `30 seconds`, `5 minutes`, `2 hours`, `1 days`, or short forms like `30s`, `5m`, `2h`, `1d`. You can also combine them: `2 hours 30 minutes`.
---
### Wait until a specific time
Schedule workflow continuation for a specific date/time — useful for scheduled releases, SLA deadlines, or business-hours processing.
```json
{
"name": "scheduled_report",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["reportDate"],
"tasks": [
{
"name": "prepare_report",
"taskReferenceName": "prepare",
"type": "SIMPLE"
},
{
"name": "wait_until_publish_time",
"taskReferenceName": "schedule_wait",
"type": "WAIT",
"inputParameters": {
"until": "${workflow.input.reportDate}"
}
},
{
"name": "publish_report",
"taskReferenceName": "publish",
"type": "HTTP",
"inputParameters": {
"uri": "https://api.example.com/reports/publish",
"method": "POST",
"body": {"reportId": "${prepare.output.reportId}"}
}
}
]
}
```
The `until` field supports formats: `yyyy-MM-dd HH:mm z` (e.g., `2025-06-15 09:00 GMT+00:00`), `yyyy-MM-dd HH:mm`, or `yyyy-MM-dd`.
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @scheduled_report.json
curl -X POST 'http://localhost:8080/api/workflow/scheduled_report' \
-H 'Content-Type: application/json' \
-d '{"reportDate": "2025-06-15 09:00 GMT+00:00"}'
```
---
### Wait for an external signal
Pause a workflow until an external system (or human) completes the task via API — useful for approvals, manual QA, or third-party callbacks.
```json
{
"name": "order_with_manual_approval",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["orderId", "amount"],
"tasks": [
{
"name": "validate_order",
"taskReferenceName": "validate",
"type": "HTTP",
"inputParameters": {
"uri": "https://api.example.com/orders/${workflow.input.orderId}/validate",
"method": "GET"
}
},
{
"name": "wait_for_approval",
"taskReferenceName": "approval",
"type": "WAIT"
},
{
"name": "fulfill_order",
"taskReferenceName": "fulfill",
"type": "HTTP",
"inputParameters": {
"uri": "https://api.example.com/orders/${workflow.input.orderId}/fulfill",
"method": "POST",
"body": {
"approvedBy": "${approval.output.approvedBy}"
}
}
}
]
}
```
Complete the WAIT task externally (e.g., from a UI or webhook):
```shell
# Complete the currently blocked wait task and return the updated workflow
curl -X POST 'http://localhost:8080/api/tasks/{workflowId}/COMPLETED/signal/sync' \
-H 'Content-Type: application/json' \
-d '{"approvedBy": "manager@example.com"}'
```
The output data you pass when signaling the current blocked `WAIT` task is available in subsequent tasks via `${approval.output.approvedBy}`. See [Sending signals to workflows](sending-signals.md) for async signaling, return strategies, and timeout behavior.
# Sending signals to workflows
A signal advances a workflow that is already running and waiting. It resolves the first non-terminal WAIT task in the target execution, so the caller only needs the workflow ID. A signal never starts a new execution, cannot target an arbitrary task reference, and does not resolve HUMAN tasks.
## Define a workflow that waits for a signal
This workflow records an approval request, then waits until another system supplies the decision.
```json
{
"name": "order_approval",
"description": "Wait for an external order approval signal",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["orderId"],
"tasks": [
{
"name": "wait_for_approval",
"taskReferenceName": "approval",
"type": "WAIT"
}
],
"outputParameters": {
"orderId": "${workflow.input.orderId}",
"approval": "${approval.output}"
}
}
```
Register it with the workflow metadata API:
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @order_approval.json
```
## Start and wait for the blocking task
The synchronous execution endpoint starts the workflow and waits for a terminal state or a blocked `WAIT` task. `waitForSeconds` defaults to `10`; use `waitUntilTaskRef` when a terminal task reference should also end the wait.
```shell
curl -X POST 'http://localhost:8080/api/workflow/execute/order_approval/1?requestId=approval-demo-42&waitForSeconds=30&returnStrategy=BLOCKING_TASK_INPUT' \
-H 'Content-Type: application/json' \
-d '{"input":{"orderId":"order-42"}}'
```
`returnStrategy` controls the shape of the response:
| Value | Returns |
|---|---|
| `TARGET_WORKFLOW` | The workflow requested by ID. This is the default. |
| `BLOCKING_WORKFLOW` | The workflow that contains the current blocker; it can be a sub-workflow. |
| `BLOCKING_TASK` | The current blocking task. |
| `BLOCKING_TASK_INPUT` | The input of the current blocking task. |
## Signal the wait asynchronously
Use the asynchronous signal endpoint when the caller only needs to submit the decision. It completes the currently blocked `WAIT` task and returns immediately.
```shell
curl -X POST 'http://localhost:8080/api/tasks//COMPLETED/signal' \
-H 'Content-Type: application/json' \
-d '{"approved":true,"approvedBy":"manager@example.com","reason":"Within policy"}'
```
The signal target is the first non-terminal `WAIT` task in the workflow, including a currently running sub-workflow. It does not target `HUMAN` tasks or an arbitrary task reference. A signal does not name a task reference; use this endpoint only when that current blocking-wait behavior is what you want. When exact task targeting is required, use the task-update endpoint (`POST /api/tasks/{workflowId}/{taskRefName}/{status}`) instead.
## Signal and wait for the next workflow state
Use the synchronous variant when the caller needs the resulting workflow state in the same response. It accepts the same `returnStrategy` values and waits up to `timeoutMillis` (default: `5000`).
```shell
curl -X POST 'http://localhost:8080/api/tasks//COMPLETED/signal/sync?returnStrategy=TARGET_WORKFLOW&timeoutMillis=5000' \
-H 'Content-Type: application/json' \
-d '{"approved":true,"approvedBy":"manager@example.com"}'
```
If the workflow reaches another `WAIT` task, the response represents that next blocking state. If it completes first, the response represents the completed workflow. A synchronous signal returns `404` when there is no blocked task to signal; the asynchronous route returns after submitting the signal and does not provide that state in its response.
## Reject or fail the wait
Choose the task status from the URL to record a different decision. For example, signal `FAILED` when an approval is rejected and you want the workflow's failure path to run:
```shell
curl -X POST 'http://localhost:8080/api/tasks//FAILED/signal' \
-H 'Content-Type: application/json' \
-d '{"reason":"Order exceeds the approval limit"}'
```
The payload you send is stored as the `WAIT` task's output. Downstream tasks can reference it with expressions such as `${approval.output.approved}` or `${approval.output.reason}`.
## Next steps
# Task timeouts and retries
Practical recipes for making workers resilient. Each recipe is a complete task definition you can register with `POST /api/metadata/taskdefs`.
---
### Exponential backoff with a cap
Retries with exponential backoff for a task that calls an external API. The cap prevents the delay from growing indefinitely; jitter prevents multiple failing workers from hammering the API at the same time.
```json
{
"name": "call_payment_api",
"ownerEmail": "payments@example.com",
"retryCount": 6,
"retryLogic": "EXPONENTIAL_BACKOFF",
"retryDelaySeconds": 2,
"maxRetryDelaySeconds": 60,
"backoffJitterMs": 3000,
"responseTimeoutSeconds": 30,
"timeoutSeconds": 600,
"timeoutPolicy": "RETRY"
}
```
**Delay schedule** (`retryDelaySeconds=2`, `maxRetryDelaySeconds=60`, `backoffJitterMs=3000`):
| Attempt | Base delay | After cap | Actual range |
| :--- | :--- | :--- | :--- |
| 1 | 2s | 2s | 2.0 – 5.0s |
| 2 | 4s | 4s | 4.0 – 7.0s |
| 3 | 8s | 8s | 8.0 – 11.0s |
| 4 | 16s | 16s | 16.0 – 19.0s |
| 5 | 32s | 32s | 32.0 – 35.0s |
| 6 | 64s | **60s** | 60.0 – 63.0s |
---
### Lease extension for long-running workers
`responseTimeoutSeconds` is the heartbeat window: if the worker doesn't report back within this duration, Conductor marks the task `TIMED_OUT` and retries it. For tasks that take longer than the heartbeat window, workers extend the lease by posting an `IN_PROGRESS` update with `callbackAfterSeconds`.
**Task definition**
```json
{
"name": "transcode_video",
"ownerEmail": "media@example.com",
"retryCount": 2,
"retryLogic": "FIXED",
"retryDelaySeconds": 10,
"responseTimeoutSeconds": 30,
"timeoutSeconds": 3600,
"timeoutPolicy": "RETRY"
}
```
`responseTimeoutSeconds: 30` — Conductor will reschedule the task if the worker is silent for 30 seconds.
`timeoutSeconds: 3600` — the task itself can take up to 1 hour across all heartbeats.
**Worker: extend the lease every 25 seconds**
```python
import time
from conductor.client.http.models import TaskResult
def transcode_video(task):
task_id = task.task_id
workflow_id = task.workflow_instance_id
for chunk in video_chunks(task.input_data["file_url"]):
transcode_chunk(chunk)
# Extend the lease before responseTimeoutSeconds (30s) expires.
# callbackAfterSeconds tells Conductor to leave this task invisible
# in the queue for another 25s — resetting the response clock.
heartbeat = TaskResult(
task_id=task_id,
workflow_instance_id=workflow_id,
status="IN_PROGRESS",
callback_after_seconds=25,
output_data={"progress": chunk.index / len(video_chunks)}
)
conductor_client.update_task(heartbeat)
return TaskResult(
task_id=task_id,
workflow_instance_id=workflow_id,
status="COMPLETED",
output_data={"output_url": upload_result.url}
)
```
**What happens without a heartbeat:**
```
t=0s Worker polls task → IN_PROGRESS
t=30s responseTimeoutSeconds expires → TIMED_OUT → retry scheduled
t=40s Worker finishes (too late, task already terminated)
```
**What happens with a heartbeat every 25s:**
```
t=0s Worker polls task → IN_PROGRESS
t=25s Worker: POST IN_PROGRESS, callbackAfterSeconds=25 → clock resets
t=50s Worker: POST IN_PROGRESS, callbackAfterSeconds=25 → clock resets
...
t=90s Worker: POST COMPLETED → task done
```
---
### Hard SLA with `totalTimeoutSeconds`
Use `totalTimeoutSeconds` when you need a guaranteed upper bound on how long a task can take across all of its retries. This is independent of `retryCount` — whichever limit is hit first wins.
```json
{
"name": "sync_crm_record",
"ownerEmail": "crm@example.com",
"retryCount": 20,
"retryLogic": "FIXED",
"retryDelaySeconds": 5,
"totalTimeoutSeconds": 120,
"responseTimeoutSeconds": 15,
"timeoutPolicy": "TIME_OUT_WF"
}
```
`retryCount: 20` — would normally allow 20 retries.
`totalTimeoutSeconds: 120` — but if the 2-minute wall-clock budget is consumed first, no more retries are queued and the workflow is failed.
This is useful for SLA-sensitive tasks where you need to know that, regardless of transient failures, the workflow will either succeed or surface as failed within a bounded time window.
**Timeline example** (`retryDelaySeconds=5`, `totalTimeoutSeconds=30`):
```
t=0s Attempt 1 → FAILED
t=5s Attempt 2 → FAILED
t=10s Attempt 3 → FAILED
t=15s Attempt 4 → FAILED
t=20s Attempt 5 → FAILED
t=25s Attempt 6 → FAILED
t=30s totalTimeoutSeconds exceeded → workflow FAILED, no more retries
(10 retries still remained in retryCount)
```
---
### Thundering herd prevention
When hundreds of tasks fail simultaneously (e.g., a downstream service goes down), all retries are scheduled at the same time. Without jitter, they all hit the recovering service at once. `backoffJitterMs` spreads them across a time window.
```json
{
"name": "send_webhook",
"ownerEmail": "platform@example.com",
"retryCount": 5,
"retryLogic": "EXPONENTIAL_BACKOFF",
"retryDelaySeconds": 1,
"maxRetryDelaySeconds": 30,
"backoffJitterMs": 5000,
"responseTimeoutSeconds": 10,
"concurrentExecLimit": 200
}
```
With `backoffJitterMs: 5000`, 500 tasks that all fail at `t=0` will retry at uniformly random times between `t=1s` and `t=6s` — spreading the retry load across 5 seconds instead of hitting the service in a single burst.
---
### Choosing the right combination
| Scenario | Recommended config |
| :--- | :--- |
| External API with rate limits | `EXPONENTIAL_BACKOFF` + `maxRetryDelaySeconds` + `backoffJitterMs` |
| Long-running processing job | `responseTimeoutSeconds` (short) + heartbeats from worker + `timeoutSeconds` (long) |
| SLA-bounded task | `totalTimeoutSeconds` + `FIXED` or `EXPONENTIAL_BACKOFF` |
| High fan-out with many concurrent failures | `backoffJitterMs` + `concurrentExecLimit` |
| Non-retryable error | Return `FAILED_WITH_TERMINAL_ERROR` from the worker |
See the [Task Definition reference](../../documentation/configuration/taskdef.md) for all available parameters.
# Event-driven recipes
Read [Event orchestration](../how-tos/event-bus.md) first for action support, provider configuration, delivery, and idempotency semantics.
## Publish an internal event
```json
{
"name": "publish_order_event",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["orderId", "status"],
"tasks": [
{
"name": "publish_order_status",
"taskReferenceName": "publish_order_status",
"type": "EVENT",
"sink": "conductor:order-status",
"inputParameters": {
"orderId": "${workflow.input.orderId}",
"status": "${workflow.input.status}",
"eventVersion": 1
}
}
]
}
```
Register and run the workflow. Its `conductor:order-status` sink expands to `conductor:publish_order_event:order-status`.
## Start a workflow from the event
Register the target workflow first:
```json
{
"name": "fulfill_order",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["orderId", "sourceEventId"],
"tasks": [
{
"name": "record_fulfillment_start",
"taskReferenceName": "record_fulfillment_start",
"type": "SET_VARIABLE",
"inputParameters": {
"orderId": "${workflow.input.orderId}",
"sourceEventId": "${workflow.input.sourceEventId}"
}
}
],
"outputParameters": {
"orderId": "${workflow.input.orderId}"
}
}
```
```json
{
"name": "start_fulfillment_on_order_ready",
"event": "conductor:publish_order_event:order-status",
"condition": "$.status == 'READY'",
"actions": [
{
"action": "start_workflow",
"start_workflow": {
"name": "fulfill_order",
"version": 1,
"correlationId": "${orderId}",
"input": {
"orderId": "${orderId}",
"sourceEventId": "${workflowInstanceId}"
}
}
}
],
"active": true
}
```
```bash
curl -sS -X POST 'http://localhost:8080/api/event' \
-H 'Content-Type: application/json' \
--data-binary @docs/devguide/cookbook/examples/events/start-workflow-handler.json
```
The payload expression is rooted directly at the Event task's published JSON.
## Wait for an external approval
Workflow:
```json
{
"name": "wait_for_order_approval",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["orderId"],
"tasks": [
{
"name": "wait_for_approval",
"taskReferenceName": "approval",
"type": "WAIT"
}
],
"outputParameters": {
"approval": "${approval.output}"
}
}
```
Handler:
```json
{
"name": "complete_order_approval",
"event": "kafka:order-approvals",
"condition": "$.approved == true",
"actions": [
{
"action": "complete_task",
"complete_task": {
"workflowId": "${workflowId}",
"taskRefName": "approval",
"output": {
"approved": "${approved}",
"approvedBy": "${approvedBy}",
"eventId": "${eventId}"
}
}
}
],
"active": true
}
```
Representative broker payload:
```json
{
"eventId": "approval-7f3d",
"workflowId": "6f3f6db1-2b5f-4b34-a145-82b7ae814e91",
"approved": true,
"approvedBy": "reviewer@example.com"
}
```
Replace the representative `workflowId` with the ID returned when the waiting workflow starts. A correlation ID alone cannot target the WAIT task.
## Use an external provider
Change `event`/`sink` to a registered provider identifier and its provider-specific URI, for example `kafka:order-approvals`, `sqs:https://sqs.us-east-1.amazonaws.com/123/order-events`, `nats:orders.ready`, `jsm:orders.ready`, `nats_stream:orders.ready`, `amqp_queue:orders`, or `amqp_exchange:orders`. Enable the matching module and properties described in the guide.
# AI & LLM orchestration recipes
Build durable agents and LLM workflows with Conductor's native AI capabilities. Every recipe below runs with full durable execution guarantees — retries, state persistence, and crash recovery.
### Chat completion
A single-step workflow that sends a question to an LLM and returns the answer.
```json
{
"name": "chat_workflow",
"version": 1,
"schemaVersion": 2,
"tasks": [
{
"name": "chat_task",
"taskReferenceName": "chat",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{"role": "system", "message": "You are a helpful assistant."},
{"role": "user", "message": "${workflow.input.question}"}
],
"temperature": 0.7,
"maxTokens": 500
}
}
],
"inputParameters": ["question"],
"outputParameters": {
"answer": "${chat.output.result}"
}
}
```
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @chat_workflow.json
curl -X POST 'http://localhost:8080/api/workflow/chat_workflow' \
-H 'Content-Type: application/json' \
-d '{"question": "What is workflow orchestration?"}'
```
---
### RAG pipeline with vector database (search + answer)
A vector database workflow for retrieval-augmented generation: vector search retrieves relevant documents, then an LLM generates an answer grounded in those results.
```json
{
"name": "rag_workflow",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["question"],
"tasks": [
{
"name": "search_knowledge_base",
"taskReferenceName": "search",
"type": "LLM_SEARCH_INDEX",
"inputParameters": {
"vectorDB": "postgres-prod",
"namespace": "kb",
"index": "articles",
"embeddingModelProvider": "openai",
"embeddingModel": "text-embedding-3-small",
"query": "${workflow.input.question}",
"llmMaxResults": 3
}
},
{
"name": "generate_answer",
"taskReferenceName": "answer",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "anthropic",
"model": "claude-sonnet-4-20250514",
"messages": [
{"role": "system", "message": "Answer based on the following context: ${search.output.result}"},
{"role": "user", "message": "${workflow.input.question}"}
],
"temperature": 0.3
}
}
],
"outputParameters": {
"answer": "${answer.output.result}",
"sources": "${search.output.result}"
}
}
```
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @rag_workflow.json
curl -X POST 'http://localhost:8080/api/workflow/rag_workflow' \
-H 'Content-Type: application/json' \
-d '{"question": "How do I configure retry policies?"}'
```
!!! note "Prerequisites"
Requires a vector database (pgvector, Pinecone, or MongoDB Atlas) configured as a Conductor integration, plus at least one LLM provider. See [AI provider configuration](#ai-provider-configuration) below.
---
### MCP AI agent with function calling
A four-step agentic workflow demonstrating AI agent orchestration with function calling: discover available tools via MCP, ask an LLM to pick the right tool, execute it via tool use, and summarize the result.
```json
{
"name": "mcp_ai_agent_workflow",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["task"],
"tasks": [
{
"name": "list_available_tools",
"taskReferenceName": "discover_tools",
"type": "LIST_MCP_TOOLS",
"inputParameters": {
"mcpServer": "http://localhost:3001/mcp"
}
},
{
"name": "decide_which_tools_to_use",
"taskReferenceName": "plan",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "anthropic",
"model": "claude-sonnet-4-20250514",
"messages": [
{"role": "system", "message": "You are an AI agent. Available tools: ${discover_tools.output.tools}. User wants to: ${workflow.input.task}"},
{"role": "user", "message": "Which tool should I use and what parameters? Respond with JSON: {method: string, arguments: object}"}
],
"temperature": 0.1,
"maxTokens": 500
}
},
{
"name": "execute_tool",
"taskReferenceName": "execute",
"type": "CALL_MCP_TOOL",
"inputParameters": {
"mcpServer": "http://localhost:3001/mcp",
"method": "${plan.output.result.method}",
"arguments": "${plan.output.result.arguments}"
}
},
{
"name": "summarize_result",
"taskReferenceName": "summarize",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{"role": "user", "message": "Summarize this result for the user: ${execute.output.content}"}
],
"maxTokens": 200
}
}
],
"outputParameters": {
"summary": "${summarize.output.result}",
"rawToolOutput": "${execute.output.content}"
}
}
```
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @mcp_ai_agent_workflow.json
curl -X POST 'http://localhost:8080/api/workflow/mcp_ai_agent_workflow' \
-H 'Content-Type: application/json' \
-d '{"task": "Look up the latest order status for customer 42"}'
```
---
### Image generation
Generate images from a text prompt using DALL-E or another supported provider.
```json
{
"name": "image_gen_workflow",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["prompt"],
"tasks": [
{
"name": "generate_image",
"taskReferenceName": "image",
"type": "GENERATE_IMAGE",
"inputParameters": {
"llmProvider": "openai",
"model": "dall-e-3",
"prompt": "${workflow.input.prompt}",
"width": 1024,
"height": 1024,
"n": 1,
"style": "vivid"
}
}
],
"outputParameters": {
"imageUrl": "${image.output.result}"
}
}
```
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @image_gen_workflow.json
curl -X POST 'http://localhost:8080/api/workflow/image_gen_workflow' \
-H 'Content-Type: application/json' \
-d '{"prompt": "A futuristic city skyline at sunset, digital art"}'
```
---
### LLM report to PDF pipeline
An LLM generates a structured markdown report, then Conductor converts it to a downloadable PDF.
```json
{
"name": "llm_to_pdf_pipeline",
"description": "LLM generates a markdown report, then converts it to PDF",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["topic", "audience"],
"tasks": [
{
"name": "generate_report_markdown",
"taskReferenceName": "llm_report",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{"role": "system", "message": "You are a professional report writer. Generate well-structured markdown reports."},
{"role": "user", "message": "Write a detailed report about: ${workflow.input.topic}\nTarget audience: ${workflow.input.audience}"}
],
"temperature": 0.7,
"maxTokens": 2000
}
},
{
"name": "convert_to_pdf",
"taskReferenceName": "pdf_output",
"type": "GENERATE_PDF",
"inputParameters": {
"markdown": "${llm_report.output.result}",
"pageSize": "A4",
"theme": "default",
"baseFontSize": 11,
"pdfMetadata": {
"title": "${workflow.input.topic}",
"author": "Conductor AI Pipeline"
}
}
}
],
"outputParameters": {
"reportMarkdown": "${llm_report.output.result}",
"pdfLocation": "${pdf_output.output.result.location}"
}
}
```
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @llm_to_pdf_pipeline.json
curl -X POST 'http://localhost:8080/api/workflow/llm_to_pdf_pipeline' \
-H 'Content-Type: application/json' \
-d '{"topic": "Microservices observability best practices", "audience": "Platform engineering team"}'
```
---
### Web search — real-time information retrieval
Enable the LLM's built-in web search to answer questions about current events or find up-to-date information. No MCP server or external tool needed — the provider handles the search natively.
```json
{
"name": "web_search_workflow",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["question"],
"tasks": [
{
"name": "web_search_chat",
"taskReferenceName": "chat",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{"role": "system", "message": "Use web search to find current information."},
{"role": "user", "message": "${workflow.input.question}"}
],
"webSearch": true,
"maxTokens": 1000
}
}
],
"outputParameters": {
"answer": "${chat.output.result}"
}
}
```
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @web_search_workflow.json
curl -X POST 'http://localhost:8080/api/workflow/web_search_workflow' \
-H 'Content-Type: application/json' \
-d '{"question": "What are the latest developments in AI regulation?"}'
```
!!! note "Provider support"
Web search is supported by OpenAI, Anthropic, and Google Gemini. Set `"webSearch": true` — the same parameter works across all providers.
---
### Code execution — sandboxed code interpreter
Let the LLM write and run code in a sandboxed environment. Useful for data analysis, calculations, chart generation, and tasks that benefit from executable code.
```json
{
"name": "code_execution_workflow",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["task"],
"tasks": [
{
"name": "code_chat",
"taskReferenceName": "chat",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "google_gemini",
"model": "gemini-2.5-flash",
"messages": [
{"role": "system", "message": "Use code execution to compute results and analyze data."},
{"role": "user", "message": "${workflow.input.task}"}
],
"codeInterpreter": true,
"maxTokens": 2000
}
}
],
"outputParameters": {
"result": "${chat.output.result}"
}
}
```
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @code_execution_workflow.json
curl -X POST 'http://localhost:8080/api/workflow/code_execution_workflow' \
-H 'Content-Type: application/json' \
-d '{"task": "Calculate the first 100 prime numbers and find the average gap between consecutive primes"}'
```
!!! note "Provider support"
Code execution is supported by OpenAI (`code_interpreter`), Anthropic (`code_execution`), and Google Gemini (`codeExecution`). Set `"codeInterpreter": true` — the same parameter works across all providers.
---
### Coding agent — plan, code, and review
A three-step agent that plans an implementation, writes and executes the code using the code interpreter, and reviews the result. This pattern is useful for automated code generation tasks.
```json
{
"name": "coding_agent",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["task"],
"tasks": [
{
"name": "plan",
"taskReferenceName": "plan",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o",
"messages": [
{"role": "system", "message": "Break down the coding task into clear numbered steps."},
{"role": "user", "message": "${workflow.input.task}"}
],
"temperature": 0.2,
"maxTokens": 1000
}
},
{
"name": "write_and_run",
"taskReferenceName": "code",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o",
"messages": [
{"role": "system", "message": "Write the code, run it, verify the output, and fix any errors."},
{"role": "user", "message": "Plan:\n${plan.output.result}\n\nTask: ${workflow.input.task}"}
],
"codeInterpreter": true,
"temperature": 0.1,
"maxTokens": 4000
}
},
{
"name": "review",
"taskReferenceName": "review",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{"role": "system", "message": "Review the implementation for correctness and code quality."},
{"role": "user", "message": "Task: ${workflow.input.task}\n\nCode:\n${code.output.result}"}
],
"maxTokens": 1000
}
}
],
"outputParameters": {
"code": "${code.output.result}",
"review": "${review.output.result}"
}
}
```
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @coding_agent.json
curl -X POST 'http://localhost:8080/api/workflow/coding_agent' \
-H 'Content-Type: application/json' \
-d '{"task": "Write a Python function that converts Roman numerals to integers, with unit tests"}'
```
---
### Extended thinking — complex reasoning
Give the LLM a token budget for step-by-step reasoning before generating its final response. Useful for math, logic, code review, and complex analysis.
```json
{
"name": "extended_thinking_workflow",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["problem"],
"tasks": [
{
"name": "think_deeply",
"taskReferenceName": "think",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "anthropic",
"model": "claude-sonnet-4-20250514",
"messages": [
{"role": "user", "message": "${workflow.input.problem}"}
],
"thinkingTokenLimit": 10000,
"maxTokens": 16000
}
}
],
"outputParameters": {
"answer": "${think.output.result}"
}
}
```
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @extended_thinking_workflow.json
curl -X POST 'http://localhost:8080/api/workflow/extended_thinking_workflow' \
-H 'Content-Type: application/json' \
-d '{"problem": "Prove that the square root of 2 is irrational."}'
```
!!! note "Provider support"
Extended thinking is supported by Anthropic (`thinkingTokenLimit`) and Google Gemini (`thinkingBudgetTokens`). OpenAI uses `"reasoningEffort": "high"` for a similar effect.
---
### Multi-turn conversation chaining with previousResponseId
Chain multiple LLM calls as a conversation without resending the full message history. The first call returns a `responseId`; pass it as `previousResponseId` to the next call. OpenAI's Responses API stores the conversation server-side, saving tokens and latency.
```json
{
"name": "multi_turn_chain",
"description": "Two-step conversation using previousResponseId to avoid resending history",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["topic"],
"tasks": [
{
"name": "first_turn",
"taskReferenceName": "turn1",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o",
"messages": [
{"role": "system", "message": "You are a technical architect. Be concise."},
{"role": "user", "message": "Design a high-level architecture for: ${workflow.input.topic}"}
],
"temperature": 0.3,
"maxTokens": 2000
}
},
{
"name": "follow_up",
"taskReferenceName": "turn2",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o",
"messages": [
{"role": "user", "message": "Now list the key risks and mitigations for this architecture."}
],
"previousResponseId": "${turn1.output.responseId}",
"temperature": 0.3,
"maxTokens": 2000
}
}
],
"outputParameters": {
"architecture": "${turn1.output.result}",
"risks": "${turn2.output.result}"
}
}
```
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @multi_turn_chain.json
curl -X POST 'http://localhost:8080/api/workflow/multi_turn_chain' \
-H 'Content-Type: application/json' \
-d '{"topic": "Real-time collaborative document editor"}'
```
The second call sends only the new user message — OpenAI already has the full conversation context from `previousResponseId`. This is especially useful for long agent loops where resending the full history each iteration would be expensive.
!!! note "Provider support"
`previousResponseId` is supported by OpenAI and Azure OpenAI (Responses API). Other providers require sending the full message history in each call.
---
### Web research agent — search, synthesize, PDF
A multi-step agent that uses web search to gather information, an LLM with extended thinking to synthesize a report, and converts it to PDF. Combines three built-in capabilities in a single workflow.
```json
{
"name": "web_research_agent",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["topic"],
"tasks": [
{
"name": "gather_information",
"taskReferenceName": "research",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o",
"messages": [
{"role": "system", "message": "Use web search to find comprehensive, current information. Search for multiple perspectives and recent developments."},
{"role": "user", "message": "Research this topic thoroughly: ${workflow.input.topic}"}
],
"webSearch": true,
"temperature": 0.3,
"maxTokens": 3000
}
},
{
"name": "synthesize_report",
"taskReferenceName": "report",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "anthropic",
"model": "claude-sonnet-4-20250514",
"messages": [
{"role": "system", "message": "Synthesize the research into a well-structured markdown report with sections, key findings, and citations."},
{"role": "user", "message": "Topic: ${workflow.input.topic}\n\nResearch:\n${research.output.result}\n\nWrite a comprehensive report."}
],
"thinkingTokenLimit": 5000,
"maxTokens": 8000
}
},
{
"name": "convert_to_pdf",
"taskReferenceName": "pdf",
"type": "GENERATE_PDF",
"inputParameters": {
"markdown": "${report.output.result}",
"pageSize": "A4",
"pdfMetadata": {
"title": "${workflow.input.topic}",
"author": "Conductor Research Agent"
}
}
}
],
"outputParameters": {
"report": "${report.output.result}",
"pdf": "${pdf.output.result.location}"
}
}
```
**Register and run:**
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' \
-H 'Content-Type: application/json' \
-d @web_research_agent.json
curl -X POST 'http://localhost:8080/api/workflow/web_research_agent' \
-H 'Content-Type: application/json' \
-d '{"topic": "The state of WebAssembly adoption in 2026"}'
```
---
### AI provider configuration
Set environment variables before starting the server. Conductor auto-enables providers when their API key is present.
```bash
# OpenAI (required for most examples)
export OPENAI_API_KEY=sk-your-openai-api-key
# Anthropic (for RAG, extended thinking examples)
export ANTHROPIC_API_KEY=sk-ant-your-anthropic-key
# Google Gemini — API key (simplest)
export GEMINI_API_KEY=your-gemini-api-key
# Or Vertex AI (for enterprise/GCP) — set project and location in application.properties
```
For vector database and other advanced configuration, add to `application.properties`:
```properties
# PostgreSQL Vector DB (for RAG examples)
conductor.vectordb.instances[0].name=postgres-prod
conductor.vectordb.instances[0].type=postgres
conductor.vectordb.instances[0].postgres.datasourceURL=jdbc:postgresql://localhost:5432/vectors
conductor.vectordb.instances[0].postgres.user=conductor
conductor.vectordb.instances[0].postgres.password=secret
conductor.vectordb.instances[0].postgres.dimensions=1536
```
---
## More examples
For additional AI workflow definitions, see the [AI workflow examples on GitHub](https://github.com/conductor-oss/conductor/tree/main/ai/examples).
# Dynamic Workflows with AI
Use an LLM to select the best workflow for a user's request while keeping execution durable. The LLM sees a catalog of workflow names and descriptions, returns one selection as JSON, and a dynamic `SUB_WORKFLOW` runs that selected, registered workflow.
The catalog is intentional: a dynamic `SUB_WORKFLOW` can start only a workflow definition registered under the selected name. Keep the workflow names in the prompt aligned with the child workflows registered in Conductor; an invented name fails before any child workflow starts.
## Example: route a customer request
This router can choose one of three registered workflows. The complete runnable fixtures are in [`ai/examples/36-ai-workflow-routing.json`](https://github.com/conductor-oss/conductor/blob/main/ai/examples/36-ai-workflow-routing.json) and its paired `36a`–`36c` child workflows.
| Workflow | Description |
|---|---|
| `ai_route_support_ticket` | Use for product defects, access problems, and troubleshooting requests. |
| `ai_route_refund_request` | Use for returns, refunds, and duplicate-charge requests. |
| `ai_route_sales_lead` | Use for pricing, procurement, and enterprise sales requests. |
```json
{
"name": "ai_workflow_router",
"description": "Select an approved workflow for a customer request",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["request"],
"tasks": [
{
"name": "select_workflow",
"taskReferenceName": "select_workflow",
"type": "LLM_CHAT_COMPLETE",
"inputParameters": {
"llmProvider": "openai",
"model": "gpt-4o-mini",
"messages": [
{
"role": "system",
"message": "You route customer requests to approved workflows. Choose exactly one workflow from this json catalog and return valid json only. Catalog: [{\"workflow\":\"ai_route_support_ticket\",\"description\":\"Product defects, access problems, and troubleshooting.\"},{\"workflow\":\"ai_route_refund_request\",\"description\":\"Returns, refunds, and duplicate charges.\"},{\"workflow\":\"ai_route_sales_lead\",\"description\":\"Pricing, procurement, and enterprise sales.\"}]"
},
{
"role": "user",
"message": "Customer request: ${workflow.input.request}. Return valid json with workflow and reason."
}
],
"temperature": 0,
"maxTokens": 120,
"jsonOutput": true
}
},
{
"name": "run_selected_workflow",
"taskReferenceName": "run_selected_workflow",
"type": "SUB_WORKFLOW",
"inputParameters": {
"request": "${workflow.input.request}",
"routingReason": "${select_workflow.output.result.reason}"
},
"subWorkflowParam": {
"name": "${select_workflow.output.result.workflow}",
"version": 1
}
}
],
"outputParameters": {
"selectedWorkflow": "${select_workflow.output.result.workflow}",
"routingReason": "${select_workflow.output.result.reason}",
"subWorkflowId": "${run_selected_workflow.output.subWorkflowId}",
"subWorkflowOutput": "${run_selected_workflow.output}"
}
}
```
## Register the router and its approved destinations
Register each destination workflow before registering or starting the router. For a local end-to-end trial, these minimal destinations make each branch visible without calling an external system:
```json
{
"name": "ai_route_support_ticket",
"version": 1,
"schemaVersion": 2,
"inputParameters": ["request", "routingReason"],
"tasks": [{"name": "record_ticket", "taskReferenceName": "record_ticket", "type": "NOOP"}]
}
```
Create equivalent placeholder definitions named `ai_route_refund_request` and `ai_route_sales_lead`, then register all four definitions:
```shell
curl -X POST 'http://localhost:8080/api/metadata/workflow' -H 'Content-Type: application/json' -d @ai_route_support_ticket.json
curl -X POST 'http://localhost:8080/api/metadata/workflow' -H 'Content-Type: application/json' -d @ai_route_refund_request.json
curl -X POST 'http://localhost:8080/api/metadata/workflow' -H 'Content-Type: application/json' -d @ai_route_sales_lead.json
curl -X POST 'http://localhost:8080/api/metadata/workflow' -H 'Content-Type: application/json' -d @ai_workflow_router.json
```
Start the router:
```shell
curl -X POST 'http://localhost:8080/api/workflow/ai_workflow_router' \
-H 'Content-Type: application/json' \
-d '{"request":"I was charged twice for an order I returned."}'
```
The router records the selected workflow, the model's routing reason, and the child workflow ID in its output. `SUB_WORKFLOW` waits for the selected child to complete; the child output is available on `${run_selected_workflow.output}`.
## Adapt the catalog safely
To add a route, update both places together:
1. Add the workflow name and description to the LLM's catalog.
2. Register version `1` of a workflow whose name exactly matches the catalog entry.
The sub-workflow name is resolved at runtime from the LLM output. A name not present in the metadata registry cannot start a child workflow.
## Related recipes
- [AI Cookbook](../ai/cookbook/index.md) — production starters for chat, RAG, MCP agents, and native AI tasks.
- [Dynamic workflows as code](dynamic-workflows.md) — build workflow definitions in Python when the graph itself must be generated.
# Scheduled workflow recipes
These recipes reuse the checked-in fixtures under `scheduler/examples/`. Start with the [scheduling guide](../how-tos/Workflows/scheduling-workflows.md) for semantics and the [Scheduler API](../../documentation/api/scheduler.md) for the exact REST contract.
## Every minute
```json
{
"name": "every-minute-demo-schedule",
"cronExpression": "0 * * * * *",
"zoneId": "UTC",
"startWorkflowRequest": {
"name": "daily_report_workflow",
"version": 1,
"input": {}
},
"runCatchupScheduleInstances": false,
"paused": false
}
```
```bash
conductor schedule create scheduler/examples/every-minute-schedule.json
```
## Weekdays in a named timezone
```json
{
"name": "daily-report-schedule",
"cronExpression": "0 0 9 * * MON-FRI",
"zoneId": "America/New_York",
"startWorkflowRequest": {
"name": "daily_report_workflow",
"version": 1,
"input": {}
},
"scheduleStartTime": 0,
"scheduleEndTime": 0,
"runCatchupScheduleInstances": false,
"paused": false
}
```
The IANA zone follows local daylight-saving transitions. The correlation ID, if supplied, is literal; use the injected `_executionId` inside the workflow for per-run identity.
## Catch up missed cron slots
```json
{
"name": "catchup-demo-schedule",
"cronExpression": "0 * * * * *",
"zoneId": "UTC",
"runCatchupScheduleInstances": true,
"paused": false,
"startWorkflowRequest": {
"name": "catchup_demo_workflow",
"version": 1,
"input": {}
}
}
```
Catchup can create a burst after downtime. Make the target workflow idempotent and capacity-aware.
## Bound a schedule to a window
`scheduler/examples/bounded-schedule-template.json` contains `__START_MS__` and `__END_MS__` placeholders. Replace them with epoch-millisecond numbers before posting the file; the template itself is intentionally not valid as a final schedule payload.
```bash
curl -sS -X POST 'http://localhost:8080/api/scheduler/schedules' \
-H 'Content-Type: application/json' \
--data-binary @bounded-schedule.json
```
## Read scheduler metadata in a workflow
The canonical workflow uses `_scheduledTime` and `_executedTime` to compute a reporting window:
```json
{
"name": "input_param_demo_workflow",
"description": "Demonstrates scheduler-injected workflow input. Uses _scheduledTime and _executedTime to compute a 24-hour reporting window ending at the scheduled time.",
"version": 1,
"tasks": [
{
"name": "compute_report_window",
"taskReferenceName": "compute_report_window",
"type": "INLINE",
"inputParameters": {
"scheduledTime": "${workflow.input._scheduledTime}",
"executionTime": "${workflow.input._executedTime}",
"evaluatorType": "javascript",
"expression": "function toISO(ms) { return new Date(ms).toISOString(); } ({ reportWindowStart: toISO($.scheduledTime - 86400000), reportWindowEnd: toISO($.scheduledTime), scheduledAt: toISO($.scheduledTime), triggeredAt: toISO($.executionTime) })"
}
}
],
"outputParameters": {
"reportWindowStart": "${compute_report_window.output.result.reportWindowStart}",
"reportWindowEnd": "${compute_report_window.output.result.reportWindowEnd}",
"scheduledAt": "${compute_report_window.output.result.scheduledAt}",
"triggeredAt": "${compute_report_window.output.result.triggeredAt}"
},
"schemaVersion": 2,
"restartable": true,
"ownerEmail": "demo@example.com",
"timeoutPolicy": "ALERT_ONLY",
"timeoutSeconds": 30
}
```
Its paired schedule is:
```json
{
"name": "input-param-demo-schedule",
"cronExpression": "0 * * * * *",
"zoneId": "UTC",
"runCatchupScheduleInstances": false,
"startWorkflowRequest": {
"name": "input_param_demo_workflow",
"version": 1,
"input": {
"reportOwner": "platform-team",
"alertThreshold": 100
}
}
}
```
The other injected values are `_startedByScheduler`, `_executionId`, and `_schedulerCron`.
## Demonstrate overlapping runs
```json
{
"name": "concurrent-demo-schedule",
"cronExpression": "0 * * * * *",
"zoneId": "UTC",
"runCatchupScheduleInstances": false,
"startWorkflowRequest": {
"name": "concurrent_demo_workflow",
"version": 1,
"input": {}
}
}
```
Conductor has no native overlap policy. The paired `concurrent-workflow.json` demonstrates that the next slot can start while the prior execution remains active.
## More canonical fixtures
The fixture family also includes retry, `DO_WHILE`, and parallel multi-step workflows. Register workflow files with the metadata API or CLI before creating their paired schedule. See [`scheduler/examples/README.md`](https://github.com/conductor-oss/conductor/blob/main/scheduler/examples/README.md) for the complete local walkthrough.
# Dynamic workflows in code
## Workflow as code
Conductor supports a code-first workflow approach — build workflows programmatically using the Python SDK instead of writing JSON by hand. This workflow as code pattern lets you chain tasks with the `>>` operator, add conditional logic, loops, and parallel branches — all in Python. Code-first workflows are ideal for dynamic workflows where the task graph is determined at runtime.
### Simple sequential workflow
Chain tasks with the `>>` operator. Worker functions decorated with `@worker_task` become reusable task building blocks.
```python
from conductor.client.workflow.conductor_workflow import ConductorWorkflow
from conductor.client.worker.worker_task import worker_task
@worker_task(task_definition_name='fetch_order')
def fetch_order(order_id: str) -> dict:
return {'order_id': order_id, 'amount': 99.99, 'item': 'Widget'}
@worker_task(task_definition_name='process_payment')
def process_payment(order_id: str, amount: float) -> dict:
return {'transaction_id': 'txn_abc123', 'status': 'charged'}
@worker_task(task_definition_name='ship_order')
def ship_order(order_id: str, transaction_id: str) -> dict:
return {'tracking': 'TRACK-456', 'carrier': 'FedEx'}
workflow = ConductorWorkflow(name='order_fulfillment', version=1, executor=executor)
fetch = fetch_order(task_ref_name='fetch', order_id=workflow.input('order_id'))
pay = process_payment(
task_ref_name='pay',
order_id=workflow.input('order_id'),
amount=fetch.output('amount'),
)
ship = ship_order(
task_ref_name='ship',
order_id=workflow.input('order_id'),
transaction_id=pay.output('transaction_id'),
)
workflow >> fetch >> pay >> ship
workflow.output_parameters({
'tracking': ship.output('tracking'),
'transaction_id': pay.output('transaction_id'),
})
workflow.register(overwrite=True)
```
---
### Conditional branching with Switch
Route execution based on task output or workflow input. Each case gets its own task chain.
```python
from conductor.client.workflow.conductor_workflow import ConductorWorkflow
from conductor.client.workflow.task.switch_task import SwitchTask
workflow = ConductorWorkflow(name='route_by_priority', version=1, executor=executor)
classify = classify_ticket(
task_ref_name='classify',
description=workflow.input('description'),
)
switch = SwitchTask(task_ref_name='priority_router', case_expression=classify.output('priority'))
# Each case is a list of tasks to execute
switch.switch_case('critical', [
page_oncall(task_ref_name='page', ticket_id=workflow.input('ticket_id')),
escalate(task_ref_name='escalate', ticket_id=workflow.input('ticket_id')),
])
switch.switch_case('high', [
assign_senior(task_ref_name='assign', ticket_id=workflow.input('ticket_id')),
])
switch.default_case([
add_to_backlog(task_ref_name='backlog', ticket_id=workflow.input('ticket_id')),
])
workflow >> classify >> switch
workflow.register(overwrite=True)
```
---
### Parallel execution with Fork/Join
Run independent tasks in parallel and wait for all to complete.
```python
from conductor.client.workflow.conductor_workflow import ConductorWorkflow
from conductor.client.workflow.task.fork_task import ForkTask
from conductor.client.workflow.task.join_task import JoinTask
workflow = ConductorWorkflow(name='parallel_enrichment', version=1, executor=executor)
# Define independent tasks
credit_check = check_credit(task_ref_name='credit', customer_id=workflow.input('customer_id'))
fraud_check = check_fraud(task_ref_name='fraud', customer_id=workflow.input('customer_id'))
kyc_check = check_kyc(task_ref_name='kyc', customer_id=workflow.input('customer_id'))
# Fork runs all branches in parallel
fork = ForkTask(
task_ref_name='parallel_checks',
forked_tasks=[
[credit_check],
[fraud_check],
[kyc_check],
],
)
# Join waits for all branches
join = JoinTask(task_ref_name='wait_all', join_on=['credit', 'fraud', 'kyc'])
# Merge results
decide = make_decision(
task_ref_name='decide',
credit_score=credit_check.output('score'),
fraud_risk=fraud_check.output('risk_level'),
kyc_status=kyc_check.output('status'),
)
workflow >> fork >> join >> decide
workflow.output_parameters({'decision': decide.output('result')})
workflow.register(overwrite=True)
```
---
### Loops with Do/While
Repeat a set of tasks until a condition is met — useful for polling, retries, or iterative AI agent loops.
```python
from conductor.client.workflow.conductor_workflow import ConductorWorkflow
from conductor.client.workflow.task.do_while_task import DoWhileTask
workflow = ConductorWorkflow(name='agent_loop', version=1, executor=executor)
# The task(s) to repeat each iteration
think = call_llm(
task_ref_name='think',
prompt=workflow.input('goal'),
)
act = execute_tool(
task_ref_name='act',
tool=think.output('tool'),
args=think.output('args'),
)
# Loop until the LLM says it's done (max 10 iterations)
loop = DoWhileTask(
task_ref_name='agent_loop',
termination_condition='if ($.act["output"]["done"] == true) { false; } else { true; }',
tasks=[think, act],
)
loop.input_parameters.update({'max_iterations': 10})
summarize = summarize_results(task_ref_name='summarize', results=act.output('results'))
workflow >> loop >> summarize
workflow.register(overwrite=True)
```
---
### HTTP + system tasks mixed with workers
Combine built-in system tasks (HTTP, Wait, JQ Transform) with custom workers — no extra deployment needed for system tasks.
{% raw %}
```python
from conductor.client.workflow.conductor_workflow import ConductorWorkflow
from conductor.client.workflow.task.http_task import HttpTask
from conductor.client.workflow.task.json_jq_task import JsonJQTask
from conductor.client.workflow.task.wait_task import WaitTask
workflow = ConductorWorkflow(name='data_pipeline', version=1, executor=executor)
# HTTP task — fetch data from an external API (no worker needed)
fetch = HttpTask(task_ref_name='fetch_data', http_input={
'uri': 'https://api.example.com/records',
'method': 'GET',
'headers': {'Authorization': ['Bearer ${workflow.input.api_key}']},
})
# JQ Transform — reshape the response (no worker needed)
transform = JsonJQTask(
task_ref_name='transform',
script='.body.records | map({id: .id, value: .metrics.total})',
)
transform.input_parameters.update({
'records': fetch.output('response.body'),
})
# Custom worker — run business logic
enrich = enrich_records(
task_ref_name='enrich',
records=transform.output('result'),
)
# Wait — pause for 5 seconds before the next step
cooldown = WaitTask(task_ref_name='cooldown', wait_for_seconds=5)
# Custom worker — store results
store = save_to_database(task_ref_name='store', records=enrich.output('enriched'))
workflow >> fetch >> transform >> enrich >> cooldown >> store
workflow.output_parameters({'stored': store.output('count')})
workflow.register(overwrite=True)
```
{% endraw %}
---
### Sub-workflows
Break large workflows into reusable pieces. A parent workflow invokes child workflows as tasks.
```python
from conductor.client.workflow.conductor_workflow import ConductorWorkflow
from conductor.client.workflow.task.sub_workflow_task import SubWorkflowTask
# Child workflow (registered separately)
child = ConductorWorkflow(name='process_single_item', version=1, executor=executor)
validate = validate_item(task_ref_name='validate', item=child.input('item'))
transform = transform_item(task_ref_name='transform', item=validate.output('validated'))
child >> validate >> transform
child.output_parameters({'result': transform.output('transformed')})
child.register(overwrite=True)
# Parent workflow invokes the child
parent = ConductorWorkflow(name='batch_processor', version=1, executor=executor)
prepare = prepare_batch(task_ref_name='prepare', batch_id=parent.input('batch_id'))
run_child = SubWorkflowTask(
task_ref_name='process_item',
workflow_name='process_single_item',
version=1,
)
run_child.input_parameters.update({'item': prepare.output('first_item')})
aggregate = aggregate_results(
task_ref_name='aggregate',
result=run_child.output('result'),
)
parent >> prepare >> run_child >> aggregate
parent.register(overwrite=True)
```
---
### Runtime-generated dynamic workflow
Build a workflow definition at runtime and execute it without pre-registration. This runtime workflow pattern enables dynamic workflows where the task graph is generated on-the-fly — useful for AI agents, data pipelines, and any scenario where the steps are not known ahead of time.
{% raw %}
```python
from conductor.client.configuration.configuration import Configuration
from conductor.client.orkes_clients import OrkesClients
from conductor.client.http.models import StartWorkflowRequest
config = Configuration()
clients = OrkesClients(configuration=config)
executor = clients.get_workflow_executor()
# Build the workflow definition dynamically
steps = ['validate', 'enrich', 'store'] # determined at runtime
tasks = []
for i, step in enumerate(steps):
tasks.append({
'name': step,
'taskReferenceName': f'{step}_{i}',
'type': 'SIMPLE',
'inputParameters': {
'data': '${workflow.input.data}' if i == 0 else f'${{{steps[i-1]}_{i-1}.output.result}}',
},
})
# Start with inline definition — no pre-registration needed
request = StartWorkflowRequest(
name='dynamic_pipeline',
workflow_def={
'name': 'dynamic_pipeline',
'version': 1,
'tasks': tasks,
'outputParameters': {
'result': f'${{{steps[-1]}_{len(steps)-1}.output.result}}',
},
},
input={'data': {'key': 'value'}},
)
workflow_id = executor.start_workflow(request)
print(f'Started dynamic workflow: {workflow_id}')
```
{% endraw %}
This pattern is powerful for AI agents that generate execution plans at runtime — the LLM produces the list of steps, your code builds the workflow definition, and Conductor executes it with full durability, retries, and observability.
---
### Execute and wait for result
Run a workflow synchronously and get the result inline — useful for APIs and interactive applications.
```python
from conductor.client.configuration.configuration import Configuration
from conductor.client.orkes_clients import OrkesClients
config = Configuration()
clients = OrkesClients(configuration=config)
executor = clients.get_workflow_executor()
# Execute synchronously — blocks until the workflow completes
run = executor.execute(
name='order_fulfillment',
version=1,
workflow_input={'order_id': 'ORD-789'},
)
print(f'Status: {run.status}')
print(f'Output: {run.output}')
print(f'View: {config.ui_host}/execution/{run.workflow_id}')
```
---
## Setup
All examples above assume a `WorkflowExecutor` instance. Here is the standard setup:
```python
from conductor.client.configuration.configuration import Configuration
from conductor.client.orkes_clients import OrkesClients
config = Configuration() # reads CONDUCTOR_SERVER_URL from env
clients = OrkesClients(configuration=config)
executor = clients.get_workflow_executor()
```
```shell
pip install conductor-python
export CONDUCTOR_SERVER_URL=http://localhost:8080/api
```
For more Python SDK examples, see the [Python SDK documentation](../../documentation/clientsdks/python-sdk.md) and the [examples on GitHub](https://github.com/conductor-oss/python-sdk/tree/main/examples).
# Workflow Definition
The Workflow Definition contains all the information necessary to define the behavior of a workflow. The most important part of this definition is the `tasks` property, which is an array of [**Task Configurations**](#task-configurations).
For the formal JSON Schema definitions of workflow and task structures, see [Schemas](../schemas.md). The linked source schemas are the field-level contract.
## Workflow Properties
| Field | Type | Description | Notes |
|:------------------------------|:---------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| :------------------------------------------------------------------------------------------------ |
| name | string | Name of the workflow | |
| description | string | Description of the workflow | Optional |
| version | number | Numeric field used to identify the version of the schema. Use incrementing numbers. | When starting a workflow execution, if not specified, the definition with highest version is used |
| tasks | array of object(s) | An array of task configurations. [Details](#task-configurations) | |
| inputParameters | array of string(s) | List of input parameters. Used for documenting the required inputs to workflow | Optional. |
| outputParameters | object | JSON template used to generate the output of the workflow | If not specified, the output is defined as the output of the _last_ executed task |
| inputTemplate | object | Default input values. See [Using inputTemplate](#default-input-with-inputtemplate) | Optional. |
| failureWorkflow | string | Workflow to be run on current Workflow failure. Useful for cleanup or post actions on failure. [Explanation](#failure-workflow) | Optional. |
| failureWorkflowVersion | number | When `failureWorkflow` parameter is specified, sets the _failure workflow version_ to be run on current Workflow failure. If not specified, the latest version will be used. | Optional. |
| schemaVersion | number | Current Conductor Schema version. schemaVersion 1 is discontinued. | Must be 2 |
| restartable | boolean | Flag to allow Workflow restarts | Defaults to true |
| workflowStatusListenerEnabled | boolean | Enable status callback. [Explanation](#workflow-status-listener) | Defaults to false |
| ownerEmail | string | Email address of the team that owns the workflow | Required |
| timeoutSeconds | number | The timeout in seconds after which the workflow will be marked as `TIMED_OUT` if it hasn't been moved to a terminal state | No timeouts if set to 0 |
| timeoutPolicy | string ([enum](#timeout-policy)) | Workflow's timeout policy | Defaults to `TIME_OUT_WF` |
### Failure Workflow
The failure workflow gets the _original failed workflow’s input_ along with 3 additional items,
* `workflowId` - The id of the failed workflow which triggered the failure workflow.
* `reason` - A string containing the reason for workflow failure.
* `failureStatus` - A string status representation of the failed workflow.
* `failureTaskId` - The id of the failed task of the workflow that triggered the failure workflow.
### Timeout Policy
* TIME_OUT_WF: Workflow is marked as TIMED_OUT and terminated
* ALERT_ONLY: Registers a counter (workflow_failure with status tag set to `TIMED_OUT`)
### Workflow Status Listener
Setting the `workflowStatusListenerEnabled` field in your Workflow Definition to `true` enables notifications.
To add a custom implementation of the Workflow Status Listener. Refer to the [Workflow Status Listener extension guide](../../advanced/extend.md#workflow-status-listener).
The listener can be implemented in such a way as to either send a notification to an external system or to send an event on the conductor queue to complete/fail another task in another workflow as described in the [event handlers guide](../eventhandlers.md).
### Default Input with `inputTemplate`
* `inputTemplate` allows you to define default input values, which can optionally be overridden at runtime (when the workflow is invoked).
* Eg: In your Workflow Definition, you can define your inputTemplate as:
```json
"inputTemplate": {
"url": "https://some_url:7004"
}
```
And `url` would be `https://some_url:7004` if no `url` was provided as input to your workflow.
## Task Configurations
The `tasks` property in a Workflow Definition defines an array of *Task Configurations*. This is the blueprint for the workflow. Task Configurations can reference different types of Tasks.
* Simple Tasks
* System Tasks
* Operators
Note: Task Configuration should not be confused with **Task Definitions**, which are used to register SIMPLE (worker based) tasks.
| Field | Type | Description | Notes |
| :---------------- | :------ | :--------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------- |
| name | string | Name of the task. MUST be registered as a Task Type with Conductor before starting workflow | |
| taskReferenceName | string | Alias used to refer the task within the workflow. MUST be unique within workflow. | |
| type | string | Type of task. SIMPLE for tasks executed by remote workers, or one of the system task types | |
| description | string | Description of the task | optional |
| optional | boolean | true or false. When set to true - workflow continues even if the task fails. The status of the task is reflected as `COMPLETED_WITH_ERRORS` | Defaults to `false` |
| inputParameters | object | JSON template that defines the input given to the task. Only one of `inputParameters` or `inputExpression` can be used in a task. | See [Using Expressions](#using-expressions) for details |
| inputExpression | object | JSONPath expression that defines the input given to the task. Only one of `inputParameters` or `inputExpression` can be used in a task. | See [Using Expressions](#using-expressions) for details |
| asyncComplete | boolean | `false` to mark status COMPLETED upon execution; `true` to keep the task IN_PROGRESS and wait for an external event to complete it. | Defaults to `false` |
| startDelay | number | Time in seconds to wait before making the task available to be polled by a worker. | Defaults to 0. |
In addition to these parameters, System Tasks have their own parameters. Check out [System Tasks](systemtasks/index.md) for more information.
### Using Expressions
Each executed task is given an input based on the `inputParameters` template or the `inputExpression` configured in the task configuration. Only one of `inputParameters` or `inputExpression` can be used in a task.
#### inputParameters
`inputParameters` can use JSONPath **expressions** to extract values out of the workflow input and other tasks in the workflow.
For example, workflows are supplied an `input` by the client/caller when a new execution is triggered. The workflow `input` is available via an *expression* of the form `${workflow.input...}`. Likewise, the `input` and `output` data of a previously executed task can also be extracted using an *expression* for use in the `inputParameters` of a subsequent task.
Generally, `inputParameters` can use *expressions* of the following syntax:
> `${SOURCE.input/output.JSONPath}`
| Field | Description |
| ------------ | ------------------------------------------------------------------------ |
| SOURCE | Can be either `"workflow"` or the reference name of any task |
| input/output | Refers to either the input or output of the source |
| JSONPath | JSON path expression to extract JSON fragment from source's input/output |
!!! note "JSON Path Support"
Conductor supports [JSONPath](http://goessner.net/articles/JsonPath/) specification and uses the [jayway/JsonPath](https://github.com/jayway/JsonPath) Java implementation.
!!! note "Escaping expressions"
To escape an expression, prefix it with an extra _$_ character (ex.: ```$${workflow.input...}```).
#### inputExpression
`inputExpression` can be used to select an entire object from the workflow input, or the output of another task. The field supports all [definite](https://github.com/json-path/JsonPath#what-is-returned-when) JSONPath expressions.
The syntax for mapping values in `inputExpression` follows the pattern,
> `SOURCE.input/output.JSONPath`
**NOTE:** The ```inputExpression``` field does not require the expression to be wrapped in `${}`.
See [example](#example-3-inputexpression) below.
## Examples
### Example 1 - A Basic Workflow Definition
Assume your business logic is to simply to get some shipping information and then do the shipping. You start by
logically partitioning them into two tasks:
1. *shipping_info* - The first task takes the provided account number, and outputs an address.
2. *shipping_task* - The 2nd task takes the address info and generates a shipping label.
We can configure these two tasks in the `tasks` array of our Workflow Definition. Let's assume that ```shipping info``` takes an account number, and returns a name and address.
```json
{
"name": "mail_a_box",
"description": "shipping Workflow",
"version": 1,
"tasks": [
{
"name": "shipping_info",
"taskReferenceName": "shipping_info_ref",
"inputParameters": {
"account": "${workflow.input.accountNumber}"
},
"type": "SIMPLE"
},
{
"name": "shipping_task",
"taskReferenceName": "shipping_task_ref",
"inputParameters": {
"name": "${shipping_info_ref.output.name}",
"streetAddress": "${shipping_info_ref.output.streetAddress}",
"city": "${shipping_info_ref.output.city}",
"state": "${shipping_info_ref.output.state}",
"zipcode": "${shipping_info_ref.output.zipcode}",
},
"type": "SIMPLE"
}
],
"outputParameters": {
"trackingNumber": "${shipping_task_ref.output.trackingNumber}"
},
"failureWorkflow": "shipping_issues",
"failureWorkflowVersion": 1,
"restartable": true,
"workflowStatusListenerEnabled": true,
"ownerEmail": "conductor@example.com",
"timeoutPolicy": "ALERT_ONLY",
"timeoutSeconds": 0,
"variables": {},
"inputTemplate": {}
}
```
Upon completion of the 2 tasks, the workflow outputs the tracking number generated in the 2nd task. If the workflow fails, a second workflow named ```shipping_issues``` is run.
### Example 2 - Task Configuration
Consider a task `http_task` with input configured to use input/output parameters from workflow and a task named `loc_task`.
```json
{
"name": "encode_workflow",
"description": "Encode movie.",
"version": 1,
"inputParameters": [
"movieId", "fileLocation", "recipe"
],
"tasks": [
{
"name": "loc_task",
"taskReferenceName": "loc_task_ref",
"taskType": "SIMPLE",
...
},
{
"name": "http_task",
"taskReferenceName": "http_task_ref",
"taskType": "HTTP",
"inputParameters": {
"movieId": "${workflow.input.movieId}",
"url": "${workflow.input.fileLocation}",
"lang": "${loc_task.output.languages[0]}",
"http_request": {
"method": "POST",
"url": "http://example.com/${loc_task.output.fileId}/encode",
"body": {
"recipe": "${workflow.input.recipe}",
"params": {
"width": 100,
"height": 100
}
},
"headers": {
"Accept": "application/json",
"Content-Type": "application/json"
}
}
}
}
],
"ownerEmail": "conductor@example.com",
"variables": {},
"inputTemplate": {}
}
```
Consider the following as the _workflow input_
```json
{
"movieId": "movie_123",
"fileLocation":"s3://moviebucket/file123",
"recipe":"png"
}
```
And the output of the _loc_task_ as the following;
```json
{
"fileId": "file_xxx_yyy_zzz",
"languages": ["en","ja","es"]
}
```
When scheduling the task, Conductor will merge the values from workflow input and `loc_task`'s output and create the input to the `http_task` as follows:
```json
{
"movieId": "movie_123",
"url": "s3://moviebucket/file123",
"lang": "en",
"http_request": {
"method": "POST",
"url": "http://example.com/file_xxx_yyy_zzz/encode",
"body": {
"recipe": "png",
"params": {
"width": 100,
"height": 100
}
},
"headers": {
"Accept": "application/json",
"Content-Type": "application/json"
}
}
}
```
### Example 3 - inputExpression
Given the following task configuration:
```json
{
"name": "loc_task",
"taskReferenceName": "loc_task_ref",
"taskType": "SIMPLE",
"inputExpression": {
"expression": "workflow.input",
"type": "JSON_PATH"
}
}
```
When the workflow is invoked with the following _workflow input_
```json
{
"movieId": "movie_123",
"fileLocation":"s3://moviebucket/file123",
"recipe":"png"
}
```
When the task `loc_task` is scheduled, the entire workflow input object will be passed in as the task input:
```json
{
"movieId": "movie_123",
"fileLocation":"s3://moviebucket/file123",
"recipe":"png"
}
```
# Start Workflow API
## Start a Workflow (Asynchronous)
```
POST /api/workflow
```
Starts a new workflow execution asynchronously. Returns the workflow ID immediately.
### Request Body
| Field | Description | Required |
|---|---|---|
| `name` | Workflow name (must be registered) | Yes |
| `version` | Workflow version | No (defaults to latest) |
| `input` | JSON object with input parameters for the workflow | No |
| `correlationId` | Unique ID to correlate multiple workflow executions | No |
| `taskToDomain` | Task-to-domain mapping. See [Task Domains](taskdomains.md). | No |
| `workflowDef` | Inline [Workflow Definition](../configuration/workflowdef/index.md) for dynamic workflows. See [Dynamic Workflows](#dynamic-workflows). | No |
| `externalInputPayloadStoragePath` | Path to external payload storage. See [External Payload Storage](../advanced/externalpayloadstorage.md). | No |
| `priority` | Priority level (0–99) for tasks within this workflow | No |
### Example
```shell
curl -X POST 'http://localhost:8080/api/workflow' \
-H 'Content-Type: application/json' \
-d '{
"name": "myWorkflow",
"version": 1,
"correlationId": "order-123",
"priority": 1,
"input": {
"customerId": "CUST-456",
"amount": 99.99
},
"taskToDomain": {
"*": "mydomain"
}
}'
```
**Response** `200 OK` — returns the workflow ID as plain text:
```
3a5b8c2d-1234-5678-9abc-def012345678
```
### Start with Path Parameters
```
POST /api/workflow/{name}
```
Alternative way to start a workflow — specify the name in the path and pass input as the request body.
| Parameter | Type | Description | Required |
|---|---|---|---|
| `name` | Path | Workflow name | Yes |
| `version` | Query | Workflow version | No |
| `correlationId` | Query | Correlation ID | No |
| `priority` | Query | Priority 0–99 (default: 0) | No |
```shell
curl -X POST 'http://localhost:8080/api/workflow/myWorkflow?version=1&correlationId=order-123' \
-H 'Content-Type: application/json' \
-d '{"customerId": "CUST-456", "amount": 99.99}'
```
**Response** `200 OK` — returns the workflow ID as plain text.
---
## Execute a Workflow (Synchronous)
```
POST /api/workflow/execute/{name}/{version}
```
Starts a workflow and **waits for completion** (or a specified condition) before returning the result. This eliminates the need to poll for workflow status.
| Parameter | Type | Description | Required |
|---|---|---|---|
| `name` | Path | Workflow name | Yes |
| `version` | Path | Workflow version (use `0` for latest) | Yes |
| `requestId` | Query | Idempotency key | No (auto-generated) |
| `waitUntilTaskRef` | Query | Comma-separated task reference names to wait for | No |
| `waitForSeconds` | Query | Maximum wait time in seconds | No (default: 10) |
| `consistency` | Query | `DURABLE` or `EVENTUAL` | No (default: `DURABLE`) |
| `returnStrategy` | Query | Controls which workflow state is returned | No (default: `TARGET_WORKFLOW`) |
Request body: a StartWorkflowRequest object (same format as the [async start](#start-a-workflow-asynchronous)).
### Example
```shell
curl -X POST 'http://localhost:8080/api/workflow/execute/my_workflow/1?waitForSeconds=30' \
-H 'Content-Type: application/json' \
-d '{
"name": "my_workflow",
"version": 1,
"input": {
"url": "https://api.example.com/data"
}
}'
```
**Response** `200 OK` — returns the workflow execution result:
```json
{
"workflowId": "3a5b8c2d-1234-5678-9abc-def012345678",
"requestId": "req-uuid",
"status": "COMPLETED",
"output": {
"response": {...}
},
"tasks": [...]
}
```
### Wait Behavior
- If `waitUntilTaskRef` is specified, the API returns when any listed task reaches a terminal state (or a WAIT task is encountered)
- If the workflow completes before the timeout, the result is returned immediately
- If the timeout is reached, the current workflow state is returned — the workflow continues running in the background
- Sub-workflow WAIT tasks are detected recursively
---
## Dynamic Workflows
Start a one-time workflow without pre-registering its definition. Provide the full workflow definition inline via the `workflowDef` field.
```shell
curl -X POST 'http://localhost:8080/api/workflow' \
-H 'Content-Type: application/json' \
-d '{
"name": "my_adhoc_workflow",
"workflowDef": {
"ownerApp": "my_app",
"ownerEmail": "owner@example.com",
"name": "my_adhoc_workflow",
"version": 1,
"tasks": [
{
"name": "fetch_data",
"type": "HTTP",
"taskReferenceName": "fetch_data",
"inputParameters": {
"uri": "${workflow.input.uri}",
"method": "GET"
},
"taskDefinition": {
"name": "fetch_data",
"retryCount": 0,
"timeoutSeconds": 3600,
"timeoutPolicy": "TIME_OUT_WF",
"responseTimeoutSeconds": 3000
}
}
]
},
"input": {
"uri": "https://api.example.com/data"
}
}'
```
**Response** `200 OK` — returns the workflow ID as plain text.
!!! note
If a `taskDefinition` is already registered via the Metadata API, it does not need to be included inline in the dynamic workflow definition.
# Scheduler API
The scheduler controller is mounted at `/api/scheduler`. It is present only when `conductor.scheduler.enabled=true`. All endpoints below return `200 OK` on success unless noted otherwise.
## Schedule model
| Field | Type | Required | Runtime default or behavior |
|---|---|---|---|
| `name` | string | Yes | Unique key used for create-or-update |
| `cronExpression` | string | One cron form required | Legacy single expression |
| `zoneId` | string | No | `UTC` |
| `cronSchedules` | array | One cron form required | Non-empty array takes precedence over `cronExpression`/`zoneId`; entry `zoneId` defaults to `UTC` |
| `startWorkflowRequest` | object | Yes | Standard workflow start request |
| `runCatchupScheduleInstances` | boolean | No | `false` |
| `paused` | boolean | No | `false` |
| `pausedReason` | string | No | Set by pause operation |
| `scheduleStartTime` | long | No | Epoch-millisecond lower bound |
| `scheduleEndTime` | long | No | Epoch-millisecond upper bound |
| `description` | string | No | User description |
| `createTime`, `updatedTime`, `createdBy`, `updatedBy`, `nextRunTime` | server fields | No | Populated by the service |
A `cronSchedules` entry contains `cronExpression` and optional `zoneId`. `startWorkflowRequest.correlationId` is copied literally. The scheduler adds `_startedByScheduler`, `_scheduledTime`, `_executedTime`, `_executionId`, and `_schedulerCron` to workflow input.
## Create or update
```http
POST /api/scheduler/schedules
Content-Type: application/json
```
The body is one schedule object. The response is the stored schedule, including computed state such as `nextRunTime`.
```bash
curl -sS -X POST 'http://localhost:8080/api/scheduler/schedules' \
-H 'Content-Type: application/json' \
--data-binary @scheduler/examples/every-minute-schedule.json
```
## List and get
```http
GET /api/scheduler/schedules?workflowName={workflowName}
GET /api/scheduler/schedules/{name}
```
`workflowName` is optional. List returns an array; get returns one schedule or the service's not-found response.
## Search schedules
```http
GET /api/scheduler/schedules/search
```
| Query | Type | Default |
|---|---|---|
| `workflowName` | string | unset |
| `scheduleName` | string | unset |
| `paused` | boolean | unset |
| `freeText` | string | `*` |
| `start` | integer | `0` |
| `size` | integer | `100` |
| `sort` | comma-separated string | empty |
Returns `SearchResult`.
## Pause and resume
```http
PUT /api/scheduler/schedules/{name}/pause?reason={reason}
PUT /api/scheduler/schedules/{name}/resume
```
`reason` is optional. Both operations return an empty `200 OK` response.
## Bulk pause and resume
```http
PUT /api/scheduler/bulk/pause
PUT /api/scheduler/bulk/resume
Content-Type: application/json
```
Each body is a JSON array of schedule names. The response is a `BulkResponse`, with successful names and per-name errors. These endpoints are registered with the same scheduler condition as the rest of the Scheduler API.
```json
["nightly-report", "hourly-cleanup"]
```
## Delete
```http
DELETE /api/scheduler/schedules/{name}
```
Returns an empty `200 OK` response.
## Preview next times
```http
GET /api/scheduler/nextFewSchedules?cronExpression={cron}&scheduleStartTime={ms}&scheduleEndTime={ms}&limit={n}
```
`cronExpression` is required. Bounds are optional. `limit` defaults to 5 and the implementation caps results at 5. Preview uses `conductor.scheduler.schedulerTimeZone`, not a request or schedule timezone, because this endpoint accepts no `zoneId`.
## Search scheduled executions
```http
GET /api/scheduler/search/executions
```
| Query | Type | Default |
|---|---|---|
| `query` | string | unset |
| `freeText` | string | `*` |
| `start` | integer | `0` |
| `size` | integer | `100` |
| `sort` | comma-separated string | empty |
Returns `SearchResult`. Execution records include the scheduler execution ID, scheduled and execution times, workflow name/ID, state, and failure details where applicable.
## Administrator endpoints
```http
GET /api/scheduler/admin/requeue
GET /api/scheduler/admin/pause
GET /api/scheduler/admin/resume
```
These operate on scheduler internals for recovery/debugging. They are not per-schedule pause/resume endpoints and should be access-controlled.
## Unsupported operations
The controller has no run-now endpoint, manual-backfill endpoint, overlap-policy field, or correlation-template expansion. Use direct workflow start for an ad hoc run, and implement concurrency/idempotency policy in the workflow or downstream system.
# Consume and route events with event handlers
An event handler consumes one provider event, evaluates an optional condition, and dispatches one or more actions. Register it with the [Event Handlers API](../api/eventhandlers.md); only active handlers are subscribed for processing.
```json
{
"name": "start_fulfillment_on_order_ready",
"event": "conductor:publish_order_event:order-status",
"condition": "$.status == 'READY'",
"actions": [
{
"action": "start_workflow",
"start_workflow": {
"name": "fulfill_order",
"version": 1,
"correlationId": "${orderId}",
"input": {
"orderId": "${orderId}",
"sourceEventId": "${workflowInstanceId}"
}
}
}
],
"active": true
}
```
## Event identifier
The format is `provider:`. Runtime parsing splits at the first colon. Valid registered provider keys are `conductor`, `kafka`, `sqs`, `nats`, `jsm`, `nats_stream`, `amqp_queue`, and `amqp_exchange` when their modules are enabled.
## Conditions and payload expressions
- `active` defaults to `false`.
- An absent condition is treated as true.
- Conditions evaluate against the payload root, for example `$.status == 'READY'`. If `evaluatorType` identifies a registered evaluator, Conductor uses it; otherwise it evaluates the condition with the default script evaluator.
- Action placeholders also resolve from the payload root, for example `${orderId}`.
- `expandInlineJSON: true` expands stringified JSON fields before expressions resolve.
## Action capability matrix
| Action | OSS Conductor | Orkes | Behavior |
|---|:---:|:---:|---|
| `start_workflow` | Yes | Yes | Starts the named workflow and adds Conductor event metadata to its input |
| `complete_task` | Yes | Yes | Completes an identified task |
| `fail_task` | Yes | Yes | Fails an identified task; can set `reasonForIncompletion` |
| `terminate_workflow` | No | Yes | Terminates the targeted workflow |
| `update_workflow_variables` | No | Yes | Updates variables on the targeted workflow |
For `complete_task` and `fail_task`, specify either `taskId`, or both `workflowId` and `taskRefName`. Those are exact task-targeting mechanisms; an OSS handler does not resolve a business correlation key to a waiting task. `terminate_workflow` and `update_workflow_variables` exist in the shared model but are not implemented by the OSS action processor.
## Concurrency and deduplication
Actions run concurrently and are not atomic. Each action is recorded separately using the broker message ID plus its action index. A stable broker message ID enables persisted duplicate detection after the event-execution record is stored, but downstream workflow starts, task updates, and external side effects still require idempotency.
For a condition that evaluates to false, Conductor records a skipped event execution and runs no actions. For a practical first-use walkthrough, see [Consume and route events](../../devguide/how-tos/consume-route-events.md); use this page as the action and expression reference.
# Event Handlers API
The controller is mounted at `/api/event`. Successful mutating operations return an empty `200 OK` response.
## Endpoints
| Method | Path | Request/response |
|---|---|---|
| `POST` | `/api/event` | Create one event-handler object; empty response |
| `PUT` | `/api/event` | Replace/update one handler object; empty response |
| `GET` | `/api/event` | Array of all handlers |
| `DELETE` | `/api/event/{name}` | Remove by handler name; empty response |
| `GET` | `/api/event/{event}?activeOnly=true` | Handlers for the exact event; `activeOnly` defaults to `true` |
The `{event}` path value can contain provider separators and must be URL-encoded when required by the client/proxy.
## Create example
```bash
curl -sS -X POST 'http://localhost:8080/api/event' \
-H 'Content-Type: application/json' \
--data-binary @docs/devguide/cookbook/examples/events/start-workflow-handler.json
```
## Handler fields
| Field | Required | Behavior |
|---|---|---|
| `name` | Yes | Non-empty, unique handler name |
| `event` | Yes | `provider:`; split at first colon |
| `condition` | No | Evaluated against payload root; omitted means true |
| `actions` | Yes | Non-empty list; actions execute concurrently |
| `active` | No | Defaults to `false` |
| `evaluatorType` | No | Selects a registered evaluator; otherwise the default script evaluator is used |
The shared model declares five enum values, but the OSS action processor implements only `start_workflow`, `complete_task`, and `fail_task`. Requests using `terminate_workflow` or `update_workflow_variables` can deserialize but fail during processing as unsupported.
## Task targeting
For `complete_task` and `fail_task`, provide `taskId`, or `workflowId` plus `taskRefName`. `reasonForIncompletion` is meaningful for `fail_task`. Output fields are expression-resolved from the event payload root.
## Status and delivery behavior
A false condition records `SKIPPED`. Each action has its own persisted event-execution record. Duplicate suppression depends on a stable broker message ID and the persisted record; actions are concurrent and not atomic.
See [Event handler configuration](../configuration/eventhandlers.md) for the data model and [Event orchestration](../../devguide/how-tos/event-bus.md) for provider configuration and operating guidance.
# Publish events with the Event task
`EVENT` publishes a JSON message through a registered event-queue provider. It is the generic publishing task: use [`KAFKA_PUBLISH`](kafka-publish-task.md) when the message contract needs Kafka-specific keys, headers, serializers, or producer controls.
## Task parameters
| Parameter | Required | Behavior |
|---|---|---|
| `sink` | Yes | `provider:`; expressions resolve at runtime |
| `inputParameters` | No | User payload fields |
| `asyncComplete` | No | Defaults to `false`; when true the task remains `IN_PROGRESS` after publish |
In OSS, registered provider identifiers are `conductor`, `kafka`, `sqs`, `nats`, `jsm`, `nats_stream`, `amqp_queue`, and `amqp_exchange`, subject to the corresponding server module being enabled. The provider owns the destination grammar after the first colon; for example, it might be a Kafka topic, an SQS queue URL, a NATS subject, or an AMQP queue/exchange.
## Conductor sink expansion
- `conductor` becomes `conductor::`.
- `conductor:` becomes `conductor::`.
The event handler must listen on the expanded name.
## Published payload and output
The task begins with its resolved input parameters and adds workflow metadata:
| Field | Value |
|---|---|
| `workflowInstanceId` | Parent workflow execution ID |
| `workflowType` | Parent workflow name |
| `workflowVersion` | Parent version |
| `correlationId` | Parent correlation ID |
| `taskToDomain` | Parent domain map |
The task output also contains `event_produced`, the expanded sink. The published message is the task output without `event_produced`. The Event task uses its task ID as the broker message identity, so consumers can use that stable value for duplicate detection.
## Completion behavior
With `asyncComplete: false`, a successful publish completes the task. With `asyncComplete: true`, publishing succeeds but the task remains `IN_PROGRESS`; an external task update or an event-handler `complete_task`/`fail_task` action must resolve it.
## Example
```json
{
"name": "publish_order_status",
"taskReferenceName": "publish_order_status",
"type": "EVENT",
"sink": "conductor:order-status",
"inputParameters": {
"orderId": "${workflow.input.orderId}",
"status": "READY"
},
"asyncComplete": false
}
```
For a practical first-use walkthrough, see [Publish events](../../../../devguide/how-tos/publish-events.md). Use [Event-Driven Orchestration](../../../../devguide/how-tos/event-bus.md) for the provider matrix, routing, webhooks, signals, and delivery observability.