Workflows
Durable workflow: run dependent steps in your process or on agents, record each finished step on the log, and resume after a restart without repeating finished steps
A workflow runs steps in dependency order and records each finished step on the agent.workflow_journal topic. A step runs in your process, or as a request to an agent that offers a capability. When a run stops part way, starting it again with the same run ID skips the finished steps and continues from the first step that did not finish. Workflows run on plain Apache Iggy, Laser Stack, and LaserData Cloud.
Use a workflow when a job has several dependent steps and must survive a restart, such as ingest, then analyze, then report. For a single request and reply, send the request to the agent directly.
Quick example
This program runs a three-step durable workflow in its own process. Each step builds its output from the output of the step before it. The program prints the run ID, the run's status, the finished steps in order, and the last step's output as JSON. It runs on a scratch stream with a unique name and deletes that stream at the end, so it leaves nothing to clean up.
Set LASER_CONNECTION_STRING before you run it. With an API key instead, add the resolve block below.
// npm init -y && npm pkg set type=module && npm install @laserdata/laser-sdk tsx
// Run with: npx tsx main.ts
import { Laser } from "@laserdata/laser-sdk"
const decode = (bytes?: Uint8Array) => new TextDecoder().decode(bytes)
await using root = await Laser.connectEnv()
await using scratch = await root.scratchStream("workflow")
const outcome = await scratch.laser
.workflow("pipeline")
.stepLocal("ingest", () => "frames from sat-7")
.stepLocal("analyze", ({ outputs }) => `analyzed ${decode(outputs.get("ingest"))}`)
.after("ingest")
.stepLocal("report", ({ outputs }) => `report on ${decode(outputs.get("analyze"))}`)
.after("analyze")
.run()
console.log(
JSON.stringify({
run_id: outcome.runId.toString(),
status: outcome.status,
completed_steps: outcome.completedSteps,
report: outcome.text("report")
})
)Workflows need the agent feature of the laser-sdk crate.
// cargo add laser-sdk --features agent
// cargo add tokio --features macros,rt-multi-thread
// cargo add serde_json --features preserve_order
use laser_sdk::prelude::full::*;
use serde_json::json;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let root = Laser::connect_env().await?;
let scratch = root.scratch_stream("workflow").await?;
let result = run_pipeline(scratch.laser()).await;
scratch.delete().await?;
let outcome = result?;
let result = json!({
"run_id": outcome.run_id.to_string(),
"status": outcome.status,
"completed_steps": outcome.completed_steps,
"report": outcome.text("report"),
});
println!("{result}");
Ok(())
}
async fn run_pipeline(laser: &Laser) -> Result<WorkflowOutcome, LaserError> {
laser
.workflow("pipeline")
.step_local("ingest", |_ctx: &StepContext<'_>| b"frames from sat-7".to_vec())
.step_local("analyze", |ctx: &StepContext<'_>| {
let ingested = ctx.outputs.get("ingest").cloned().unwrap_or_default();
[b"analyzed ".to_vec(), ingested].concat()
})
.after("ingest")
.step_local("report", |ctx: &StepContext<'_>| {
let analyzed = ctx.outputs.get("analyze").cloned().unwrap_or_default();
[b"report on ".to_vec(), analyzed].concat()
})
.after("analyze")
.run()
.await
}A closure step is synchronous and returns Vec<u8>, so an async closure does not compile. A step that awaits is a type that implements StepFn, passed in place of the closure, as in .step_local("analyze", Analyze):
// cargo add async-trait
// cargo add tokio --features time
use std::time::Duration;
struct Analyze;
#[async_trait::async_trait]
impl StepFn for Analyze {
async fn build(&self, ctx: &StepContext<'_>) -> Result<Vec<u8>, LaserError> {
let ingested = ctx.outputs.get("ingest").cloned().unwrap_or_default();
tokio::time::sleep(Duration::from_secs(1)).await;
Ok([b"analyzed ".to_vec(), ingested].concat())
}
}import asyncio
import json
import laser_sdk as ls
async def main():
async with await ls.Laser.connect_env() as root:
async with root.scratch_stream("workflow") as scratch:
outcome = await (
scratch.laser.workflow("pipeline")
.step("ingest", run=lambda outputs: "frames from sat-7")
.step("analyze", after=["ingest"], run=lambda outputs: b"analyzed " + outputs["ingest"])
.step("report", after=["analyze"], run=lambda outputs: b"report on " + outputs["analyze"])
.run()
)
print(json.dumps({
"run_id": outcome.run_id,
"status": outcome.status,
"completed_steps": outcome.completed_steps,
"report": outcome.text("report"),
}))
asyncio.run(main())The output looks like this in every language, with your own run ID. Python separates the fields with a space:
{"run_id":"01JD3W8Q4M6V2X9TQK7R5N1B0C","status":"completed","completed_steps":["ingest","analyze","report"],"report":"report on analyzed frames from sat-7"}A step body gets the outputs of the steps before it and returns its own output as bytes or text. In TypeScript and Python it can be async, and in Rust an async step implements StepFn. It runs inside the step deadline, 30 seconds unless you set one. status is completed for every returned outcome, because a failed, canceled, or over-budget run raises an error instead. completed_steps (TypeScript: completedSteps) lists the finished steps in the order the journal recorded them, and text(label) returns a step's output as text. The run creates the journal and session topics on its first write, so the program needs no bootstrap. The scratch stream is deleted with the journal and every other topic when its scope ends. To keep the journal so a run can resume, use a named stream, as in Resume a stopped run. To delete a stream yourself, call laser.stream(name).delete().
Run it with an API key
With a LaserData API key in LD_API_KEY instead of a connection string, add this block to the program. It turns the key into the connection string in three calls: the key's organization from /tenants, the deployments the key can read from /tenants/{tenant_id}/deployments, and the login from the deployment's /credentials. Set LD_DEPLOYMENT to the deployment's name or ID when the key sees more than one deployment. The block then exits and lists the ones it sees. The key's role needs deployment:read and deployment:credentials:read on the deployment's environment. The block keeps the connection string in memory and never prints it.
// Paste after the imports. It sets LASER_CONNECTION_STRING from LD_API_KEY when it is not set.
const API_URL = process.env.LD_API_URL ?? "https://api.laserdata.cloud"
async function laserdata(url: string): Promise<any> {
const headers = { "ld-api-key": process.env.LD_API_KEY ?? "", "user-agent": "laserdata-docs-example" }
const response = await fetch(url, { headers })
if (!response.ok) throw new Error(`GET ${url} answered ${response.status}: ${await response.text()}`)
// IDs are 64-bit integers, so keep each number JavaScript cannot hold exactly as its source text.
return JSON.parse(await response.text(), (_key, value, context?: { source?: string }) =>
typeof value === "number" && !Number.isSafeInteger(value) ? context?.source : value
)
}
async function connectionString(): Promise<string> {
const [tenant] = await laserdata(`${API_URL}/tenants`)
const deployments: any[] = []
for (let page = 1, totalPages = 1; page <= totalPages; page++) {
const reply = await laserdata(`${API_URL}/tenants/${tenant.id}/deployments?results=100&page=${page}`)
deployments.push(...reply.items)
totalPages = reply.total_pages
}
const wanted = process.env.LD_DEPLOYMENT
const matches = deployments.filter(
(deployment) => !wanted || deployment.name === wanted || String(deployment.id) === wanted
)
if (matches.length !== 1) {
const visible = deployments.map((deployment) => `${deployment.name} (${deployment.id})`).join(", ")
console.error(`Set LD_DEPLOYMENT to the name or ID of one deployment. Visible: ${visible || "none"}`)
process.exit(1)
}
const [deployment] = matches
if (!deployment.supervisor_url) throw new Error(`Deployment ${deployment.name} has no supervisor_url yet`)
const credentials = await laserdata(`${deployment.supervisor_url}/deployments/${deployment.id}/credentials`)
return credentials.connection_string
}
process.env.LASER_CONNECTION_STRING ||= await connectionString()// cargo add reqwest --features json
// cargo add serde_json
// Add these two functions, then connect with Laser::connect(&connection_string().await?) in main.
use serde_json::Value;
async fn laserdata(client: &reqwest::Client, url: &str) -> Result<Value, Box<dyn std::error::Error>> {
let key = std::env::var("LD_API_KEY")?;
let response = client.get(url).header("ld-api-key", key).header("user-agent", "laserdata-docs-example").send().await?;
let status = response.status();
if !status.is_success() {
return Err(format!("GET {url} answered {status}: {}", response.text().await?).into());
}
Ok(response.json().await?)
}
async fn connection_string() -> Result<String, Box<dyn std::error::Error>> {
if let Ok(value) = std::env::var("LASER_CONNECTION_STRING") {
return Ok(value);
}
let api_url = std::env::var("LD_API_URL").unwrap_or_else(|_| "https://api.laserdata.cloud".to_owned());
let client = reqwest::Client::new();
let tenants = laserdata(&client, &format!("{api_url}/tenants")).await?;
let mut deployments = Vec::new();
for page in 1_u64.. {
let url = format!("{api_url}/tenants/{}/deployments?results=100&page={page}", tenants[0]["id"]);
let reply = laserdata(&client, &url).await?;
deployments.extend(reply["items"].as_array().cloned().unwrap_or_default());
if page >= reply["total_pages"].as_u64().unwrap_or(0) {
break;
}
}
let wanted = std::env::var("LD_DEPLOYMENT").ok();
let matches: Vec<&Value> = deployments
.iter()
.filter(|deployment| wanted.as_deref().is_none_or(|wanted| deployment["name"] == wanted || deployment["id"].to_string() == wanted))
.collect();
let [deployment] = matches.as_slice() else {
let visible: Vec<String> = deployments.iter().map(|deployment| format!("{} ({})", deployment["name"].as_str().unwrap_or_default(), deployment["id"])).collect();
eprintln!("Set LD_DEPLOYMENT to the name or ID of one deployment. Visible: {}", visible.join(", "));
std::process::exit(1);
};
let supervisor_url = deployment["supervisor_url"].as_str().ok_or("The deployment has no supervisor_url yet")?;
let credentials = laserdata(&client, &format!("{supervisor_url}/deployments/{}/credentials", deployment["id"])).await?;
Ok(credentials["connection_string"].as_str().ok_or("The credentials answer has no connection_string")?.to_owned())
}# Paste after the imports. It sets LASER_CONNECTION_STRING from LD_API_KEY when it is not set.
import json
import os
import urllib.request
API_URL = os.environ.get("LD_API_URL", "https://api.laserdata.cloud")
def laserdata(url):
# The API's edge protection refuses urllib's default User-Agent with 403 and error code 1010.
headers = {"ld-api-key": os.environ["LD_API_KEY"], "user-agent": "laserdata-docs-example"}
with urllib.request.urlopen(urllib.request.Request(url, headers=headers), timeout=30) as response:
return json.load(response)
def connection_string():
tenant_id = laserdata(f"{API_URL}/tenants")[0]["id"]
deployments, page, total_pages = [], 1, 1
while page <= total_pages:
reply = laserdata(f"{API_URL}/tenants/{tenant_id}/deployments?results=100&page={page}")
deployments += reply["items"]
page, total_pages = page + 1, reply["total_pages"]
wanted = os.environ.get("LD_DEPLOYMENT")
matches = [deployment for deployment in deployments if not wanted or wanted in (deployment["name"], str(deployment["id"]))]
if len(matches) != 1:
visible = ", ".join(f"{deployment['name']} ({deployment['id']})" for deployment in deployments)
raise SystemExit(f"Set LD_DEPLOYMENT to the name or ID of one deployment. Visible: {visible or 'none'}")
deployment = matches[0]
if not deployment.get("supervisor_url"):
raise SystemExit(f"Deployment {deployment['name']} has no supervisor_url yet")
credentials = laserdata(f"{deployment['supervisor_url']}/deployments/{deployment['id']}/credentials")
return credentials["connection_string"]
if not os.environ.get("LASER_CONNECTION_STRING"):
os.environ["LASER_CONNECTION_STRING"] = connection_string()The block calls nothing else: it never changes access rules and never needs info:read. A Free deployment admits every address, so the program connects at once. A paid deployment admits only the addresses its access rules allow, and a connect from any other address times out naming the Iggy server to accept the connection. Agent access shows how to admit an address, and When no deployment exists shows how to create one.
Run steps on agents
A step can also be a request to an agent that offers a capability, so the work runs in another process or on another machine. This program starts one worker agent and runs the same three-step pipeline against it on the named stream telemetry. To resume a run, set RUN_ID to the run ID it printed. Local and routed steps mix in one workflow. The program keeps the stream so a run can resume. Delete it when you are done with laser.stream("telemetry").delete().
// npm init -y && npm pkg set type=module && npm install @laserdata/laser-sdk tsx
// Run with: npx tsx main.ts
import {
Agent,
AgentId,
AgentTopic,
ConversationId,
Laser,
TopicRetention,
WorkflowBudget,
agentMessageBody,
routeToCapable
} from "@laserdata/laser-sdk"
const text = new TextEncoder()
const decode = (bytes: Uint8Array) => new TextDecoder().decode(bytes)
const step = () => routeToCapable("telemetry-step", { kind: "any" })
await using root = await Laser.connectEnv()
const laser = root.withDefaultStream("telemetry")
await laser.bootstrap(1, TopicRetention.expireAfter(86_400_000))
await using worker = Agent.builder()
.id(AgentId.new("telemetry-worker"))
.listenOn(AgentTopic.Sessions)
.respondOn(AgentTopic.Sessions)
.capabilities([{ skillId: "telemetry-step" }])
.ackOnPickup()
.handler({
handle: (message, ctx) =>
ctx.respond(text.encode(`done: ${decode(agentMessageBody(message))}`))
})
.build()
.spawn(laser)
await worker.ready()
let workflow = laser
.workflow("telemetry-pipeline")
.inboxRoute({ kind: "fixed", topic: AgentTopic.Sessions })
.budget(WorkflowBudget.unlimited().invocations(6).wallClock(60_000))
if (process.env.RUN_ID) workflow = workflow.runId(ConversationId.parse(process.env.RUN_ID))
const outcome = await workflow
.step("ingest", step(), () => text.encode("ingest frames from sat-7"))
.step("analyze", step(), ({ outputs }) =>
text.encode(`analyze ${decode(outputs.get("ingest") ?? new Uint8Array())}`)
)
.after("ingest")
.step("report", step(), ({ outputs }) =>
text.encode(`report ${decode(outputs.get("analyze") ?? new Uint8Array())}`)
)
.after("analyze")
.run()
const report = decode(outcome.outputs.get("report") ?? new Uint8Array())
console.log(`Run ${outcome.runId.toString()} finished: ${report}`)// cargo add laser-sdk --features agent
// cargo add tokio --features macros,rt-multi-thread
use laser_sdk::prelude::full::*;
use laser_sdk::wire::agent::CapabilityDescriptor;
use std::time::Duration;
struct Worker;
impl AgentHandler for Worker {
async fn handle(&self, message: &AgentMessage, ctx: &AgentCtx<'_>) -> Result<(), LaserError> {
let task = String::from_utf8_lossy(message.body());
ctx.respond(format!("done: {task}")).await
}
}
#[tokio::main]
async fn main() -> Result<(), LaserError> {
let laser = Laser::connect_env().await?.with_default_stream("telemetry");
laser
.bootstrap(1, TopicRetention::expire_after(Duration::from_secs(86_400)))
.await?;
let mut worker = Agent::builder()
.id("telemetry-worker".parse()?)
.listen_on(AgentTopic::Sessions)
.respond_on(AgentTopic::Sessions)
.capabilities(vec![CapabilityDescriptor {
skill_id: "telemetry-step".to_owned(),
..Default::default()
}])
.ack_on_pickup(true)
.handler(Worker)
.build()
.spawn(laser.clone());
worker.ready().await?;
let step = || Router::to_capable("telemetry-step", RoutePolicy::Any);
let mut workflow = laser
.workflow("telemetry-pipeline")
.inbox_route(InboxRoute::Fixed(AgentTopic::Sessions))
.budget(WorkflowBudget::unlimited().invocations(6).wall_clock(Duration::from_secs(60)));
if let Ok(run_id) = std::env::var("RUN_ID") {
workflow = workflow.run_id(run_id.parse()?);
}
let outcome = workflow
.step("ingest", step(), |_ctx: &StepContext<'_>| b"ingest frames from sat-7".to_vec())
.step("analyze", step(), |ctx: &StepContext<'_>| {
let ingested = ctx.outputs.get("ingest").cloned().unwrap_or_default();
[b"analyze ".to_vec(), ingested].concat()
})
.after("ingest")
.step("report", step(), |ctx: &StepContext<'_>| {
let analyzed = ctx.outputs.get("analyze").cloned().unwrap_or_default();
[b"report ".to_vec(), analyzed].concat()
})
.after("analyze")
.run()
.await?;
let report = outcome.outputs.get("report").cloned().unwrap_or_default();
println!(
"Run {} finished: {}",
outcome.run_id,
String::from_utf8_lossy(&report)
);
worker.shutdown().await
}import asyncio
import os
import laser_sdk as ls
async def work(ctx, message):
await ctx.respond(b"done: " + bytes(message.body()))
async def main():
async with await ls.Laser.connect_env() as root:
laser = root.with_default_stream("telemetry")
await laser.bootstrap(1, ls.TopicRetention.expire_after(86_400_000))
async with laser.spawn_agent(
"telemetry-worker",
ls.AgentTopic.Sessions,
work,
respond_on=ls.AgentTopic.Sessions,
capabilities=["telemetry-step"],
ack_on_pickup=True,
):
workflow = laser.workflow("telemetry-pipeline", fixed_inbox=ls.AgentTopic.Sessions)
workflow.budget(ls.WorkflowBudget.unlimited().invocations(6).wall_clock(60_000))
if os.environ.get("RUN_ID"):
workflow.run_id(os.environ["RUN_ID"])
workflow.step(
"ingest",
to_capable="telemetry-step",
build=lambda outputs: b"ingest frames from sat-7",
)
workflow.step(
"analyze",
to_capable="telemetry-step",
after=["ingest"],
build=lambda outputs: b"analyze " + outputs["ingest"],
)
workflow.step(
"report",
to_capable="telemetry-step",
after=["analyze"],
build=lambda outputs: b"report " + outputs["analyze"],
)
outcome = await workflow.run()
report = bytes(outcome.outputs["report"]).decode()
print(f"Run {outcome.run_id} finished: {report}")
asyncio.run(main())The output looks like this, with your own run ID. Each step's reply wraps the request it got:
Run 01JD3W8Q4M6V2X9TQK7R5N1B0C finished: done: report done: analyze done: ingest frames from sat-7Common mistakes
- One connection advertises one agent. A second agent spawned on the same connection fails with
PresenceConflictError(connection already advertises agent ...). In Python,async with laser.spawn_agent(...)andready()report it asagent stopped before ready, andshutdown()then raises thePresenceConflictError. Serve every step from one agent whose handler reads the request, as the example does, or open one connection per agent. - A worker replies on the topic it responds on. Without one, the agent refuses
ack_on_pickupwithack_on_pickup requires respond_on, or a step fails because the agent has no respond_on topic configured. Set it when you build the agent:respond_onin Python,respondOnin TypeScript,respond_onin Rust. - In Python,
message.body()is a method that returns the task body.message.payloadis the whole encoded message, so decoding it as text fails.
Resume a stopped run
Every run has a run ID. The run writes each finished step to agent.workflow_journal under that ID, so a run resumes only on a stream that outlives the process, not on a scratch stream. When the process stops between steps, start it again on the same stream with the same run ID, as the agent program above does with RUN_ID:
- A step recorded as finished does not run again. Its stored output feeds the next step.
- A step that runs in your process and stopped before its finished record runs again, so it runs at least once.
- A step that was sent but has no recorded reply is answered by its reply when one landed. Otherwise it is sent again with the same idempotency key. A worker can drop a duplicate while its deduplication record still exists. The default in-memory deduplication window does not survive every worker restart, so the handler must tolerate redelivery.
- A step copy that a worker dropped because its deadline had passed never ran, so it does not stop the resent step from running. That holds on a worker that refills its deduplication window on restart too, unless its warm dedup policy is
all. - When the worker's process stopped too, the streaming server keeps that worker's partitions until it ends the stopped process's session, 36 seconds or more after the stop with default server settings. A run resumed sooner can fail with a
timed_outstep error. Resume it again with the same run ID and the step runs once. - A run whose journal was cut by topic retention fails with a journal truncated error instead of running every step again.
The journal records what the workflow did. An agent that changes something outside the log, such as a payment or an email, still needs its own protection against doing it twice. Make the step exclusive and use a fenced write, or key the external call with the request's idempotency key.
When a step fails
A step fails when its agent does not reply before the step deadline (30 seconds unless you set one), when the reply is an error, or when a verifier rejects the output. A step in your process fails when its body runs past the deadline, and an error its body raises comes back unchanged. The run then fails with a step failed error that names the run, the step, and the reason, and for a timed-out step also the target and the deadline in milliseconds. Steps that already finished run their compensations in reverse order. Only a timed_out failure is worth retrying as is.
| Error | Rust | TypeScript and Python | What to do |
|---|---|---|---|
| Step failed | LaserError::StepFailed | StepFailedError | Read the reason, fix the agent or the input, resume with the same run ID |
| Budget exceeded | LaserError::BudgetExceeded | BudgetExceededError | Raise the budget, or split the run |
| Journal truncated | LaserError::JournalTruncated | JournalTruncatedError | Start a new run, and keep the journal topic longer than your longest run |
| No capable agent | LaserError::NoCapableAgent | NoCapableAgentError | Start an agent that offers the capability and wait for ready(), then resume with the same run ID |
When no agent offers the capability a step asks for, the run fails at once with the no capable agent error instead of waiting for the step deadline. Start the agents first and wait for ready(), as the example does.
Next steps
Memory
Long-term agent memory: remember facts about a user, supersede a fact that changed in a later conversation, recall the current facts from any agent, and trace each one back to the log
Sessions
Record every agent interaction as a durable session you can inspect, replay, resume after a restart, and control