Orloj: Agent Infrastructure as Code
Review Orloj's infrastructure-as-code approach to multi-agent systems: declarative resources, governance, workers, retries, isolation, and trade-offs.
Orloj: Agent Infrastructure as Code
An agent prototype is usually a loop, a model client, and a few tools. A production agent system also needs identities, permissions, secrets, schedules, retry rules, worker ownership, budgets, audit trails, and a way to answer “what is running right now?” Orloj’s thesis is that these concerns should be versioned resources, not another thousand lines of application glue.
Orloj is an open-source orchestration runtime written in Go. You declare agents, models, tools, policies, tasks, and graphs in YAML; an API server stores desired state and schedules work; workers claim tasks and execute them under runtime policy. The Kubernetes analogy is intentional, but it should not be taken literally: Orloj borrows the control-plane mindset, not the container orchestration ecosystem.
TL;DR
- Orloj is an operations layer for agent systems, not a model SDK or a better prompting library.
- Its strongest ideas are declarative resources, fail-closed tool policy, lease-based task ownership, idempotency, dead-letter states, and inspectable traces.
- It fits teams operating recurring, governed multi-agent workloads better than teams still exploring one conversational agent.
- The project is pre-1.0 and under active development; schemas and APIs may change.
- YAML centralizes intent, but it does not remove the need to test tools, policies, retries, and model behavior end to end.
What Orloj actually runs
The Orloj documentation divides the runtime into three operational pieces:
orlojdhosts the REST API, resource store, scheduler, and background services.orlojworkerinstances claim and execute tasks, route model requests, and run tools.orlojctlapplies manifests, submits work, and inspects resources, logs, graphs, traces, and events.
Orloj separates an API and scheduler from one or more task workers, with an embedded-worker mode for development. Source: Orloj Docs.
For local development, the server can use in-memory storage and an embedded worker. A production-shaped deployment can use PostgreSQL for state and NATS JetStream for message-driven workers. That progression is useful because the manifest does not need to become a different application merely to move beyond a laptop.
Resources instead of framework objects
Orloj’s initial release exposed 15 resource kinds. The current stack goes further, but the core groups remain recognizable:
| Concern | Representative resources |
|---|---|
| Execution | Agent, AgentSystem, Task, Worker |
| Models and tools | ModelEndpoint, Tool, McpServer |
| State and credentials | Memory, Secret, SealedSecret |
| Governance | AgentPolicy, AgentRole, ToolPermission, ToolApproval |
| Triggers | TaskSchedule, TaskWebhook |
A resource has versioned metadata, desired specification, and observable status. That gives platform teams a stable place to review changes. A pull request can show that a research agent gained one web tool, its token cap rose, or a workflow added a review edge without hiding the decision in Python control flow.
The resource model also creates a common vocabulary for the console, CLI, API, and SDKs. The official Python SDK can apply graph-as-code definitions and watch tasks over server-sent events, but the resulting objects still live in the Orloj control plane rather than inside one Python process.
A production task is a distributed-systems problem
The feature that separates Orloj from many agent frameworks is not the DAG. Graph execution is common. The more consequential features are lease-based claiming, capped retries with jitter, idempotency tracking, and dead-letter transitions.
Imagine a worker claims a task, invokes an external API, and dies before recording completion. Another worker eventually receives the expired lease. If the runtime simply repeats every step, the external action may happen twice. “Retry” is not reliability unless side effects have an idempotency boundary.
Orloj can track task identity and execution state, but tool authors still own the last mile. A tool that creates a ticket, sends an email, or pushes a commit should accept an idempotency key or perform a read-before-write check. The runtime cannot infer whether an arbitrary shell command is safe to replay.
The repository frames agent teams as distributed systems with worker ownership, retries, idempotency, and dead-letter handling. Source: OrlojHQ/orloj.
Lease duration also needs tuning. Too short, and a slow model call looks like a dead worker; too long, and genuine failures take ages to recover. Traces should distinguish an original attempt from a replay and preserve the lease owner responsible for each side effect.
Governance is enforced in the execution path
Orloj lets teams declare which tools an agent may use, which models are allowed, and how tokens, cost, steps, and time are bounded. The important claim is fail-closed enforcement: an unauthorized call is rejected at runtime rather than merely discouraged by the system prompt.
That is the correct layer. A prompt saying “do not write files” competes with every instruction the model later reads. A runtime that never exposes file_write, or rejects the call before execution, removes that decision from the model.
A sensible role split might look like this:
- researcher: allowlisted network reads, no shell;
- analyst: no external tools, only upstream artifacts;
- developer: scoped repository tools inside a sandbox;
- reviewer: read-only diff and test output;
- publisher: one gated write API with mandatory approval.
Policy must cover both synchronous and message-driven execution paths. The public Go package documentation explicitly notes that policy enforcement is required in both, which is the sort of invariant worth adding to an integration test rather than trusting as architecture prose.
Tool isolation has several levels
Orloj documents container, WASM, and sandboxed tool execution. MCP servers can run in containers with a read-only filesystem, dropped Linux capabilities, and CPU or memory limits. Secrets can be exposed as mounted files instead of environment variables when a server requires them.
These are useful primitives, not a security guarantee. A container with unrestricted egress can still exfiltrate its inputs. A read-only root filesystem does not prevent writes to mounted workspaces. The deployment needs explicit network policy, narrow mounts, non-root users, image pinning, and secret scoping.
The addition of SealedSecret is operationally meaningful because encrypted manifests can live in Git and be reconciled into runtime secrets. It also creates key-management duties: back up and rotate the sealing key, restrict the unseal path, and define what happens when the key is lost.
Graphs are useful, but loops need hard boundaries
Orloj supports pipelines, hierarchies, fan-out/fan-in, and swarm-style loops. Pipelines are easiest to reason about. Fan-out can reduce wall time when branches are independent. Loops are the dangerous shape because a model evaluates whether another iteration is necessary.
Every loop should have a deterministic maximum iteration count plus step, token, cost, and wall-clock caps. Its exit decision should be recorded as data. For a code-review loop, prefer “tests pass and no blocking findings remain” over a free-form “looks good” signal.
The same principle applies to joins. Define whether one failed branch blocks all synthesis, produces a partial result, or enters a dead-letter state. Silent partial success is especially dangerous when downstream agents assume they received a complete evidence set.
The changelog is part of the documentation
Orloj is explicitly pre-1.0. Its changelog shows fast expansion: container-backed MCP, ephemeral sessions, sealed secrets, SDKs, evaluation, console features, and fixes for provider-specific tool-call histories.
The changelog exposes both new operational primitives and fixes to model-specific execution behavior. Source: Orloj Changelog.
Pin the server, worker, CLI, and SDK versions together. Run manifest validation during upgrade, then replay a representative set of tasks in a non-production namespace. Provider adapters deserve regression tests for streaming, tool errors, rejected approvals, and interrupted multi-step calls.
Orloj versus an agent framework
LangGraph, Microsoft Agent Framework, CrewAI, and OMA help application developers express agent behavior and collaboration. Orloj is aiming at the operational plane around that behavior. The overlap is real—Orloj has graphs, tools, and model routing—but its differentiator is fleet lifecycle and governance.
If a Python team is still changing prompts and graph structure daily, an in-process framework is faster. If multiple teams need common identity, policy, scheduling, cost attribution, and worker operations, a separate control plane begins to earn its complexity.
This also explains how Orloj differs from Open Multi-Agent’s runtime DAGs. OMA drops into a TypeScript application and makes dynamic task graphs inspectable. Orloj asks teams to operate agent systems as declarative platform resources.
A responsible adoption path
Begin with one low-risk recurring workflow and an embedded worker. Write manifests for agents, tools, limits, and policy; keep high-impact tools unavailable. Capture a golden set of inputs and expected structural outcomes.
Next, separate the worker and switch to persistent storage. Kill workers during model calls and tool execution. Confirm lease recovery, idempotency, trace continuity, and dead-letter behavior. Add one approval-gated write tool only after read-only flows are reliable.
Finally, test an upgrade by replaying the golden set and comparing cost, steps, denied calls, outputs, and latency. The console is helpful for humans, but these checks should also run automatically.
FAQ
Is Orloj another Kubernetes distribution?
No. It adopts declarative resources and control-plane ideas, but it is a purpose-built agent runtime with its own server, workers, scheduler, and policies.
Does Orloj write the agents for me?
No. You define agent instructions, tools, models, graphs, and policies. Orloj operates and governs their execution.
Is it ready for stable production APIs?
The project says it is under active development and schemas may change before 1.0. Production evaluation is possible, but version pinning and upgrade tests are essential.
Do retries make tool calls exactly once?
No generic runtime can guarantee exactly-once side effects for arbitrary tools. Use idempotency keys and design every mutating tool for replay.
When is Orloj too much infrastructure?
When you have one short-lived agent, little governance need, and no recurring operational burden. An in-process framework will usually be simpler.
Bottom line
Orloj’s useful insight is that prompts and graphs are only a fraction of a production agent system. Ownership, permissions, retries, secrets, costs, and audit evidence need an operational home too.
Declarative YAML makes that home reviewable; it does not make it correct automatically. The platform earns trust when teams test policy enforcement and failure recovery with the same seriousness as model quality.
