How Duolingo Built Their Production Agent Platform

TL;DR — Duolingo’s engineering team was rebuilding the same infrastructure—observability, retries, orchestration, evaluation—for every new agent project. Their solution: a shared platform where you define an agent once with a declarative spec, and the platform handles execution, orchestration, observability, and evaluation. The key architectural bet: treat agent runs as durable Temporal workflows, not one-off processes. This decouples agent definitions from execution runtimes, letting teams swap between Claude Agents SDK, Codex CLI, or OpenAI Agents SDK without rewriting their agent logic.

The third time I watched a team at a large company rebuild agent retry logic from scratch, I knew something was broken. Not with the models—they’re good enough now. Not with the frameworks—there are plenty. The problem was that every team starting an agent project had to solve the same infrastructure problems: how do I make this thing durable? How do I observe what it’s doing? How do I evaluate whether it’s working? How do I orchestrate multi-step runs that might take minutes or hours?

Duolingo hit this wall too. On August 4, 2026, they published a detailed engineering blog post explaining how they solved it. The post is a blueprint for any organization running more than two agent projects in production. What they built isn’t a new framework—it’s a platform layer that sits between declarative agent definitions and the messy reality of production execution.

The Problem: Infrastructure Groundhog Day

Duolingo’s AI teams had been shipping agents for various internal and product-facing use cases. Each project was successful on its own, but when the engineering leadership looked across projects, a pattern emerged: every team was independently solving the same problems.

Team A built retry logic and state persistence for their content generation agent. Team B built nearly identical retry logic and state persistence for their curriculum evaluation agent. Team C did the same for their localization agent. Each solution was slightly different, slightly incompatible, and slightly under-maintained once the team moved on.

The specific problems being re-solved:

• Durable execution: Agents that run for minutes or hours need to survive process restarts, network blips, and deployment rollovers
• Observability: What is the agent doing right now? What tools did it call? What did each step cost?
• Orchestration: How do you coordinate an agent with upstream data sources, downstream consumers, and human approval gates?
• Evaluation: Is this agent actually working? How do you measure that continuously, not just at launch?
• Cost tracking: Which team’s agent just burned through $400 in API calls at 3am?

The usual response to this pattern is “let’s write a shared library.” Duolingo went further. They built a platform.

The Solution: Define Once, Run Anywhere

The core insight in Duolingo’s design is a clean separation between what an agent is and how it executes. They call their system the Agent Platform, and it has two halves:

1. Declarative agent definitions — a structured spec that describes the agent’s identity, capabilities, and constraints
2. A platform runtime — the execution engine that takes a definition and runs it with full production infrastructure

An agent definition looks like this:

name: curriculum-evaluator
description: "Evaluates lesson sequences for pedagogical coherence and difficulty progression"
system_prompt: |
  You are an expert in language pedagogy and curriculum design.
  Evaluate the provided lesson sequence for...
model: claude-sonnet-4-20250514
mcp_servers:
  - curriculum-api
  - learner-data
output_type: structured

That’s it. No retry logic. No state management. No observability wiring. No deployment configuration. The agent author focuses on what the agent should do—its identity, its tools, its output contract. The platform handles everything else.

This is the same principle behind Kubernetes (declare what you want, the system figures out how to run it) applied to agent execution. But where Kubernetes orchestrates containers, Duolingo’s platform orchestrates agent runs.

Architecture: Temporal as the Backbone

The heart of the platform is Temporal, a workflow engine originally developed at Uber. If you’re not familiar with Temporal, here’s the short version: it lets you write code that looks like normal sequential logic but is actually durable—if the process crashes mid-execution, it picks up exactly where it left off.

Here’s why Temporal was the right fit for agent orchestration:

Agent Need	Temporal Capability
Agent runs that survive crashes	Durable execution with automatic replay
Retrying failed tool calls	Built-in retry policies per activity
Long-running agents (minutes to hours)	Workflow timers and heartbeats
Coordinating with external systems	Signal and query APIs
Observability into agent progress	Event history and workflow state
Timeout and budget enforcement	Workflow and activity timeouts

The platform defines an AgentWorkflow — a Temporal workflow that follows a consistent lifecycle:

AgentWorkflow: loads definition → prepares environment → runs agent → returns output

In more detail:

1. Load definition — Fetch the declarative agent spec from the registry
2. Prepare environment — Spin up MCP servers, establish connections, configure the model endpoint through their LLM Gateway
3. Run agent — Execute the agent using the appropriate runtime (more on this below)
4. Return output — Validate output against the declared output_type, persist results, emit metrics

Each step is a Temporal activity, meaning each step has its own retry policy, timeout, and observability. If the MCP server for curriculum-api is temporarily down at step 2, Temporal retries it with backoff. If the LLM call at step 3 times out, it retries. If the entire workflow is interrupted by a deployment, it resumes from wherever it stopped.

This is fundamentally different from the typical “start a process, hope it finishes” approach that most agent deployments use. As we’ve discussed in why production agents need a runtime layer, the gap between a working demo and a production agent is exactly this kind of durable execution infrastructure.

Multi-Runtime Support: The Smart Bet

Here’s where Duolingo’s design gets interesting. The platform doesn’t commit to a single agent execution runtime. It supports multiple:

• OpenAI Agents SDK — their primary runtime for most production agents
• Claude Agents SDK — for agents that benefit from Anthropic’s models
• Codex CLI — for code-generation and repository-manipulation agents

The declarative definition is runtime-agnostic. The platform decides which runtime to use based on the definition’s model field and configuration. This means a team can switch an agent from OpenAI to Claude by changing one line in their definition—the platform handles the rest.

This decoupling is crucial for a fast-moving field. In 2024, committing to one SDK meant rewriting when something better appeared six months later. Duolingo’s architecture lets them adopt new runtimes without touching agent definitions. When a new provider ships an SDK with better tool-use performance, they add a runtime adapter and every agent can benefit.

OpenAI Agents SDK Integration: The Details

The blog post goes deep on their OpenAI Agents SDK integration because it’s their most-used runtime. Two details stood out:

MCP tool calls as Temporal activities. When an agent calls an MCP tool (like querying the curriculum API), that call isn’t just a function invocation—it’s a Temporal activity. This means every tool call gets:

• Independent retry with configurable backoff
• Timeout enforcement
• Full observability (duration, result, errors) in the Temporal UI
• Automatic replay if the workflow restarts

This is a level of operational control you simply don’t get from running an agent in a bare process.

LLM Gateway proxy for cost and usage tracking. All model calls route through Duolingo’s internal LLM Gateway, which acts as a proxy between the agent runtime and model providers. The gateway handles:

• Per-team and per-agent cost attribution
• Usage quotas and rate limiting
• Model routing (A/B testing different models)
• Request/response logging for debugging

This solves the “who ran up the bill” problem that every organization hits when agents go from prototype to production.

Agent Evaluation: Continuous, Not One-Off

Most teams evaluate their agents once—during development—and then hope nothing drifts. Duolingo built evaluation into the platform as a first-class concern.

Their eval system works like this:

1. Author scenarios — Define test cases with specific inputs and expected behaviors
2. Run real agents — Execute the actual agent (not a mock) against these scenarios
3. Capture outputs and diffs — Record what the agent produced, including any file changes or state mutations
4. Grade with structured assertions — Evaluate outputs against predefined criteria using both deterministic checks and LLM-as-judge

The key principle: evals run the real agent in a real environment. No mocks, no stubs, no simplified versions. This means the eval catches problems that unit tests miss—like an MCP server returning unexpected data, or a model producing valid-but-wrong structured output.

Evals run on a schedule and on every agent definition change, giving teams continuous confidence that their agents still work. When a model provider updates their model, evals catch regressions before users do.

Key Design Patterns

Looking across Duolingo’s platform, several patterns emerge that are applicable to any organization building agent infrastructure:

Pattern	What It Means	Why It Matters
Declarative definitions	Agent spec is data, not code	Enables tooling, validation, version control, and non-engineer authoring
Durable workflows	Agent runs survive failures	Agents can run for hours without fear of losing progress
Runtime abstraction	Definition ≠ execution	Swap SDKs without rewriting agents; adopt new capabilities incrementally
Tool calls as activities	Each tool invocation is independently managed	Retry, timeout, and observe at the tool-call level
Gateway-mediated LLM access	All model calls go through a proxy	Cost attribution, rate limiting, routing, and logging in one place
Continuous evaluation	Evals are automated infrastructure, not manual testing	Catch regressions from model updates, prompt drift, and data changes

What This Means for the Industry

Duolingo’s platform validates a thesis we’ve been tracking: the future of production AI agents isn’t better frameworks—it’s better infrastructure. The models are good enough. The frameworks are good enough. What’s missing is the production layer that makes agents reliable, observable, and maintainable at organizational scale.

This is the same trajectory that web development followed. Individual developers didn’t need Kubernetes. But organizations running hundreds of services did. Similarly, individual developers building one agent don’t need a platform. But organizations running dozens of agents across multiple teams absolutely do.

The specific architectural choices—Temporal for durability, declarative definitions for separation of concerns, multi-runtime support for flexibility—aren’t the only valid choices. But the pattern of decoupling definition from execution and treating agent runs as durable workflows rather than one-off processes? That’s going to be the standard approach. If you’re exploring open-source agent frameworks, consider how they compose with this kind of infrastructure layer.

FAQ

Q: Do I need Temporal specifically to build something like this?

No. Temporal is one option for durable workflow execution. Alternatives include Restate, Inngest, and Hatchet. The key requirement is durable execution with activity-level retries and observability. Temporal happens to be battle-tested at scale (Uber, Netflix, Snap), which matters when you’re betting your agent infrastructure on it.

Q: How is this different from just using LangGraph or CrewAI?

Frameworks like LangGraph handle orchestration within a single agent’s reasoning loop. Duolingo’s platform handles orchestration around the agent—lifecycle management, retries, evaluation, cost tracking, multi-runtime support. They’re complementary layers. You could use LangGraph as one of the runtimes inside this kind of platform.

Q: Can small teams benefit from this pattern?

The full platform is overkill for a team with one or two agents. But the core insight—treat agent runs as durable workflows, not scripts—applies at any scale. Even a single agent benefits from Temporal-style durability if it runs for more than a few minutes or calls unreliable external services.

Q: How do they handle secrets and credentials for MCP servers?

The blog post mentions that environment preparation (step 2 in the AgentWorkflow) handles credential injection. The agent definition references MCP servers by name; the platform resolves those names to running instances with appropriate credentials at runtime. Agent authors never handle secrets directly.

Q: What’s the cold-start time for spinning up an agent?

The post doesn’t give exact numbers, but notes that MCP server pools are kept warm for frequently-used agents. For rarely-used agents, there’s a setup cost at the “prepare environment” step that adds seconds to the first run.

The Bottom Line

Duolingo’s agent platform isn’t revolutionary in any single dimension. Temporal isn’t new. Declarative definitions aren’t new. Multi-runtime support isn’t new. What’s compelling is the composition: taking proven infrastructure patterns and applying them specifically to the problem of running AI agents in production at organizational scale.

The key insight worth internalizing: treat agent runs as durable workflows, not one-off processes. Everything else—the declarative definitions, the multi-runtime support, the continuous evaluation—flows from that foundational decision. Once you decide that an agent run is a workflow that must be durable, observable, and recoverable, the rest of the architecture designs itself.

If you’re at an organization where multiple teams are building agents and you’re seeing the same infrastructure being rebuilt for each one, Duolingo just published your playbook.