Explanation: architecture and design rationale --- # Explanation > Background on Everruns, why durable execution, why events are the primary store, how the agentic loop and capability layering fit together. Source: These pages discuss *why* Everruns is shaped the way it is. They don’t tell you how to do anything (see [How-to guides](https://docs.everruns.com/how-to/)) and they aren’t lookup tables (see [Reference](https://docs.everruns.com/api/)). They’re here to give you a mental model so the other docs make sense. Read these when: * You’re evaluating Everruns and want to understand the design. * A reference page tells you *what* something is but you want to know *why* it exists. * You’re about to make an architectural decision and want to know which guarantees you can rely on. ## Topics * [Core concepts](https://docs.everruns.com/explanation/concepts/), the entity model: harnesses, agents, sessions, capabilities, and how they compose into a runtime. * [The agentic loop](https://docs.everruns.com/explanation/agentic-loop/), the reason–act cycle, execution phases, and why turns are bounded. * [Architecture](https://docs.everruns.com/explanation/architecture/), control plane, workers, and the API-first design. * [Durable execution](https://docs.everruns.com/explanation/durable-execution/), why agents survive crashes, and the trade-offs of a PostgreSQL-backed engine. * [Events as the primary store](https://docs.everruns.com/explanation/events/), why an append-only event log is the source of truth, not a side-channel. --- # The agentic loop > Why agent execution is a bounded reason–act cycle, what execution phases mean, and how multi-step tool flows stay coherent across turns. Source: An “agent” is a loop, not a function call. This page explains the loop Everruns runs and why it’s shaped the way it is. ## Reason, then act Each **turn** is one iteration of: 1. **Reason.** Send the full conversation history (system prompt + messages + previous tool results) to the LLM. The model either produces text or requests tool calls. 2. **Act.** If the model requested tool calls, execute them in parallel. The results become new messages in the conversation. 3. **Loop.** If there were tool calls, go back to step 1. If the model produced a final text response, the turn completes. A turn is capped at **10 iterations** by default. That cap exists for one reason: a misbehaving prompt can drive the model into an infinite tool-calling spiral, and an unbounded loop costs real money in tokens. The cap is configurable per session. ## Why parallel act? The model often emits several tool calls in one response, “read these three files” or “fetch these two URLs”. Executing them sequentially would serialize independent work for no reason. Everruns runs all tool calls from a single reason step concurrently and only re-enters reason once every call has completed (or failed). This means tool implementations cannot assume ordering between calls within a single act phase. If two tools must run in order, the model has to emit them across two turns. ## Execution phases When an assistant message includes tool calls, the model hasn’t given a final answer yet, it’s just narrating its plan. When the message has no tool calls, that’s the final answer. Everruns labels each assistant message with an **execution phase**: `Commentary` (intermediate, before/between tool calls) or `FinalAnswer` (completed response). Two consumers care: * **The model.** Some providers (OpenAI Responses API on the GPT-5.4 and GPT-5.5 families) accept and return phase annotations on replayed history. Without them, models can mistake earlier commentary for completed answers and stop early on long flows. * **The UI.** Phase tells the chat surface whether to keep the “thinking” indicator on or render the message as a final response. Phases are derived from message state (presence of tool calls) and stored on the message. Providers that don’t accept phases on the wire still get accurate internal tracking. ## Turn lifecycle Every turn emits these events in order: ```plaintext turn.started reason.started → reason.completed (one LLM call) act.started → act.completed (zero or more tool calls) ... (repeat reason/act until the model produces a final answer or max iterations) ... turn.completed | turn.failed | turn.cancelled ``` Streaming text and tool calls produce `output.message.delta` and `tool.started` / `tool.completed` events in between. The full event catalog is in the [Event Reference](https://docs.everruns.com/event-reference/). ## Why turns are durable, not in-memory A naïve implementation of the loop would hold all turn state in worker memory. Everruns doesn’t, because a worker crash mid-turn would lose work the user already paid for. Instead each step (reason, each tool call) is a separate durable task. The worker persists state after every step. If a worker crashes between steps, the control plane detects the missed heartbeat and re-queues the next task on a different worker. From the application’s perspective: a brief delay, then the stream continues. No retry button required. This trade-off, paying for a database write on every step, is what makes Everruns a *durable* agentic harness rather than a thin LLM wrapper. See [Durable execution](https://docs.everruns.com/explanation/durable-execution/). ## What happens when the loop “gets stuck” Three failure modes show up in practice: * **Runaway tool calls.** The model keeps calling tools without converging. Mitigated by the iteration cap; the turn fails with `turn.failed` once the cap is hit. * **A tool that hangs.** Tool calls have configurable timeouts. The act phase reports the failure as a tool result so the model can recover on the next reason step. * **The model rejects the prompt as too large.** Context [compaction](https://docs.everruns.com/advanced/compaction/) runs reactively and the request is retried. The conversation continues with older messages compressed. In all three cases the session stays usable. You don’t lose the conversation; you lose at most one turn. --- # Architecture > How Everruns is structured, control plane, workers, REST API, and the design choices behind the API-first, horizontally-scalable, headless approach. Source: Everruns is **API-first** and **headless**. The web UI is optional, the SDKs are optional, and the entire platform is reachable through a documented REST API. This page explains why. ## Two processes, one database A running Everruns deployment is two kinds of process plus PostgreSQL: * The **control plane** exposes the REST API, owns auth, serves SSE event streams, and persists all state in PostgreSQL. * **Workers** execute the agentic loop. They claim durable tasks, call LLMs, run tools, and report results back. They hold no long-lived state, restart any worker at any time and nothing breaks.  The control plane scales vertically and is fronted by a load balancer. Workers scale horizontally, add more for throughput, remove some to save cost. PostgreSQL is the only piece of stateful infrastructure. ## Why a separate worker tier? You could run the agentic loop in the API server process and avoid the extra hop. Everruns doesn’t, for three reasons: 1. **LLM calls are long.** A single turn easily spends 10–60 seconds in network I/O. Tying that to the request thread caps API throughput and makes deploys painful. 2. **Failure isolation.** A misbehaving tool that hangs the worker process should not take down the API. They’re separate concerns and they get separate processes. 3. **Durability.** Workers can crash and the task gets reclaimed by another worker. Tying execution to a request would lose the work the moment the connection drops. The control plane and workers talk over gRPC inside your VPC. End users only ever see the REST API. ## Why headless Most agent platforms ship a chat UI and treat the API as a side-channel. Everruns inverts this: the API is the product, and the UI is one client among many (the SDK, the CLI, customer-built apps, and the bundled management UI all consume the same endpoints). The consequences: * Every feature has to land in the API before it lands in the UI. There are no “UI-only” features. * You can ship Everruns as backend infrastructure for a product whose UI looks nothing like ours. * Multitenancy, auth, and quotas are enforced at the API layer, so you cannot accidentally bypass them by using a different client. The management UI exists for operators, configuring providers, browsing sessions, debugging events, not as the primary interaction surface. ## Why REST + SSE, not WebSockets Sessions need a bidirectional flow: the client posts messages, the server streams events. WebSockets would be a natural fit, but Everruns uses **REST for writes and SSE for reads**. * REST is cacheable, debuggable with `curl`, and survives every reverse proxy on the planet. * SSE is a one-way streaming protocol that works through HTTP/1.1, HTTP/2, and HTTP/3 with no special infrastructure. * The events you’d want to push to the server (cancel, new message) are infrequent enough that a `POST` is the right shape. Combined with `since_id` resumption and 5-minute connection cycling, SSE gives you reconnection-as-a-feature instead of reconnection-as-a-bug. ## Multitenancy and isolation Everruns is organization-scoped end-to-end: * All resources (agents, sessions, capabilities, providers) belong to exactly one organization. * API keys carry an org membership; the API enforces the boundary on every request. * Sessions are isolated at the database row level, there is no cross-session filesystem or key-value access. * Secrets are encrypted at rest with an organization-scoped key chain. ## Where the boundaries are Things the control plane owns: auth, durable task queue, event log, virtual filesystem, key-value storage, capability registry, MCP catalog. Things workers own: nothing persistent. They borrow the database, run their tasks, and report back. Things your application owns: the agent definitions, the prompt design, and the channel-specific glue (Slack apps, webhooks, schedules). The platform is intentionally neutral about *what* you build with it. --- # Core concepts > Understand the entity model behind Everruns, harnesses, agents, sessions, capabilities, and how they merge into a runtime. Source: Everruns has five entities that you’ll meet over and over: **harness**, **agent**, **session**, **capability**, and **event**. This page explains how they relate and why each exists separately. For field-by-field schemas, see the [API reference](https://docs.everruns.com/api/) and the [Concepts cheat-sheet](https://docs.everruns.com/getting-started/concepts/). ## Configuration vs. runtime The single most useful distinction in Everruns: | Layer | Entities | Lifetime | | ----------------- | --------------------------- | ------------------------------------------------ | | **Configuration** | Harness, Agent, Capability | Long-lived. You author these. | | **Runtime** | Session, Turn, RuntimeAgent | Created per conversation. The server owns these. | | **Data** | Event, Message | Append-only log produced during runtime. | Your application creates **configuration**, starts **runtime**, and consumes **data**. Reference pages that mix these layers, and there are some, read as if everything was one bag of “stuff to set”. The three roles never collapse into each other. ## Why three configuration layers (harness, agent, session)? You could imagine flattening everything into one “agent config” object. Everruns deliberately doesn’t. * A **harness** answers *“what environment am I running in?”*, model defaults, baseline tools, network access. The same harness is reused across many agents, and each agent holds a reference to the one it runs on. * An **agent** answers *“what role am I playing?”*, system prompt, domain capabilities, the agent’s voice. * A **session** answers *“what’s true for this one conversation?”*, extra tools the user just unlocked, an overridden model, a tighter network policy. When a session starts, all three layers merge into a single `RuntimeAgent` via an associative fold of overlays. Earlier layers form the base; later layers override or add. System prompts concatenate; network policies can only narrow (allow lists intersect, blocklists union); capabilities are deduplicated by ID. The motivation: operators control harnesses, app authors control agents, end users (or the runtime) control sessions. Each layer has a different blast radius, and the merge rules reflect that. None of the three is the agent loop. The loop, the thing that assembles context, calls the model, and dispatches tools, is the runtime, and it is not configured as an entity. This is worth stating because “agent harness” names that loop in most other projects, and Everruns uses the word that way itself when it calls the product a durable agentic harness engine. The `Harness` entity is a different thing with the same name: the configuration a session runs on top of. ## Why capabilities are first-class Tools could just be free-floating function declarations attached to an agent. Capabilities exist because three concerns travel together: 1. The **tool definition** (name, schema, handler). 2. The **system prompt addition** that teaches the model when to use it. 3. The **session state** the tool needs (mount points in the filesystem, secrets, dependencies on other capabilities). A capability bundles those three. Enabling `web_fetch` on an agent gives you not just the tool but also the prompt fragment explaining how to use it. Enabling `bashkit_shell` automatically pulls in `session_file_system` because of the declared dependency. Capability ordering is meaningful, earlier capabilities’ prompt fragments appear first in the merged system prompt. MCP servers and skills are also capabilities. They participate in the same merge, dependency resolution, and tool-name prefixing. This keeps the model surface uniform regardless of where a tool came from. ## Sessions are stateful; turns are bounded A session is a long-lived conversation: it owns an isolated virtual filesystem, a key/value store, and the full event log. Sessions don’t terminate, they go `idle` waiting for the next input. Inside a session, work happens in **turns**. A turn is one cycle of *reason* (call the LLM) and *act* (execute tools), repeated until the model produces a final answer. Turns are bounded, by default capped at 10 iterations, to make runaway loops impossible. The reason–act loop is described in more detail in [The agentic loop](https://docs.everruns.com/explanation/agentic-loop/). ## Messages are derived, events are stored There is no `messages` table in Everruns. The primary store is the **event log**: an immutable, append-only sequence of records per session. Messages are *reconstructed* from those events when needed. This sounds backwards until you remember what the application actually wants: * The UI wants a stream of *deltas* and *tool calls* and *state transitions*, not just finished messages. * Observability wants a trace of every LLM call and every tool invocation. * Replay and durability want a deterministic record of what happened. A message log can’t carry that. An event log can, and you can derive a message log from it cheaply. So events come first. See [Events as the primary store](https://docs.everruns.com/explanation/events/) for the consequences. ## Endpoints connect Agents to the outside world A bare Agent has no way to receive messages from external users. An **Endpoint** belongs to one Agent and exposes it through a transport such as Slack, AG-UI, A2A, FCP, or Public Chat. Each endpoint owns its inbound authentication, session-routing strategy, transport configuration, version policy, and publish state. One Agent can have several endpoints, and each endpoint can be published or revoked independently. Create and manage them from the Agent’s **Integrations** tab. For proactive scheduled work, use [Agent triggers](https://docs.everruns.com/features/agent-triggers/) instead of an endpoint. --- # Durable execution > Why Everruns persists every step of agent execution to PostgreSQL, what guarantees that provides, and the trade-offs of avoiding Temporal-style infrastructure. Source: The word “durable” in “durable agentic harness engine” means a specific thing: **every step of an agent’s execution survives a process restart**. This page explains why that matters, how it works, and what it costs. ## The problem An agent turn looks simple from the outside, send a message, get a streamed response. Internally it’s a chain of network calls that each take seconds: 1. Call the LLM provider. 2. Parse tool calls, dispatch them. 3. Wait for each tool to return. 4. Call the LLM again with the results. 5. …repeat… Anywhere in that chain, the worker process can crash, the container can be reaped, the network can blip. A naïve implementation loses all the work done so far and forces the user to retry. For a 30-second turn that already burned tokens, this is unacceptable. ## The mechanism Everruns runs every step as a **durable task**. Each task: * Has a typed input and output. * Persists its result to PostgreSQL before acknowledging completion. * Has retry and timeout policies. * Heartbeats while running, so the control plane can detect a crashed worker. A turn is a small state machine over those tasks. The state lives in `durable_workflow_events`, an append-only event log just for the workflow engine. To replay a workflow, you load its events and feed them back into the state machine, same input, same decisions, same output. When a worker crashes mid-turn, the control plane sees the missed heartbeats, marks the in-flight task as failed, and re-queues it. Another worker picks it up. The application sees a momentary stall in the SSE stream, then it continues. ## Why a custom engine The obvious alternative is Temporal (or Cadence, or Restate). Everruns deliberately built its own minimal durable engine, `everruns-durable`, instead. The reasoning: 1. **Single dependency.** PostgreSQL is the only stateful infrastructure. Operators don’t need to run a second cluster with its own ops story. 2. **Co-located with the rest of the platform.** Workflow events and session events live in the same database, in the same transaction when needed. There’s no eventual consistency between “what happened” and “what was reported.” 3. **Tight scope.** Everruns runs agentic workflows specifically, limited fan-out, short-to-medium duration, well-understood failure modes. We don’t need the full Temporal feature set, and the operational surface area of a tightly-scoped engine is much smaller. The trade-off: no multi-region replication beyond what PostgreSQL itself offers, no language-agnostic SDK (the engine is Rust-only inside the platform), no visual workflow designer. For agent execution these are not missed. ## Guarantees What durable execution gives you: * **No work lost on crash.** If a worker dies, another worker resumes from the last persisted step. Tokens already paid for are not paid for again. * **Exactly-once tool execution.** Tool calls are persisted by their result, not their attempt. A tool that completed but failed to ack will not be re-run. * **Deterministic replay.** Reloading a session reproduces the same message sequence, which makes traces and exports authoritative. What it doesn’t give you: * **Idempotence of side effects.** If your tool POSTs to an external API, the external API will see one call per *successful* execution but a retried-task scenario can still cause duplicates if a tool completes externally and crashes before persisting. Tools that have external side effects must include their own idempotency keys. * **Real-time latency guarantees.** Persisting every step adds tens of milliseconds per task. For agent workloads (already dominated by LLM latency) this is invisible; for hot-loop workloads it would be costly. ## When the database becomes the bottleneck Everruns is designed to run on a single PostgreSQL primary. Read replicas help for reporting; write-heavy session loads are handled by partitioning the durable workflow tables by workflow ID hash and by keeping event payloads compact. For deployments that outgrow a single primary, the migration path is to a sharded PostgreSQL setup keyed by organization, but in practice, LLM provider rate limits cap throughput long before the database does. ## Further reading * [The agentic loop](https://docs.everruns.com/explanation/agentic-loop/), what each step inside a turn looks like. * [Architecture](https://docs.everruns.com/explanation/architecture/), how the control plane and workers interact. --- # Events as the primary store > Why Everruns uses an append-only event log as the source of truth for sessions instead of a conventional message table. Source: Most chat systems store messages in a `messages` table and emit events as a side-channel. Everruns inverts this: the **event log is the source of truth**, and messages are derived from events. This page explains why. ## What an event is An **event** is an immutable record of something that happened in a session. Examples: * A user submitted a message. * The LLM produced a token of streaming output. * A tool was called. * A tool completed. * A turn started, completed, failed, or was cancelled. Every event has a session-local monotonic sequence number, a stable ID (UUID v7), a type in dot notation (`turn.completed`, `output.message.delta`), and a typed payload. Events are appended to the log and never modified or deleted. The full catalog is in the [Event Reference](https://docs.everruns.com/event-reference/). ## Why not a `messages` table? A conventional design has: * A `messages` table for the conversation. * A separate change-feed (Kafka, NATS, a `pg_notify` channel) for real-time updates. * A traces table for observability. This gives you three sources of truth that can disagree. When the SSE stream and the messages table disagree, which one is right? When a message is edited mid-stream, what does the change-feed say? When a tool call is observed but no resulting message is persisted, did it happen? Everruns avoids the question by making the event log authoritative for all three concerns: * **Conversation history** is reconstructed from events. A `Message` is a projection of a contiguous range of `output.*` and `input.*` events. * **The real-time stream** is just a tail of the same log. SSE clients receive events as they’re appended; resumption with `since_id` is a database read against the same table. * **Tracing and replay** consume the log directly. Every LLM call, tool execution, and state transition is there. One log, one ordering, one source of truth. ## Consequences This design has visible consequences in the API: * **You can’t UPDATE a message.** What you can do is append a new event that supersedes part of the projection. Messages can’t be edited because they don’t exist as rows. * **Ordering is by sequence number, not timestamp.** Two events with the same wall-clock time still have a strict order. Timestamps are informational. * **Resumption is cheap.** `since_id` is a primary-key scan. You can disconnect and reconnect to an SSE stream millions of events later and get exactly the missing tail. * **Audit is free.** Everything that happened in a session is in one table, in order, immutably. ## Compatibility guarantees Because events are the API contract, Everruns treats them like a public protocol. The compatibility rules are: | Change | Allowed | | ------------------------------------------ | ------------------------------------------- | | Add a new event type | yes | | Add an optional field to an existing event | yes | | Add a new enum value to an existing field | yes | | Remove a field | no, breaking change | | Change the type of a field | no, breaking change | | Reuse a sequence number | no, sequence numbers are atomic per session | Consumers must follow the dual: ignore unknown fields, ignore unknown event types, and treat optional fields as optional. The SDK clients do this automatically. ## What about volume? A long agent session can produce thousands of events, streaming deltas alone can be hundreds per turn. Two things keep this manageable: * **Delta events are batched** at \~100ms. The model produces tokens faster than that, but consumers don’t need every token as a separate event. * **Sessions are bounded by intent, not duration.** Even busy sessions rarely exceed five-figure event counts. PostgreSQL handles that easily. For very long-running sessions, [context compaction](https://docs.everruns.com/advanced/compaction/) reduces the *prompt* size but does not modify the event log. The full history remains available via [Infinity Context](https://docs.everruns.com/capabilities/infinity-context/) and the event API. ## Further reading * [Event Reference](https://docs.everruns.com/event-reference/), every event type and payload. * [The agentic loop](https://docs.everruns.com/explanation/agentic-loop/), what produces events during a turn. * [How-to: stream events](https://docs.everruns.com/how-to/stream-events/), practical patterns.