Update an observer.

PATCH

/v1/observers/{observer_id}

const url = 'https://app.everruns.com/api/v1/observers/example';
const options = {
  method: 'PATCH',
  headers: {'Content-Type': 'application/json'},
  body: '{"description":"Score replies from the support agent","match":{"agent_ids":["example"],"harness_ids":["example"],"session_tags":["example"]},"name":"Support quality","sampling_rate":0.5,"scorers":[{"model_id":"example","pass_threshold":1,"rubric":"example","method":"llm_judge","key":"example","scope":"turn"}],"status":"active"}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}

curl --request PATCH \
  --url https://app.everruns.com/api/v1/observers/example \
  --header 'Content-Type: application/json' \
  --data '{ "description": "Score replies from the support agent", "match": { "agent_ids": [ "example" ], "harness_ids": [ "example" ], "session_tags": [ "example" ] }, "name": "Support quality", "sampling_rate": 0.5, "scorers": [ { "model_id": "example", "pass_threshold": 1, "rubric": "example", "method": "llm_judge", "key": "example", "scope": "turn" } ], "status": "active" }'

Parameters

Path Parameters

observer_id

required

string

Prefixed public identifier

Request Body^required

application/json

Request to update an observer. Omitted fields are unchanged.

object

description

Human-readable description. Safe to render in user-facing messages.

string | null

Example

Score replies from the support agent

match

One of:

object
null

Predicates selecting which production sessions an observer scores. All present predicates must match (AND); within a list, any entry matches (OR). An empty match block matches all org traffic. Sessions tagged eval are always excluded so synthetic eval-run sessions are never scored.

object

agent_ids

Match sessions running any of these agents.

Array<string> | null

harness_ids

Match sessions on any of these harnesses.

Array<string> | null

session_tags

Match sessions carrying any of these tags.

Array<string> | null

name

Human-readable name. Safe to render in user-facing messages.

string | null

Example

Support quality

sampling_rate

Fraction of matching turns to score (0.0–1.0).

number | null format: double

Example

0.5

scorers

Array | null

One scorer inside an observer. key names the score series in listings and future dashboards; scope selects the trace slice; method is how it grades.

object

One of:

object
object

Deterministic rule. file_contains is rejected for observers (session filesystems are not part of the observable trace contract).

object

method

required

string

Allowed values: rule

rule

required

One of:

Final assistant message contains substring.

object

text

required

string

type

required

string

Allowed values: contains

weight

number format: double

Agent called named tool at least min times.

object

min

integer format: int32

tool

required

string

type

required

string

Allowed values: tool_called

weight

number format: double

Citation faithfulness: the answer’s citations must cover the claim and be verified as supported. Scored from the TextAnnotations on the final message — pair with the citation_verification capability so verdicts are present.

object

min_citations

Minimum number of citations the answer must carry.

integer format: int32

Example

pass_threshold

Minimum fraction of citations verified entailed to pass.

number format: double

Example

0.8

type

required

string

Allowed values: citation_faithful

weight

Relative weight of this scorer in the case’s weighted average.

number format: double

Example

Citation faithfulness judged by an LLM: each cited claim/source pair is graded by a model, so the eval works even without the citation_verification capability.

object

model_id

One of:

string
null

Judge model; the org’s default is used when absent.

string

/^model_[0-9a-f]{32}$/

Example

model_01933b5a000070008000000000000001

pass_threshold

Minimum judged score [0,1] to pass.

number format: double

Example

0.8

rubric

Rubric override; a citation-faithfulness rubric is used when absent.

string | null

Example

Score the fraction of cited claims supported by their source.

type

required

string

Allowed values: citation_judged

weight

Relative weight of this scorer in the case’s weighted average.

number format: double

Example

LLM-as-judge.

object

model_id

Org model to judge with. When None, the org’s default model is used. Judge calls go through the org’s own providers and are billed to it.

string | null

pass_threshold

Score value at/above which pass is true.

number format: double

rubric

required

Grading rubric shown to the judge model. Should describe what a high vs. low score means.

string

method

required

string

Allowed values: llm_judge

key

required

Stable name within the observer (score series name).

string

scope

Trace slice this scorer grades.

string

Allowed values: turn

status

One of:

string
null

New lifecycle status (active, paused, or archived).

string

Allowed values: active paused archived deleted

Responses

200

Success

application/json

An observer: online scoring config over production sessions.

object

archived_at

string | null format: date-time

created_at

required

string format: date-time

description

Optional description.

string | null

required

External identifier (observer_<32-hex>). Shown as “id” in API.

string

match

Which sessions to score.

object

agent_ids

Match sessions running any of these agents.

Array<string> | null

harness_ids

Match sessions on any of these harnesses.

Array<string> | null

session_tags

Match sessions carrying any of these tags.

Array<string> | null

name

required

Display name.

string

sampling_rate

required

Fraction of matching turns to score (0.0–1.0), applied after match.

number format: double

scorers

required

Scoring rules.

Array

One scorer inside an observer. key names the score series in listings and future dashboards; scope selects the trace slice; method is how it grades.

object

One of:

object
object

Deterministic rule. file_contains is rejected for observers (session filesystems are not part of the observable trace contract).

object

method

required

string

Allowed values: rule

rule

required

One of:

Final assistant message contains substring.

object

text

required

string

type

required

string

Allowed values: contains

weight

number format: double

Agent called named tool at least min times.

object

min

integer format: int32

tool

required

string

type

required

string

Allowed values: tool_called

weight

number format: double

object

min_citations

Minimum number of citations the answer must carry.

integer format: int32

pass_threshold

Minimum fraction of citations verified entailed to pass.

number format: double

type

required

string

Allowed values: citation_faithful

weight

Relative weight of this scorer in the case’s weighted average.

number format: double

Citation faithfulness judged by an LLM: each cited claim/source pair is graded by a model, so the eval works even without the citation_verification capability.

object

model_id

One of:

string
null

Judge model; the org’s default is used when absent.

string

/^model_[0-9a-f]{32}$/

pass_threshold

Minimum judged score [0,1] to pass.

number format: double

rubric

Rubric override; a citation-faithfulness rubric is used when absent.

string | null

type

required

string

Allowed values: citation_judged

weight

Relative weight of this scorer in the case’s weighted average.

number format: double

LLM-as-judge.

object

model_id

Org model to judge with. When None, the org’s default model is used. Judge calls go through the org’s own providers and are billed to it.

string | null

pass_threshold

Score value at/above which pass is true.

number format: double

rubric

required

Grading rubric shown to the judge model. Should describe what a high vs. low score means.

string

method

required

string

Allowed values: llm_judge

key

required

Stable name within the observer (score series name).

string

scope

Trace slice this scorer grades.

string

Allowed values: turn

status

required

Lifecycle status.

string

Allowed values: active paused archived deleted

updated_at

required

string format: date-time

Example

{
  "description": "Grades support replies for grounded answers.",
  "id": "observer_01933b5a000070008000000000000001",
  "name": "Support answer quality",
  "sampling_rate": 0.1,
  "scorers": [
    {
      "method": "llm_judge",
      "scope": "turn"
    }
  ],
  "status": "active"
}

404

Observer not found

Update an observer.

Parameters

Path Parameters

Request Bodyrequired

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Responses

200

Example

404

Request Body^required