vigiles API reference
    Preparing search index...

    Interface MeasureSpec

    A task run N times, scored against a Trace check vocabulary.

    interface MeasureSpec {
        allowedTools?: readonly string[];
        checks: readonly Check<RunContext>[];
        effort?: string | number;
        fixture?: Record<string, string>;
        interceptTools?: readonly ToolIntercept[];
        model?: string;
        plugin?: string;
        pluginDir?: string;
        settings?: unknown;
        skillsDir?: string;
        spacingSec?: number;
        stubSkillBodies?: boolean;
        task: string;
        timeoutMs?: number;
        trials?: number;
    }
    Index

    Properties

    allowedTools? checks effort? fixture? interceptTools? model? plugin? pluginDir? settings? skillsDir? spacingSec? stubSkillBodies? task timeoutMs? trials?

    Properties

    allowedTools?: readonly string[]

    Tools pre-approved for the run (--allowedTools); [] approves none.

    checks: readonly Check<RunContext>[]

    The checks to score across the trials. Any check over the run is accepted: Trace checks (tool/skill/output/mcp/judged) and resource checks (cost/latency/tokens, which read the eval-only usage) — all fit Check<RunContext>.

    effort?: string | number

    Reasoning budget for the run (claude --effort, e.g. "low" or an integer). Lives here beside model because it is part of the MEASUREMENT — it moves the output distribution, not the sample size — so it is hashed into the lock and the cache, and never read from an env var. Omit for the harness default.

    fixture?: Record<string, string>

    Base fixture files written for every run (path → contents).

    interceptTools?: readonly ToolIntercept[]

    Tools to intercept (the tool-call spy) — see EvalArm.interceptTools.

    model?: string

    Model alias. Default "sonnet" — measure on the model your users run.

    plugin?: string

    A real plugin/repo to load (materialized) — see EvalArm.plugin.

    pluginDir?: string

    A complete plugin dir to install natively (--plugin-dir) so skills activate — see EvalArm.pluginDir. Provide this OR skillsDir, not both.

    settings?: unknown

    .claude/settings.json (hooks/permissions) for the run.

    skillsDir?: string

    A loose <skillsDir>/<name>/SKILL.md directory (e.g. a repo's .claude/skills) to install, auto-packaged into a throwaway plugin — see EvalArm.skillsDir. Installs under vigiles-loose-skills, so skill("vigiles-loose-skills:<name>").

    spacingSec?: number

    Seconds between runs.

    stubSkillBodies?: boolean

    Stub each skill BODY (frontmatter/trigger surface kept) before the run — for checks about whether a skill FIRES (skill()), not what it produces. A selected skill stops at selection instead of running its (often expensive) procedure, so a description/firing run costs a fraction of the tokens. Do NOT combine with judged/quality checks: the body is gone, so there's nothing to grade. Requires pluginDir or skillsDir. See stubSkillBody.

    task: string

    The task prompt given to the agent.

    timeoutMs?: number

    Per-run timeout ms.

    trials?: number

    Trials. Default 5.

    Settings

    Member Visibility

    On This Page

    Properties