Record n checks against this script's run.
The tiers call it themselves, so an ordinary harness never needs to. Call it
directly when you assert some OTHER way — node:assert, vitest's expect, a
hand-rolled comparison — and want those visible to vigiles test instead of
leaving it to conclude the file did nothing.
vigiles(the package root) — the free testing surface: everything you can run without a model call, and therefore without a bill.This is the front door. There is no
vigiles/testsubpath: the bare package name IS the testing surface, because a second name for the same door is the duplication this reorganisation removed (the old.+./specpair pointed at one module under two names). A subpath is warranted when something genuinely DIFFERENT lives behind it — another runtime, another driver, another protocol. Here the one real difference is whether a call can spend money, so there are exactly two doors: this one, andvigiles/eval.What replaced what
The old split was by TEST TIER —
vigiles/unit,vigiles/integration,vigiles/e2e,vigiles/testing. Tiers are a property of test-file layout and runner config, not of a module graph: this repo already expresses them as vitest projects (test:unit,test:integration,test:e2e), so the exports map was a second, competing implementation of the same idea.src/e2e.tshad degenerated toexport * from "./integration.js"— a barrel adding zero symbols, which its own docstring called "the definition of a non-tier".Two honest costs of the new split
1. The type coupling crosses the boundary and cannot be removed. About two dozen helpers here —
assertImproves,assertNoRegression,assertReliable,assertSignificant,assertTriggerRate,reliable,improvement,significantlyBeats,compareArms,diffReports,diffToJUnit,formatBaselineDiff,parseBaselineFile,readBaseline,toBaselineFile,writeBaseline,cost,latency,tokens,inputTokens,outputTokens,cacheTokens, plusformatEvalReport/formatCheckReport/formatTriggerRateReport/checkReportToJUnit/assertRates— operate on eval RESULTS while spending nothing themselves. They belong on the free path. Their TYPES (EvalReport,CheckReport,TriggerRateReport,Metrics, …) are defined on the paid side. So those types are re-exported from BOTH barrels: a user of the free surface must never importvigiles/evalfor a type alone. The duplication is deliberate and is the price of splitting on cost.2. Free is not the same as fast. The old "real CLI, no API key" tier (
vigiles/integration) collapses into this barrel, so the import path no longer warns you thatrunHarnessTestspawns a realclaudebinary under bubblewrap and can take ~40 seconds. Nothing here bills you; plenty here is slow. Duration now lives where duration always lived in practice — in the runner config (--project integration), not in the import.What was lost, stated plainly: the symmetry with the CLI verbs is now one-sided.
vigiles/evalrhymes withvigiles eval, but the free half answers to the plain package name rather than tovigiles test. That is accepted, not overlooked.@playwright/test— an end-to-end tool by definition — ships no subpath naming a test type at all, and keeps the whole fast/slow/expensive distinction in its config rather than in its imports.