Testing Plugins

A test file is executed by the same isolated worker that runs a production handler, with the same calling convention: (arvexa, context) in, a plugin-test-report-1.0 result out. One consequence trips up every author once: module loading is denied, so a test file cannot import `src/`. There is no require, so there is no way to reach the handler under test.

What a test file can do — and what actually decides whether a package publishes — is prove that your assumptions about the platform hold: that the harness context is shaped as expected, that every capability your manifest declares a scope for is genuinely authorized, and that nothing depends on the wall clock.

Return shape:

• contractVersion — plugin-test-report-1.0
• cases — each with name, entrypoint, status (PASSED, FAILED, or SKIPPED), and message

Discovery is narrow and mirrored exactly by the CLI: only tests/*.js. A test written in src/, or as .mjs, .cjs, or .ts, is never executed — there is no transpilation step. Non-.js files under tests/ are fine as data fixtures; they simply do not run.

Harness rules that decide an attempt:

• Every key of runtime.entrypoints needs at least one executed, non-skipped case. A SKIPPED case deliberately does not satisfy coverage, or MISSING_TEST would be trivially avoidable with a stub.
• Capability calls are authorized through the real broker and then answered with deterministic mocks. A denied call fails the whole attempt, even when the case asserting the denial would itself have passed.
• context.nowIso is a fixed instant (1970-01-01T00:00:00.000Z), not the wall clock.
• Each passing file is re-run twice and the verdicts compared. A value derived from Date.now(), from randomness, or interpolated into a case *name* fails the attempt as NONDETERMINISTIC. This is the most common way an otherwise-correct plugin fails its first publish attempt.

Attempt limits: 25 test files, 2,000 ms CPU per file, 120,000 ms wall clock for the whole attempt, 200 cases per file, and 50 persisted failures (the report records when it truncated). The unit of execution is the file — each runs in its own worker with its own budget — so splitting slow or noisy checks into their own file keeps them from eating another file's budget.

Failure codes: TEST_CASE_FAILED, MISSING_TEST, NO_TEST_FILES, TEST_TIMEOUT, RESOURCE_ABUSE, UNAUTHORIZED_API_CALL, NONDETERMINISTIC, TEST_PROTOCOL_INVALID, TEST_FILE_ERROR.

Locally, run the preflight. arvexa plugin test <dir> --preflight-only never executes plugin code. It answers offline, in milliseconds, the questions that dominate first attempts: will the harness find any test file at all, is every declared entrypoint covered, and does every entrypoint's source exist in the package. A clean preflight is necessary and never sufficient — only the host executes your tests, authorizes capability calls, and replays for determinism.

Then request the real attempt. arvexa plugin test <dir> --package-id <id> asks the host to run the *uploaded* bytes; the harness cannot run a local directory. An attempt is immutable and identified by an idempotency key, so re-asking after a lost response is safe. A FAILED verdict exits 11, distinct from a request error, because nothing went wrong with the request — the verdict is the answer.

The CLI warns when the tested content hash is not the hash your directory currently packs to. That is legal (you may be testing an older upload) but silently trusting a verdict for different bytes is how stale evidence gets believed.