Runtime test evidence (experimental, opt-in)

The first sink is POSIX-process scoped; Windows/browser builds stay disabled and refuse explicit evidence opt-in with exit 74.

The runtime layer records consumed input buffers and module initialization attempts. Static test-runner completion accounting and the external observer are available through test execution evidence, which also describes the optional evidence reference added to --report.

Set SIGIL_TEST_EVIDENCE to a new absolute JSONL file path and SIGIL_TEST_RUN_ID to the external observer's nonempty run token before starting Sigil. The first VM initializes the sink before native registration and Scheme preload. The file is created exclusively, with mode 0600 and close-on-exec. An existing/conflicting file, invalid configuration, allocation failure in the writer, or write/flush error emits an evidence diagnostic and exits 74. Individual flushed records survive ordinary later failures; flushing does not promise crash durability. A missing/truncated journal or absent runner terminal is incomplete evidence.

The process owns one journal and one monotonically increasing sequence. The first VM consumes and clears both environment variables; subsequent VMs in that process share the writer with distinct monotonic VM IDs. Fork descendants are disabled by an owner-PID check before any inherited mutex is touched. Exec descendants inherit neither the configuration nor the descriptor. Such children are unqualified; an observer must never infer that an unobserved child passed. A separately launched qualified process needs its own journal and explicit observer accounting.

Every line has schema: 1, run_id, pid, vm_id, seq, and event. VM, input, module, and application IDs are monotonic within their documented scopes, not GC addresses. Input/module/application IDs are scoped to the VM. Zero module ID means the payload's module target is unknown. Zero input ID means no acquired runtime input context is available. Module IDs identify canonical module names in one VM, including replacement objects with that name; they do not identify old closures.

Source-file applications are observed at the execution seam for each compiled top-level expression; a direct define-library supplies its declared target. Unnamed/macro-generated targets remain unknown. Bytecode application observes the actual target chosen by the deserializer, including existing native modules. Compilation can register declarations before execution; explicit compile-only stamps are emitted where the compiler marks a phantom module. Other registrations have the conservative registration-or-reuse phase. A record's contextual input is not complete origin proof for arbitrary native or compile-time activity.

Generations describe initialization payload attempts, not the identity of all current bindings. Mutable bindings, old retained closures, arbitrary parsed eval, macro-created targets, mixed native/Scheme bodies, and AOT execution remain outside that claim. Nested input contexts are restored on return, including error/abort returns. A process exit or kill inside an initializer leaves the application/input without a terminal; an external observer must classify that as incomplete.

Enabled filesystem source and bytecode loading acquires a complete private buffer and checks seek, length/overflow, allocation, short-read, growth and read/close errors. Bytecode's disabled FILE-reader path and source's disabled acquisition path are preserved. Memory bytecode callers must keep the supplied buffer immutable through the synchronous load, as required for deserialization. Path labels are for explanation; hashes identify buffers. Valid UTF-8 labels are preserved; invalid individual bytes are escaped and labels are not a lossless filesystem-path API.

The internal runner bridge is intentionally finite:

• (%test-evidence-info) returns #f when disabled, otherwise #(1 run-id journal-path vm-id).
• (%test-evidence-event! event payload-json) accepts only run-plan, file-start, file-end, and run-terminal. Both arguments are strings with no embedded NUL. The payload is escaped into payload_json, which the observer parses separately; callers cannot inject raw journal records.

See test execution evidence for the static in-process --sgl runner and bounded external observer. Native/custom test harnesses and watch iterations remain unqualified.

Run the standalone runtime controls with:

guix shell -m manifest.scm -- make test-runtime-evidence

That target links its fixture against the newly built runtime, retains raw JSONL, stdout/stderr and source/SGB bytes under a disposable directory, and uses Python's independent SHA-256/JSON implementation as the oracle. On Linux it also uses a separate test-only preload library to force a short read. It does not use an installed CLI or imply full release-suite acceptance.

Release population and child libraries

make test-release-suite requires an already-built optimized CLI containing runtime/test evidence support. It builds the required C test executables and their bootstrap libraries, then records two separate populations:

• Scheme files run through test --sgl --config release. Before launching the CLI, the gate inventories tracked test/, tests/, and packages/*/test/ files using the runner's documented naming and .testignore policy. The T1 observer requires the selected set and consumed file hashes to match that independent inventory. Missing tracked files fail setup; unexpected selected files fail qualification. Empty files remain explicit file outcomes.
• Native executables come from Make's declared TEST_BINS, checked against tracked packages/sigil-lib/test/native/test-*.c sources. Missing or nonexecutable required programs fail setup. Incidental build/*/bin/test-* files cannot change either population.

The native ledger records launch/running/terminal state, PID, actual exit status, executable hashes before and after execution, and separate stdout/stderr logs. A nonzero status fails even after passing assertion lines. A claimed SUMMARY: must agree with observed PASS:/FAIL: records. Programs using only C assertions and process exit retain a process-success contract: one completed process does not become an invented assertion count. Native output and time are bounded; overflow, timeout, and interruption remain incomplete. The supervisor drains both streams and kills the child process group on termination. SIGTERM/SIGINT produce incomplete records and cleanup. SIGKILL can prevent cleanup and terminal publication; the retained launching/running state must not be accepted as PASS. The Scheme observer defaults to a 600-second timeout. For slower or contended build hosts, use make test-release-suite RELEASE_TEST_SCHEME_TIMEOUT=1800 to set a longer finite limit. A timed-out run remains incomplete and must be retained separately from any retry. Its output files do not have the native byte cap. If the supervisor is interrupted before the Scheme observer publishes a terminal record, the outer ledger retains an incomplete pending or launching/running Scheme state.

The external observer validates the journal through EOF, one record at a time, and hashes files in chunks. It retains input and runner records needed for reconciliation; repeated module lookup records do not accumulate in memory. Memory still depends on individual record sizes and the retained input/test population. Malformed records after the runner terminal still reject the run.

Each invocation uses a new evidence directory and prints its qualification.json path. Set RELEASE_TEST_EVIDENCE_DIR to choose a new destination. The gate refuses reuse. Fixture registry/key defaults remain caller-overridable. A valid run requires every required native process and every selected Scheme file to finish; there is no historical-failure allowance.

The ordinary CLI test handler now passes its startup filesystem library roots to child processes in runtime priority order. It captures those roots before adding project or auto-compile directories, anchors relative paths to the original working directory, and preserves intentional -L and SIGIL_LIB roots. A bundled child uses its own archive; the handler does not insert workspace boot/dev libraries or forward executable-local @bundle/ labels. CLI development entrypoints read SIGIL_LIB before -L and native module initialization, matching the optimized CLI's ordering. App and custom test harness entrypoints retain their existing behavior. Paths containing : cannot be represented by the existing environment transport and are rejected explicitly when test execution would pass them to children.

This is startup-context propagation, not automatic propagation of later add-library-path! calls, module reloads, or mutable bindings. The release ledger does not claim consumed-byte evidence for every arbitrary subprocess or native body. The reference parent/child controls use separate T1 observers and require the actual acquired CLI and fixture-module bytes in each process. Run them and the native supervisor/population controls with make test-release-door; this target is included in make test. For an optimized tool, invoke test/integration/test-cli-child-libraries.py with --binary, the corresponding producer --library directory, and --bundled. Keep those producer artifacts and the executable together when recording qualification.