Script startup and output profile

The implicit direct-file entrypoint now skips CLI initialization and command discovery. println and eprintln send a complete line to the port once; POSIX stdout uses line buffering for terminals, files, and pipes. Bulk-output programs can explicitly opt into block buffering with SIGIL_STDOUT_BUFFERING=block; Windows terminals remain unbuffered. The following measurements distinguish those structural changes from timing claims that still depend on workstation contention.

Visibility review measurements, 2026-11-02

Completed lines must reach files and pipes while a program is still running, including before termination. The restored POSIX default is line buffering; SIGIL_STDOUT_BUFFERING=block is an explicit throughput opt-in. Stderr writes flush preceding stdout in every mode. Reads from stdin flush partial prompts when either stdin or stdout is a terminal. Hosts that never call the stdio initializer retain newline flushing, including the combined println string. Windows uses unbuffered stdout by default and always on terminals.

The fresh native executable and released 0.24.1 were measured serially, after all local builds and tests finished, at nice 10 on CPUs 10-11. Each series has three alternating rounds of 20,000 lines with real file, pipe, and PTY sinks; the driver validates every line. The profiling JSON now records the buffering environment alongside executable hashes, raw samples, and host metadata.

Median body time, 20,000 linesFile, msPipe, msPTY, ms
Released 0.24.1 println152.311117.602214.813
Current println, default line mode120.998107.198206.344
Current println, explicit block mode141.221120.117232.006
Bash printf, baseline series133.35397.685147.339
Bash printf, default series117.26396.127175.517
Bash printf, block series147.560146.866360.870

Default PTY println takes 10.317 microseconds per line against the same series' Bash control at 8.776. Block mode reduces syscalls, but these samples do not show a uniform timing gain. Its later series also has much slower Bash controls and higher host load. No local build or test overlapped measurement; other workstation activity remained outside the experiment's control.

The trace counts only stdout writes from the actual Sigil child, excluding PTY forwarding:

20,000 println linesFile writesPipe writesPTY writes
Released 0.24.140,00040,00040,000
Current default20,00020,00020,000
Explicit block mode117117117

Both current modes use the same native executable, reporting eaa1f71b2-dirty, SHA-256 36856bfc25040cb697f00f214848d477b764caf43239aa23cd0850c9fd8e63fa. The baseline reports 9e8f48cd, SHA-256 6a7239fa301eb157a267667445482efc1c34af28fb924d6ecd15a66e2a37ff27.

The live pipe/file tests fail on the earlier implementation that used block buffering by default, and pass on fresh development and native builds, including a completed file line retained after SIGTERM. Nine C cases exercise visibility before exit, a fully buffered embedding host without initialization, explicit block mode, merged stderr, an uncaught exception without an error message, and terminal stdin with piped stdout. The 53 focused cases pass on both fresh CLI builds. Real WASM console output and all 18 browser-console assertions pass; the latter's timing check was retried without concurrent builds after its initial contended failure. These runtime tests ran on Linux; the Windows CRT branch was not executed.

Earlier measurements, 2026-11-02, before the visibility review

This series measured the earlier file/pipe block-buffering default. That default was rejected because completed lines stayed invisible in long-running programs and could be lost on termination. Its file/pipe write counts below describe the historical implementation, not the current default.

All runs use nice 10 and CPUs 10-11. Startup uses measure-script-startup.sgl: 21 alternating rounds, one retained warmup, 20 measured launches per case, midpoint p50, and nearest-rank p95. Each process runs in a neutral directory, with library/redirect environment overrides removed and exact stdout/stderr checks. Raw results retain every sample, executable hashes, versions, affinity, and load. No build or test process overlapped these final measurements.

Startup comparisonBefore p50/p95, msAfter p50/p95, msSame-run Node p50/p95, ms
Native executables1,234.100 / 1,680.820278.140 / 686.483145.962 / 194.840
Optimized VM development launchers704.742 / 840.705131.000 / 263.94897.522 / 169.931
Earlier, quieter VM comparison337.908 / 390.82861.441 / 81.43639.501 / 48.707

The native baseline is released 0.24.1 (9e8f48cd); the after executable is the working branch's release-configured native build, still reporting 0.24.2. The VM baseline uses the earlier CLI and launcher before direct-file dispatch; the after executable is the rebuilt development image. Both VM launchers use an optimized runtime. Native and VM rows are separate backend comparisons.

The final shared-host run does not establish the quiet-machine 88-ms goal or the production p95 goal of twice Node. Even Node's median rose above 44 ms. The earlier quieter VM run met both quantiles' twice-Node target, but it is not a quiet-machine qualification of the final native artifact. A regression test independently verifies that a hello script loads neither the full CLI, project loader, nor optional REPL command module.

Printing uses three alternating rounds of 20,000 lines, with body time measured inside each child. The driver checks every expected line and retains version, hash, affinity, niceness, and load metadata. Bash's loop is a comparison specimen; orchestration remains Sigil.

20,000 lines, median body timeFile, msPipe, msPTY, ms
Before println "hello"119.091111.790218.280
Earlier after println "hello"92.653141.473198.394
Earlier after one formatted display including newline7.2747.38371.287
Same-run Bash printf, before series79.20168.531156.662
Same-run Bash printf, after series74.87973.808159.379

The final PTY println result is 9.920 microseconds per line against Bash's 7.969, slower than Bash. File time improved; pipe time did not. These short, contended samples do not establish a uniform throughput gain. The large remaining difference from direct formatted display needs separate profiling. The earlier 5-to-6-microsecond Bash observation is not substituted for this run's actual control.

System-call counts give a less timing-sensitive result. The trace selects the actual Sigil child, excluding PTY forwarding writes:

20,000 println linesFile stdout writesPipe stdout writesPTY stdout writes
Before40,00040,00040,000
Earlier after, block-buffered files/pipes11711720,000

That earlier artifact's block buffer produces 117 writes for 120,000 payload bytes. A custom-port regression separately proves exactly one complete port write per println or eprintln, including Unicode. Exit/error, inherited child, explicit flush, terminal input/prompt, and terminal stdout/stderr-order regressions pass on both development and native executables. Normal exit and uncaught language errors flush; fatal OS termination cannot guarantee cleanup.

Initial investigation

On a quieter shared workstation, a hello script took about 244 ms on released Sigil 0.24.1. The 0.24.2 development build showed that most time goes into the full CLI and its command discovery, before the script runs. The same development runtime's existing bootstrap path took 18 ms with core and 41 ms with the shell prelude. That is direct evidence for a smaller scripting path below 100 ms.

This report measures startup and printing separately. All orchestration is Sigil. Measurements ran on Linux 6.17.12 at nice 10 on CPUs 10 and 11. The host was shared; warm filesystem measurements are not physical cold-start results.

Complete launches

The driver is profile-script-startup.sgl. It alternates case order across nine rounds, starts a new process per sample, and checks output and exit status. All cases include the driver's capture path.

CaseMedian, msRange, ms
Released 0.24.1 hello244.326219.243–753.570
Released empty script235.227219.195–1,917.650
Released --version236.642217.532–734.235
0.24.2 development hello265.640244.267–334.041
Development empty script267.956249.927–334.447
Development --version263.384244.289–310.441
Development bootstrap, core only18.33215.717–25.229
Development bootstrap, shell prelude41.32435.294–63.506
External true control4.1643.222–6.034
Bash empty file6.1034.437–8.176
Node empty file28.46726.859–35.600
Python empty file17.28616.654–23.254

The release is 9e8f48cd; the development source starts at 391234d347dd3c287aa1deec7cc9c88856236eaf. Development and native-release backends are different. Their absolute timings cannot establish a release-to- release speedup. Earlier hello observations above 540 ms, and later quiet observations near 240 ms, show why contention and ranges must be retained.

Timestamped phases on local source

A disposable checkout of that exact base was built with timestamp-only changes to the VM, CLI, and generated development launcher. It uses monotonic clocks. The preparation, launcher link, and collection drivers retain the method. Instrumentation did not change the reference checkout.

PhaseMedian, msRange, ms
C entry to VM call0.0190.014–0.023
VM creation2.2632.075–2.439
Load paths, native setup, core9.4088.859–11.158
Mandatory CLI module closure173.405159.590–179.929
Call CLI main0.1290.115–0.202
Capture compiler-file identities3.6013.320–4.505
Wrappers, redirects, project/dependencies0.0760.063–0.106
Command tree and optional modules59.96256.093–76.095
Script selection and context0.1060.091–0.111
Generated guard and import setup3.2042.839–3.511
Re-import loaded shell prelude0.0440.039–0.053
Read, compile, execute hello0.1690.147–0.199
Return0.0070.006–0.008
VM teardown21.04219.830–24.859

The C-entry-through-destruction median was 277.746 ms (254.915–296.367). Whole-process medians were 283.269 ms with timestamp output disabled and 285.821 ms enabled, in alternating order. Medians of phases are not additive. The timestamps add branches and generated guard forms, so the script guard's small cost is an instrumented observation rather than an exact unmodified cost. The project/dependency phase was measured from a neutral directory; a real project can add manifest and dependency work.

make-sigil-cmd calls discover-optional-commands before dispatching the invocation. That eagerly loads REPL, MCP, LSP, store, registry, and other available commands, including for a direct script or --version. The script prelude is already loaded before script selection. Its re-import takes only 0.044 ms here; its initial cost is included in the earlier module closure.

The released binary's load trace finds bundled .sgb modules. It is already using precompiled library bytecode. Binding lookup profiling reports 148,922 lookups, 326,061 iterations, and 6.4 ms inside lookup on one released run. Neither script compilation nor binding lookup explains most startup time.

What the measurements support

For a target below 100 ms, dispatch an ordinary direct-file invocation through a smaller runtime path before importing the full CLI or discovering optional commands. Preserve the existing invocation, argument handling, project checks, and error/exit contract. The measured 41-ms shell-prelude bootstrap path is a feasibility probe, not a proposed user-facing bootstrap command.

Moving optional-command discovery behind dispatch can recover about 60 ms in this development measurement. It cannot alone reach 100 ms while mandatory CLI initialization still costs about 173 ms. Lazy imports must also reduce that initial closure.

For 20–40 ms, the next experiment should reduce eager prelude initialization on the small path. Compare lazy initialization of unused facilities with selectively compiled core/prelude initialization. The core-only 18-ms probe is the measured comparison point; retaining the complete prelude costs about 41 ms. Reaching 20–40 ms still requires a prototype and compatibility tests.

Caching hello's compiled script can save at most its measured sub-millisecond read/compile/execute phase. It is useful for larger scripts, but is not the first fix for this delay. Merely supplying precompiled stdlib bytecode repeats what already exists; module initialization and the eager closure must change.

Printing and system calls

The printing driver measures loop body time inside the child, so startup is excluded from the body column. It runs three alternating rounds with 20,000 iterations and actual file, captured-pipe, and PTY sinks. PTY measurements use script -qec; a direct PTY is used separately for first-byte latency.

BodyFile, msPipe, msPTY, ms
Loop only2.7042.6682.667
Format only5.5835.6576.095
println "hello"106.58689.474142.724
println "~a" "hello"105.00889.097135.606
One display "hello\n"24.01217.75339.738
One formatted display including newline27.64421.67451.178

These are comparisons of complete paths, not an exclusive CPU attribution to individual C functions. Formatting alone is small relative to the split display/newline path. The syscall driver identifies the actual Sigil child PID, keeping PTY forwarding writes separate:

20,000 complete linesFile writesPipe writesPTY writes
println40,00040,00040,000
Single display including newline20,00020,00020,000

The native entry generated by cps-emit-native.c calls setvbuf(stdout, NULL, _IONBF, 0). println expands to format, display, then newline; native newline writes the newline and calls fflush. Consequently the released CLI's stdout is unbuffered for files and pipes too. The observed behavior is not TTY line buffering with ordinary block buffering elsewhere.

The PTY wrapper adds forwarding copies and writes of its own. Shell captures also add a pipe-drain/capture path, and default ANSI stripping adds text work. These printing pipe measurements use ansi: #t; they do not benchmark default ANSI stripping. A child-side spawn file sink writes directly to the file, while synchronous $ captures output before displaying or writing it.

A promising printing change is one formatted write for a complete line, with an explicit interactive flush policy. It must retain string/custom-port behavior, Unicode, output errors, and low-latency protocol output. The unbuffered native entry exists for MCP/pipe response latency, so removing it globally without those checks would change behavior beyond terminal printing.

Time to first byte

profile-first-byte.sgl starts a real child, checks its marker, drains/reaps it, and records first observation at a 0.1-ms polling interval. Seven quiet launches per sink gave:

SinkMedian first byte, ms
File246.201
Raw pipe245.807
Direct PTY236.310

The direct PTY measurement avoids script wrapper teardown. Its small timing advantage is not evidence that terminals accelerate startup; load varies across samples. The initial pause is dominated by startup, not by the first line's terminal write.

Reproduce

Use a disposable checkout at the recorded base, a built development reference, and the same Zig compiler used for that reference's native archives. The link driver targets build/dev/native/dev-entrypoint.c; editing the standalone runtime launcher does not instrument the generated development CLI.

sigil scripts/profile-script-startup.sgl RELEASED-SIGIL DEV-SIGIL OUTPUT
sigil scripts/prepare-startup-profile.sgl DISPOSABLE-CHECKOUT BUILT-REFERENCE

Build sigil-cli --config dev inside the disposable checkout's Guix environment, with SIGIL_JOBS=2 and the required toolchain available. Then use the private generated-launcher relink and collector:

sigil scripts/link-startup-profile.sgl DISPOSABLE-CHECKOUT MATCHING-ZIG
sigil scripts/collect-startup-phases.sgl PROFILE-SIGIL HELLO-SCRIPT OUTPUT
sigil scripts/profile-script-output.sgl RELEASED-SIGIL PRINT-OUTPUT
sigil scripts/profile-output-syscalls.sgl RELEASED-SIGIL PRINT-OUTPUT STRACE
sigil scripts/profile-first-byte.sgl RELEASED-SIGIL OUTPUT

The hello source must print STARTUP-HELLO followed by a newline. The syscall driver consumes the printing driver's specimens, so use the same print output directory. Do not overlap compilation with timing collection. Keep raw samples, backend identities, affinity, and host-load observations with each run.