Deferred native components
Deferred components keep an optional native library out of the application's eager link. The artifact builder provides declarations, task object receipts, separate shared artifacts, narrow host exports and an embedded build catalog. Generated per-VM procedures dispatch through permanent VM-local slots. Explicit caller-root activation is implemented for dynamic x86_64 Linux with glibc. Portable producer/handoff integration and the FFI bridge remain separate work; ordinary build catalog v1 does not authorize runtime loading.
Packages using these declarations must set sigil: "0.24.10", or the later release that first ships them. The literal floor lets older tools refuse the manifest before evaluating unknown fields. These declarations require a release after 0.24.9.
Declare the provider and its consumer
The canonical ID belongs to the provider. Its suffix is the library name; the prefix is the provider's stable package identity. Applications select that ID without renaming it or supplying arbitrary C exports.
(library name: 'renderer
native-linkage: 'deferred
native-id: "example/renderer:renderer"
sigil-sources: '("src/example/renderer.sgl")
native-bindings:
(list (native-module name: '(example renderer)
bindings: '(("render" "example_render" 1))))
native-imports:
(list (native-interface name: '(example host)
abi: host-interface-fingerprint
symbols: '("example_host_render"))))
;; An eager host library exports this provider-owned interface contract.
(library name: 'host c-sources: '("src/host.c")
native-interfaces:
(list (native-interface name: '(example host)
abi: host-interface-fingerprint
symbols: '("example_host_render"))))
;; Inside the consuming package:
native-components:
(lambda (ctx)
(if (equal? (config-target (context-config ctx)) "x86_64-linux-gnu")
(list (native-component provider: "example/renderer:renderer"
bindings: (list (native-interface-binding
name: '(example host)
package: "example-host" library: 'host))))
'()))Interface fingerprints are SHA-256 strings. Runtime imports use the implicit (sigil runtime) interface, version 1; other interfaces need an explicit eager host binding with matching fingerprint and all required symbols. The build resolves these bindings before native tasks. Deferred libraries cannot provide native-init; their descriptor contains immutable binding data. Each binding triple names the Scheme procedure, C implementation and existing encoded native arity. Literal define-native forms must match the declaration. Computed, conditional and included declarations refuse. Each module has one owner; a component cannot replace (sigil core) or an eager provider's module.
Build only actual task outputs
Deferred tasks require explicit evaluated selection evidence; without it they remain unselected and perform no native work. Component tasks use library: on generate-native-component-descriptor, compile-c-sources and create-native-component. Generate a descriptor include with the completed dependency SDK's exact inputs, include it once in a provider translation unit, then compile all actual task sources:
(generate-native-component-descriptor
library: 'renderer
output: "build/guide/native/component-bindings.inc"
sdk: (lambda (ctx)
(apply native-component-sdk-digest
(native-component-sdk-inputs ctx))))The selected target/configuration's sigil-lib task finishes its archive, finalizes public headers and checks the public API inventory before any provider descriptor or C task. native-component-sdk-inputs returns that dependency's header root and archive. Descriptor claims and actual compiler header inputs must match it. The final runtime uses the same archive; provider actions never build or discover another SDK. Task-only deferred libraries and interface hosts also derive the SDK dependency from their sigil: floor. A selected executable requires a resolved sigil.lock before native work. Each selected provider must have one matching immutable Git commit or registry content pin, with the loaded package version and any explicit commit declaration matching the lock. Empty, unrelated, mutable and stale lock entries refuse, including when a test runtime gathers artifacts directly. Local and workspace providers without such a pin currently refuse too. Every dependency declaration of a selected provider must name its Git or registry source. Git URLs and full explicit commits must match the lock, and the provider's actual directory must be its locked installation. Registry sources use the full integrity-derived installation directory. Package/repository redirects, undeclared providers and the stale-dependency override refuse for selected components.
The ordinary dynamic x86_64 GNU API is recorded in packages/sigil-lib/src/public-sdk-symbols-v1.txt, including public sigil__* helpers and TLS data. Its checked header profile excludes the separately linked secure-only runtime. Header drift, a missing implementation or a failed address link refuses the SDK prerequisite. Runtime imports must belong to this profile; archive-private names are not public API.
ABI 1 providers must not keep Value or SigilVM* in static, global or TLS storage, including global containers. Intern symbols on every call. Persistent state belongs in VM-owned foreign objects or explicit arguments; ABI 1 has no provider-context storage, initialization, cleanup or unload hooks. Pass the component's destination through output: when creating the shared artifact. A task-generated extra source is included in the receipts even when it is absent from c-sources. An unrelated old object is never discovered by glob and added to the link.
Compilation records the compiler bytes, target, exact flags, complete compiler include closure and resulting object digest. Identified library tasks compile fresh; receipts are rechecked before linking. Library include directories and C flags join the task's explicit arguments. Deferred objects use PIC and hidden visibility; the shared artifact exports its data descriptor. closure: maps declared sonames to audited ELF files, including transitive dependencies; roots: supplies the staged RUNPATH view. Final closure files and rewritten artifact bytes are hashed before the catalog is generated.
An eager library exposing native-interfaces also needs actual task receipts. For a package with one host library, existing compile-c-sources tasks infer that owner, including task-added sources absent from c-sources. Multiple libraries require explicit library: ownership. create-static-library requires one valid receipt per archive member and writes an archive receipt; unattributed/prebuilt objects cannot silently satisfy the interface. The final link verifies required symbols in the selected library's actual objects, the archive digest and the copied link input, and rejects duplicate eager providers. Host package cache shortcuts are disabled until they retain complete proofs. Runtime-interface imports must belong to the checked public SDK API. A needed closure DSO cannot duplicate a public SDK name. Every shipped closure DSO, including unused plugins, must remain free of duplicate declared runtime/host imports and ABI guards. Direct objects and actually extracted archive members cannot define imported host/runtime implementations or public SDK globals, including hidden globals. Unrelated local helpers, unextracted members and unused ordinary shared inputs are allowed. Another component descriptor in a closure inventory still refuses. The actual linker trace and map establish extraction, including archives introduced by linker scripts; lazy observations alone are not extraction. The audit retains a complete pre-strip ELF before honoring strip options. The artifact record includes its digest and the exact strip tool path, bytes and options. Catalog generation validates that provenance against the audit's output digest, the retained pre-strip ELF and the recorded strip options/tool identity. A definition record without extraction evidence refuses; it cannot be treated as an unused archive member. Only DT_NEEDED-reachable closure DSOs can provide imports, and the component's defined dynamic exports must contain only its descriptor.
The build embeds the catalog in each consumer executable. It includes the source-lock digest, target, exact runtime archive/public-header SDK identity, provider contracts, final artifact digests and receipt/closure identities. The SDK receipt must match the actual final link configuration, and the source lock must remain unchanged between artifact gathering and catalog generation. sigil.lock remains the source dependency lock. A neighboring diagnostic catalog file is not runtime authority. Moving or rewriting a finalized DSO requires rebuilding its catalog. Component-containing package/bundle cache shortcuts are disabled until they can prove the complete reuse identity.
The parent link retains only declared host/runtime imports, including symbols referenced solely by a DSO and held in static archives. GNU ld uses explicit retention plus a version map. Zig's driver filters required linker flags, so the build captures its literal final LLD arguments and replays them through the pinned zig ld.lld with narrow exports. It rechecks explicit inputs and audits final .dynsym; broad exports refuse. No staged image is executed. LTO and whole-program flags currently refuse for selected components.
Target and runtime boundaries
Selected components require dynamic x86_64-linux-gnu and a linked native-entry, custom, development or test runtime. Static, musl, secure-entry, unsupported targets/toolchains and effective prebuilt-runtime reuse refuse. A manifest's default runtime: 'reuse does not determine the branch: native source uses build-native-entry; bytecode with native thunks needs a custom runtime. Graphs containing deferred providers refuse wasm and web bundles at preflight, including when the evaluated selection is empty.
An empty target selection skips deferred compilation before looking for its C sources or headers. Deferred C, archives, flags and dependency closures never join eager collectors, even in mixed packages. Bundled Scheme wrappers still receive unavailable thunks, including in static headless configurations. The embedder must run the generated native-init set for each VM. Thread worker VMs cannot attach roots or activate components, including on their own owner threads. The artifact tests create VMs directly through the embedding API.
Explicit caller-root activation
An embedder runs the generated native-init set for each VM, then calls sigil_vm_native_component_attach_root with a held, read-only directory FD. This attaches only a root: the process's hidden linked runtime catalog v2 supplies the SDK, ABI, complete file inventory, baseline library identities, environment profile and exact binding contracts. No neighboring catalog file, caller-provided table or shared-library symbol can select authority. Runtime catalog v2 and descriptor ABI 1 are separate version spaces.
Import (sigil native component) and explicitly call (native-component-load! "example/renderer:renderer"). The nonactivating native-component-loaded? query reports only the calling VM's state. Unknown canonical IDs raise named errors. Imports and ordinary unselected thunks do not load libraries. Each VM must activate independently, even when another VM already uses the same resident code.
Caller roots receive complete content verification before first activation, including mutations after attachment. Paths must remain within the held root without symlinks. Component dependencies resolve through verified relative $ORIGIN paths; missing members cannot fall back to the system. The caller-root lane admits literal absolute baseline search directories that cannot shadow a component soname. Runtime activation compares its controlled environment and never sets or repairs it.
The environment check reads two copies of the environment.
environ, at activation. Library code reads its variables (GIO_*,GST_*,WEBKIT_*,MESA_*,LIBGL_*and the rest of the controlled names) fromenvironwhen it runs. Every controlled name inenvironmust be a declared key, and every declared key must have exactly its declared value, or be absent when it is declared absent. Asetenvorunsetenvinside the process before activation counts here. For example, an application that sets its engine's library directory andGIO_USE_VFSat startup, before the engine's component activates, satisfies a profile that declares those values.- The startup block, for the loader variables. ld.so reads the
LD_*variables andGLIBC_TUNABLESonce, from the environment block the kernel passed at exec, and keeps what it read;setenvandunsetenvafterwards cannot change that. In the startup block, everyLD_*orGLIBC_TUNABLESentry must be a declared key with exactly its declared value, and every declared loader value must be present. An in-processsetenvof a loader variable never satisfies the profile, and anunsetenvnever hides one that was set at exec.
Activation reads the startup block from /proc/self/environ, through a descriptor it holds for the read. A non-dumpable process (one that called prctl(PR_SET_DUMPABLE, 0), or changed its uid with suid_dumpable=0) gets EACCES there. For such a process the check uses a copy of the loader entries that libsigil's constructor takes from environ. That copy shows the environment as it was when the constructor ran, so it includes a change that code in an earlier constructor had already made; that code runs in this process before Sigil does. The copy assumes libsigil is linked into the executable, as the native build links it, so that its constructor runs after the shared libraries' constructors and before main. If /proc/self/environ cannot be read and there is no copy, activation refuses with "component startup environment unreadable (non-dumpable process?)". It refuses as well when /proc/self/environ can be opened but not read completely.
Code inside the process that overwrites the startup strings in memory is outside this check; such code can already call dlopen directly. The common benign case is setproctitle, which overwrites the argument and environment strings to change the name ps shows. After it, /proc/self/environ no longer shows the exec-time loader variables, and the startup check silently compares whatever the strings now hold. Activate components before calling setproctitle, or keep the loader variables out of the overwritten range.
A kernel-secure process (one with AT_SECURE set, such as a setuid or file-capability executable) refuses to attach roots or activate components: ld.so ignores the LD_* variables there, so the catalog's environment profile does not describe what it did.
The creating thread owns all component operations. Wrong-owner C calls return a status and a bounded caller-owned diagnostic without writing the foreign VM's error state. This does not make other VM APIs thread safe. Fork children refuse component operations until exec, including with a replacement VM. In a fork child, only VMs owned by the forking thread can be used or destroyed: the child has no other threads, so another thread's VM has no owner there. Constructor recursion poisons the entire activation. Post-load descriptor or mapping failures quarantine the handle and publish no VM implementations.
From Sigil code, every refusal raises: native-component-load!, native-component-loaded? and the component's own procedures raise in a fork child and on constructor reentry, so native-component-loaded? in a fork child never reads as loaded. Running the generated native-init set raises in the same cases: in a fork child, and on a VM whose activation a constructor reentered (SIGIL_COMPONENT_REENTRANT). A wrong-owner call is the one exception: it returns the status without writing the foreign VM's error state. The owner check runs before the reentry check, so a constructor's call on a VM that another thread owns is a wrong-thread call and does not poison the activation.
While a VM activates a provider, sigil_vm_destroy on that VM refuses and leaves the VM intact. A provider constructor that destroys the VM activating it poisons the activation: the load returns SIGIL_COMPONENT_REENTRANT and the provider stays quarantined until process exit. The embedder destroys the VM after the load returns.
The loader lock is held for the whole activation, including the provider's constructors. A constructor that starts a thread which activates a component, in any VM, and then waits for that thread deadlocks: the thread waits for the loader lock, which the constructor's activation holds. A constructor that does not wait for the thread only delays that thread's activation until its own activation finishes.
A provider must not cache a Value in static or thread-local storage. A Value such as an interned symbol belongs to the VM that made it; another VM, including one on the same thread, has its own. Keep per-VM state in foreign objects owned by that VM. The qualification's static and tls fixtures show both caches returning another VM's symbol, and the sanitizer variant reports the stale value after its VM is destroyed.
Threat model: the path between verification and dlopen
Activation verifies the provider and its dependencies through descriptors opened under the held root directory, then calls dlopen on /proc/self/fd/<root>/<path>. The kernel resolves that path again under the same held directory, so in the window between verification and dlopen a writer could replace the file at that path. dlopen takes a path, not a descriptor, so the loader cannot close this window on glibc.
Only the effective user or root can use the window. Attachment requires the root directory and every ancestor to be owned by the effective user or root, with no group or other write permission unless the directory is sticky. Verification opens every file and every directory below the root without following symlinks and requires each to be owned the same way and not group- or other-writable. Either party can already change the process, for example through ptrace or by editing the files before attachment. After dlopen the loader checks every mapped object's device and inode against the verified identities and quarantines the provider on a mismatch, but a replaced file's constructors have already run by then. Roots that other users can write are refused at attachment, not defended at load.
The ELF reader reads each file's dynamic table, string table and descriptor through its program headers, the way ld.so finds them in memory. ld.so maps the PT_LOAD segments in order, and a later one that covers an earlier one's pages replaces those bytes, so a file could show the reader one dynamic table and ld.so another. The reader therefore refuses a file whose PT_LOAD segments are not strictly ascending by address or share a page.
A VM runs only on the thread that created it, and its suspended native fibers stay on that thread. An evaluation or procedure call (sigil_eval_*, sigil_c_eval_*, sigil_apply*) from another thread stops the process, with a message naming the call, the calling thread and the VM's owner thread. The call never writes the VM: an error recorded from the foreign thread would fail an evaluation the owner was running at the time. A collection started on another thread, for example by a C API allocation, also stops the process. So do bytecode execution started any other way (loading a module or a bytecode file, a script, the REPL), a generated module's _init and sigil_vm_destroy. The owner is identified by a token the thread gets when it first creates a VM, never by its pthread_t, which pthreads hands to a later thread once the owner exits.
Destroy a VM on its owner thread before that thread exits. A thread that exits with VMs it has not destroyed frees their suspended native fibers and marks them orphaned: no thread can run an orphaned VM again, and any thread may then destroy it, which frees its heap. sigil_vm_destroy also frees the VM's suspended native fibers on the owner thread, so a worker thread that runs one VM per job does not run out of fiber slots.
A continuation whose native frames wait in a suspended fiber resumes only in the VM that captured it, and not while another VM's native code is running on the thread; either call fails with a runtime error and leaves the continuation as it was.
Separate VMs on separate threads run natively compiled code at the same time. On one thread, a VM's native code cannot start while another VM's native code is running there, for example from that VM's host callback; the call fails with a runtime error.
<sigil/native-component.h> defines the implemented borrowed-library query, guarded by SIGIL_NATIVE_COMPONENT_HAS_LIBRARY_QUERY. It returns only a stored, verified handle for an already mapped declared library of an activated provider. It does not search or open an unmapped library, and clears output on failure. The loader retains successful and quarantined code until process exit; callers must never transfer these handles to FFI close or call dlclose. VM destruction revokes component access before finalizers run, with the code still resident.
Portable component artifacts use format 1 with the verified bundled glibc loader and unavailable build-v1 thunks. Format-3 component packaging, activation/cache leases, helper wrapper execution, mapped closure checks, concurrent loads and window latency are required qualifications for portable runtime loading and consumer integration. WPE, FFI and UI adoption require that loading support.