`(sigil publish core)`

(sigil publish core) - Functional Pipeline Core

The foundation of the publishing pipeline. Every stage is a function with signature (config state) -> state. The pipeline runner threads state through each stage via fold.

The pipeline supports two execution models:

Full build: build-graph then execute-graph
Incremental: build-graph then execute-changed with previous graph

Items can have producer: thunks that are called during execution and dependencies: lists for incremental rebuild tracking.

Exports

initial-stateprocedure

Create the initial pipeline state.

The state dict accumulates data as it flows through stages:

items: list of content item dicts
assets: list of asset dicts
navigation: navigation structure (list of groups)
ref-index: cross-reference lookup dict
rendered: list of paths written (for dev server)

build-graphprocedure

Build a publishing graph without executing producers.

Takes a config dict and a list of pipeline stage functions. Each stage has signature (config state) -> state. Returns the final state (graph) after all stages complete.

The graph can then be executed with execute-graph for a full build, or execute-changed for incremental rebuilds.

(define graph
  (build-graph
    #{base-url: "https://example.com"
      output-dir: "out"}
    (list
      (scan-files "posts/**/*.md")
      filter-drafts
      build-navigation
      attach-content-producers)))

(execute-graph graph)  ; Full build

execute-graphprocedure

Execute all producers in a graph.

Calls each item's producer: thunk (if present) and each asset's producer: thunk (if present). Returns the graph with rendered: updated to list written paths.

(execute-graph (build-graph config pipeline))

execute-changedprocedure

Execute only producers for items that changed.

Compares the new graph against the previous graph to find items with changed dependencies, then executes only those producers.

;; In watch loop:
(let ((new-graph (build-graph config pipeline)))
  (execute-changed new-graph prev-graph))

graph-find-changedprocedure

Find items that have changed between two graphs.

An item is considered changed if:

It's new (not in prev-graph)
Any of its dependencies have newer mtimes
Its source file has a newer mtime

Returns list of items from new-graph that need rebuilding.

add-itemsprocedure

Add items to state, appending to existing items.

add-assetsprocedure

Add assets to state, appending to existing assets.

set-ref-indexprocedure

Set ref-index in state.

set-path-indexprocedure

Set path-index in state.

add-renderedprocedure

Add rendered paths to state.

config-optionprocedure

Get an option from config, with optional default.

Looks in config's options: dict for the key.

(config-option config draft-mode: #f)