Internals and literate programming setup

Table of Contents

This file documents the internals of daggerlib: the literate programming setup, tangling pipeline, and editor configuration. See readme.org for an introduction.

1. Known issues with the current engine version

The module pins engineVersion v0.20.3 in dagger.json.

The engine's telemetry proxy returns HTTP 500 for /v1/metrics protobuf payloads sent by module runtimes. Traces and logs work fine through the same proxy. The bug is invisible in fast calls (the Python SDK's default 60 s export interval never fires), but surfaces in DinD test runs where cold module loading takes long enough for the periodic metric export to trigger. The error is cosmetic — dagger functionality is not affected.

The root cause is a race or initialization bug in the telemetry PubSub during module loading, fixed on main as a side effect of commit db59252ad ("workspace: plumbing & compat", 2026-03-31). Upgrading past v0.20.3 will resolve it. A reproducer lives in ~/test/otel-repro/.

2. Module entry point

The Lib class and its __init__.py are defined in src/main.org, following the same literate style as the per-feature org files in src/.

3. Module context

Each function uses with_directory(".", source, include[…])= to select only the files it needs (see dind.org). The .daggerignore only excludes .git/ — which is large and never needed inside the engine. Everything else is handled by per-function includes.

.tangle-deps/ (230 MB, fetched by the tangling pipeline) is mounted as a cache volume inside the container (see dind.org) and must be excluded here — otherwise it would change the module source hash on every tangle and bust all caches. 

.git
.tangle-deps
__pycache__
.pytest_cache
.ruff_cache

4. Editor configuration

The .dir-locals.el ensures that Emacs preserves Python indentation inside org code blocks. Without this, C-c ' (org-edit-special) would re-indent Python code relative to the org heading level. 

((org-mode
  . ((org-babel-default-header-args:python
      . ((:preserve-indentation . t)))
     (org-id-link-to-org-use-id . nil)
     (eval . (load (expand-file-name "reproducibility-helpers.el"
                     (locate-dominating-file default-directory ".dir-locals.el"))
                   t)))))

5. Development

The org files are the source of truth. The Python modules, test scripts, and tangle.sh itself are all generated. To regenerate everything from scratch: 

rm -rf src/lib/alpine.py src/lib/debian.py src/lib/dind.py \
       src/lib/distroless.py src/lib/flask_venv.py src/lib/pip_tools.py \
       src/lib/user.py src/lib/main.py src/lib/__init__.py \
       tests/*.sh tests/test_use_cases.py \
       tangle.sh tangle.el test-host.sh run.sh
./bootstrap.sh
./tangle.sh

6. Source directory convention

Some functions need files from the module source tree (shell scripts, config files). There are two patterns depending on context.

6.1. dag.address(".") for Lib functions (tested via the harness)

DefaultPath(".") is resolved by the Dagger CLI, but the Python test harness calls functions directly — DefaultPath never fires. For functions that must work in both contexts, use an explicit src: dagger.Directory | None = None parameter with a fallback to dag.address(".").directory(): 

def my_function(
    self,
    src: dagger.Directory | None = None,
) -> dagger.Container:
    if src is None:
        src = dag.address(".").directory()
    return (
        dag.container().from_("...")
        .with_file("/tmp/script.sh", src.file("src/lib/script.sh"))
        .with_exec(["sh", "/tmp/script.sh"])
    )

This works everywhere:

  • dagger call (CLI resolves "." to the module context)
  • Python test harness (pytest resolves "." to the working directory)
  • Direct SDK usage (caller can pass any directory)

The shell scripts themselves live in tangled .sh files under src/lib/, keeping shell code in shell and Python code in Python.

7. Reproducibility helpers

Helpers loaded both by tangle.el (batch) and .dir-locals.el (interactive), so tangling works the same way everywhere.

Those MUST be kept as minimal as possible, to avoid drifting too far from vanilla.

The ruff post-tangle hook ensures that Python files are formatted identically whether tangled from the command line or interactively.

Each dagger call bash block carries a #+NAME: and tangles its command to tests/commands/<name> via :tangle (daggerlib-auto-test). The function reads the block's name from org-element-at-point so the name appears only once in the org source.

Expected output files (tests/expected/<name>) are generated automatically after tangling: daggerlib--write-expected-files scans the org buffer for #+RESULTS: lines matching each tangled command and writes them out. This eliminates the separate #+BEGIN_SRC org expected blocks that previously required repeating the name.

The test runner iterates over command files, runs each one, and compares stdout to the expected output. Example tests (examples/*/tests/) run with cwd set to the example directory so dagger call loads the example module. 

  ;;; reproducibility-helpers.el --- Tangle helpers for test commands

  (defun daggerlib--test-base-dir ()
    "Return the base directory for test files.
  For example org files (under examples/) use the example directory.
  For library org files (under src/) use the project root."
    (let* ((root (locate-dominating-file default-directory "dagger.json"))
           (rel (file-relative-name default-directory root)))
      (if (string-prefix-p "examples/" rel)
          default-directory
        root)))

  (defun daggerlib-auto-test ()
    "Derive tangle path for test command from the block's #+NAME."
    (let ((name (org-element-property :name (org-element-at-point))))
      (unless name (error "daggerlib-auto-test: no #+NAME on this block"))
      (expand-file-name (concat "tests/commands/" name)
                        (daggerlib--test-base-dir))))

  (defun daggerlib--extract-result (name)
    "Extract cached #+RESULTS lines for NAME in current buffer."
    (save-excursion
      (goto-char (point-min))
      (when (re-search-forward
             (format "^[ \t]*#\\+RESULTS\\(?:\\[.*\\]\\)?:[ \t]+%s"
                     (regexp-quote name))
             nil t)
        (forward-line 1)
        (let ((lines nil))
          (while (looking-at "^[ \t]*: \\(.*\\)")
            (push (match-string-no-properties 1) lines)
            (forward-line 1))
          (when lines
            (mapconcat #'identity (nreverse lines) "\n"))))))

  (defun daggerlib--write-expected-files (tangled-files)
    "For each tangled command file, write expected output from #+RESULTS.
Skip the write when content is already up-to-date — rewriting unchanged files
would bump mtimes and invalidate Dagger's Directory.diff downstream."
    (let ((root (locate-dominating-file default-directory "dagger.json")))
      (dolist (f tangled-files)
        (when (string-match "/tests/commands/\\([^/]+\\)$" f)
          (let* ((name (match-string 1 f))
                 (result (daggerlib--extract-result name)))
            (when result
              (let* ((cmd-rel (file-relative-name f root))
                     (exp-rel (replace-regexp-in-string
                               "/commands/" "/expected/" cmd-rel))
                     (expected-file (expand-file-name exp-rel root))
                     (new-content (concat result "\n"))
                     (old-content (when (file-exists-p expected-file)
                                    (with-temp-buffer
                                      (insert-file-contents-literally
                                       expected-file)
                                      (buffer-string)))))
                (unless (equal new-content old-content)
                  (make-directory (file-name-directory expected-file) t)
                  (with-temp-file expected-file
                    (insert new-content))
                  (message "auto-expected: %s" expected-file)))))))))

  ;;; reproducibility-helpers.el ends here

8. Org-mode pin

The scripts bootstrap.sh and run.sh clone a pinned org-mode commit. The pin lives in .org-pin (a plain file, not tangled) so it cannot diverge. The bash scripts inline the hash at tangle time via noweb. 

1025e3b49a98f175b124dbccd774918360fe7e11

The ensure-org block checks that the cloned org-mode matches the pin and re-clones if needed. The shell scripts reference it via noweb. 

# Ensure pinned org-mode is cloned and up-to-date
ORG_PIN="1025e3b49a98f175b124dbccd774918360fe7e11"
ORG_DIR="$SCRIPT_DIR/.tangle-deps/org"
if [ -d "$ORG_DIR" ] && [ "$(git -C "$ORG_DIR" rev-parse HEAD 2>/dev/null)" != "$ORG_PIN" ]; then
    echo "Org-mode pin changed, re-cloning..."
    rm -rf "$ORG_DIR"
fi
if [ ! -d "$ORG_DIR" ]; then
    echo "Cloning pinned org-mode ($ORG_PIN)..."
    mkdir -p "$SCRIPT_DIR/.tangle-deps"
    git clone --quiet https://git.savannah.gnu.org/git/emacs/org-mode.git "$ORG_DIR"
    git -C "$ORG_DIR" checkout --quiet "$ORG_PIN"
    # Generate org-loaddefs.el and org-version.el (needed for org to load properly)
    emacs --batch --no-init-file \
        --eval "(progn
                  (push \"$ORG_DIR/lisp\" load-path)
                  (require 'autoload)
                  (setq generated-autoload-file \"$ORG_DIR/lisp/org-loaddefs.el\")
                  (update-directory-autoloads \"$ORG_DIR/lisp\"))" 2>/dev/null
    ORG_GIT_VERSION=$(git -C "$ORG_DIR" describe --tags --match "release_*" 2>/dev/null || echo "N/A")
    ORG_RELEASE=$(echo "$ORG_GIT_VERSION" | sed 's/^release_//;s/-.*//')
    (cd "$ORG_DIR/lisp" && emacs --batch --no-init-file \
        --eval "(progn
                  (push \"$ORG_DIR/lisp\" load-path)
                  (load \"$ORG_DIR/mk/org-fixup.el\")
                  (org-make-org-version \"$ORG_RELEASE\"
                                        \"$ORG_GIT_VERSION\"))") 2>/dev/null || true
fi

9. Tangle configuration

When tangle.sh runs Emacs in batch mode, it loads tangle.el to configure org-babel before tangling. This ensures that tangled files get link comments pointing back to their source block (:comments yes) and blank lines between blocks (:padline yes). It also loads a pinned org-mode version from .tangle-deps/ for reproducibility and registers the bash and python babel languages.

Batch Emacs cannot interact with the user. If org-babel encounters an unknown language with :comments yes, it tries to ask for the comment syntax and silently fails. We guard against this by making any interactive prompt a hard error in batch mode, so the problem surfaces immediately instead of producing a silent partial tangle.

Since batch Emacs has no user init, packages like nix-mode are not available. We map the nix language to conf-mode so that org-babel recognises #+begin_src nix blocks for tangling without requiring an extra package.

After tangling, an advice on org-babel-tangle calls daggerlib--write-expected-files to auto-generate tests/expected/<name> from #+RESULTS: lines for every block that tangled to tests/commands/. 

;;; tangle.el --- Self-contained org-babel tangling for dagger lib -*- lexical-binding: t; -*-

;; Don't prompt for code block evaluation
(setq org-confirm-babel-evaluate nil)

;; In batch mode, any interactive prompt means something is misconfigured
;; (e.g. unknown language with :comments yes).  Fail loudly instead of
;; hanging or silently skipping blocks.
(when noninteractive
  (dolist (fn '(read-string read-from-minibuffer completing-read))
    (advice-add fn :before
                (lambda (&rest _)
                  (error "Tangle aborted: batch Emacs cannot interact (check language modes in tangle.el)")))))

;; Load pinned org-mode from .tangle-deps BEFORE anything else loads the
;; built-in org.  This must happen before (require 'ob-shell) since that
;; transitively loads org.
(let ((org-lisp-dir
       (expand-file-name ".tangle-deps/org/lisp"
                         (file-name-directory (or load-file-name buffer-file-name)))))
  (when (file-directory-p org-lisp-dir)
    (push org-lisp-dir load-path)
    (let ((contrib (expand-file-name "../contrib/lisp" org-lisp-dir)))
      (when (file-directory-p contrib)
        (push contrib load-path)))
    ;; Force load of pinned org (unload built-in if already loaded)
    (require 'org)))

;; Load babel languages needed for tangling
(require 'ob-shell)
(require 'ob-python)

;; Map nix to conf-mode so #+begin_src nix blocks tangle without nix-mode
(add-to-list 'org-src-lang-modes '("nix" . conf))

;; Add link comments and blank lines between blocks in tangled output
(setq org-babel-default-header-args
      (cons '(:comments . "yes")
            (cons '(:padline . "yes")
                  (assq-delete-all :comments
                    (assq-delete-all :padline
                      org-babel-default-header-args)))))

;; Preserve indentation in Python blocks so that org-level indentation
;; does not interfere with Python's significant whitespace.
(add-to-list 'org-babel-default-header-args:python
             '(:preserve-indentation . t))

;; Load reproducibility helpers (shared with interactive Emacs
;; via .dir-locals.el).
(load (expand-file-name "reproducibility-helpers.el"
         (file-name-directory (or load-file-name buffer-file-name)))
      t)

;; After tangling, auto-generate tests/expected/ files from #+RESULTS
;; for every block that tangled to tests/commands/.
(advice-add 'org-babel-tangle :filter-return
            (lambda (files)
              (daggerlib--write-expected-files files)
              files))

;; Strip trailing whitespace in the per-block tangle buffer.  Doing this
;; inside the body-hook (instead of `sed -i' post-emacs) keeps the tangle
;; buffer byte-identical to the on-disk file, so org-babel-tangle's
;; compare-buffer-substrings (ob-tangle.el:320) skips write-region — no
;; mtime change, Dagger's Directory.diff stays empty, downstream caches
;; (notably ./test.sh) survive across runs.
(add-hook 'org-babel-tangle-body-hook #'delete-trailing-whitespace)

;;; tangle.el ends here

10. Tangle script (host)

The host tangle script tangles all org files using host emacs. It reuses the same tangle_file function and ensure-org block as the bootstrap script, but tangles everything rather than just the bootstrap subset. The daily-use tangle.sh wrapper runs this inside a container for reproducibility.

No post-processing: org-babel-tangle writes tangle targets as-is and already skips write-region when the content is unchanged (ob-tangle.el:315-327). Trailing whitespace is stripped per block via org-babel-tangle-body-hook in tangle.el, not via sed -i — that way the tangle buffer stays byte-identical to the on-disk file and emacs' idempotence check keeps the mtime frozen, which in turn keeps Dagger's Directory.diff empty and preserves downstream caches across runs. 

# Tangle all org files.
# Usage:
#   ./tangle.sh                          # tangle all org files
#   ./tangle.sh src/foo.org              # tangle a specific file
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"

# Ensure pinned org-mode is cloned and up-to-date
ORG_PIN="1025e3b49a98f175b124dbccd774918360fe7e11"
ORG_DIR="$SCRIPT_DIR/.tangle-deps/org"
if [ -d "$ORG_DIR" ] && [ "$(git -C "$ORG_DIR" rev-parse HEAD 2>/dev/null)" != "$ORG_PIN" ]; then
    echo "Org-mode pin changed, re-cloning..."
    rm -rf "$ORG_DIR"
fi
if [ ! -d "$ORG_DIR" ]; then
    echo "Cloning pinned org-mode ($ORG_PIN)..."
    mkdir -p "$SCRIPT_DIR/.tangle-deps"
    git clone --quiet https://git.savannah.gnu.org/git/emacs/org-mode.git "$ORG_DIR"
    git -C "$ORG_DIR" checkout --quiet "$ORG_PIN"
    # Generate org-loaddefs.el and org-version.el (needed for org to load properly)
    emacs --batch --no-init-file \
        --eval "(progn
                  (push \"$ORG_DIR/lisp\" load-path)
                  (require 'autoload)
                  (setq generated-autoload-file \"$ORG_DIR/lisp/org-loaddefs.el\")
                  (update-directory-autoloads \"$ORG_DIR/lisp\"))" 2>/dev/null
    ORG_GIT_VERSION=$(git -C "$ORG_DIR" describe --tags --match "release_*" 2>/dev/null || echo "N/A")
    ORG_RELEASE=$(echo "$ORG_GIT_VERSION" | sed 's/^release_//;s/-.*//')
    (cd "$ORG_DIR/lisp" && emacs --batch --no-init-file \
        --eval "(progn
                  (push \"$ORG_DIR/lisp\" load-path)
                  (load \"$ORG_DIR/mk/org-fixup.el\")
                  (org-make-org-version \"$ORG_RELEASE\"
                                        \"$ORG_GIT_VERSION\"))") 2>/dev/null || true
fi

tangle_file() {
    local orgfile="$1"
    echo "Tangling $orgfile..."
    local raw_output rc=0
    raw_output=$(emacs --batch --no-init-file \
        -l "$SCRIPT_DIR/tangle.el" \
        --eval "(progn
                  (require 'org)
                  (find-file \"$orgfile\")
                  (let ((files (org-babel-tangle)))
                    (dolist (f files) (princ (format \"%s\n\" f))))
                  (kill-buffer))" 2>&1) || rc=$?
    if [ "$rc" -ne 0 ]; then
        echo "ERROR: emacs tangle failed (exit $rc):" >&2
        echo "$raw_output" >&2
        return "$rc"
    fi
}

if [ $# -eq 0 ]; then
    for f in "$SCRIPT_DIR"/readme.org "$SCRIPT_DIR"/TECHNICAL.org \
             "$SCRIPT_DIR"/src/*.org "$SCRIPT_DIR"/tests/*.org \
             "$SCRIPT_DIR"/examples/*/readme.org; do
        [ -f "$f" ] && tangle_file "$f"
    done
else
    for f in "$@"; do
        tangle_file "$(realpath "$f")"
    done
fi

11. Run configuration

Like tangle.el configures tangling, run.el configures batch execution of org-babel blocks. It provides dagger-run-file to execute bash blocks (dagger call commands) and save the results. dagger-run-ignore-cache overrides :cache to "no" globally so that all blocks re-execute regardless of whether their content changed — useful when an external dependency changed but the block source didn't. The dagger wrapper in tests/ handles suppressing TUI noise (see testing.org). 

    ;;; run.el --- Batch-execute org-babel bash blocks -*- lexical-binding: t; -*-

    (defun dagger-run-file (orgfile &optional no-cache)
      "Execute bash blocks in ORGFILE.
When NO-CACHE is non-nil, ignore cached results."
      (find-file orgfile)
      (org-babel-map-src-blocks nil
        (when (and (string= lang "bash")
                   (not (assq :init (nth 2 (org-babel-get-src-block-info t)))))
          (message "%s Executing %s %s..." (format-time-string "%H:%M:%S") lang
                   (or (org-element-property :name (org-element-at-point)) "(unnamed)"))
          (if no-cache
              (org-babel-execute-src-block nil nil '((:cache . "no")))
            (org-babel-execute-src-block))))
      (save-buffer)
      (kill-buffer))

    (defvar dagger--block-error nil)

    (defun dagger--capture-error (exit-code stderr)
      "Advice: capture block failure details (ignore stderr-only warnings)."
      (when (and (numberp exit-code) (> exit-code 0))
        (setq dagger--block-error
              (format "exit %s:\n%s" exit-code (or stderr "")))))

    (defun dagger--execute-or-die (&optional info params)
      "Execute src block; signal error if it exits non-zero."
      (setq dagger--block-error nil)
      (advice-add 'org-babel-eval-error-notify :before #'dagger--capture-error)
      (unwind-protect
          (if (and info params)
              (org-babel-execute-src-block nil info params)
            (org-babel-execute-src-block))
        (advice-remove 'org-babel-eval-error-notify #'dagger--capture-error))
      (when dagger--block-error
        (error "Block failed (%s)" dagger--block-error)))

    (defun dagger-init-file (orgfile &optional no-cache)
      "Execute init bash blocks (those with :init yes) in ORGFILE.
When NO-CACHE is non-nil, ignore cached results.
Stops at first failure."
      (find-file orgfile)
      (org-babel-map-src-blocks nil
        (let* ((info (org-babel-get-src-block-info t))
               (params (nth 2 info)))
          (when (and (string= lang "bash")
                     (assq :init params))
            (message "%s Init %s..." (format-time-string "%H:%M:%S")
                     (or (org-element-property :name (org-element-at-point)) "(unnamed)"))
            (if no-cache
                (dagger--execute-or-die info '((:cache . "no")))
              (dagger--execute-or-die)))))
      (save-buffer)
      (kill-buffer))

    ;;; run.el ends here

12. Run script (host)

Executes org-babel blocks using host emacs in batch mode and saves the results. Populates the #+RESULTS: blocks that tangling writes to tests/expected/ via noweb references. The daily-use run.sh wrapper runs this inside a container for reproducibility. 

# Execute org-babel blocks and save results.
# Usage:
#   ./run.sh                              # run all src and doc org files
#   ./run.sh --no-cache                   # ignore cached results
#   ./run.sh src/foo.org                  # run a specific file
set -eu

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"

NO_CACHE=
if [ "${1:-}" = "--no-cache" ]; then
    NO_CACHE=1
    shift
fi

# Ensure pinned org-mode is cloned and up-to-date
ORG_PIN="1025e3b49a98f175b124dbccd774918360fe7e11"
ORG_DIR="$SCRIPT_DIR/.tangle-deps/org"
if [ -d "$ORG_DIR" ] && [ "$(git -C "$ORG_DIR" rev-parse HEAD 2>/dev/null)" != "$ORG_PIN" ]; then
    echo "Org-mode pin changed, re-cloning..."
    rm -rf "$ORG_DIR"
fi
if [ ! -d "$ORG_DIR" ]; then
    echo "Cloning pinned org-mode ($ORG_PIN)..."
    mkdir -p "$SCRIPT_DIR/.tangle-deps"
    git clone --quiet https://git.savannah.gnu.org/git/emacs/org-mode.git "$ORG_DIR"
    git -C "$ORG_DIR" checkout --quiet "$ORG_PIN"
    # Generate org-loaddefs.el and org-version.el (needed for org to load properly)
    emacs --batch --no-init-file \
        --eval "(progn
                  (push \"$ORG_DIR/lisp\" load-path)
                  (require 'autoload)
                  (setq generated-autoload-file \"$ORG_DIR/lisp/org-loaddefs.el\")
                  (update-directory-autoloads \"$ORG_DIR/lisp\"))" 2>/dev/null
    ORG_GIT_VERSION=$(git -C "$ORG_DIR" describe --tags --match "release_*" 2>/dev/null || echo "N/A")
    ORG_RELEASE=$(echo "$ORG_GIT_VERSION" | sed 's/^release_//;s/-.*//')
    (cd "$ORG_DIR/lisp" && emacs --batch --no-init-file \
        --eval "(progn
                  (push \"$ORG_DIR/lisp\" load-path)
                  (load \"$ORG_DIR/mk/org-fixup.el\")
                  (org-make-org-version \"$ORG_RELEASE\"
                                        \"$ORG_GIT_VERSION\"))") 2>/dev/null || true
fi

export PATH="$SCRIPT_DIR/tests:$PATH"

run_file() {
    local orgfile="$1"
    echo "Running $orgfile..."
    local raw_output rc=0 no_cache_arg=""
    if [ -n "$NO_CACHE" ]; then
        no_cache_arg="t"
    fi
    emacs --batch --no-init-file \
        -l "$SCRIPT_DIR/tangle.el" \
        -l "$SCRIPT_DIR/run.el" \
        --eval "(dagger-run-file \"$orgfile\" $no_cache_arg)" 2>&1 || rc=$?
    if [ "$rc" -ne 0 ]; then
        echo "ERROR: emacs run failed (exit $rc)" >&2
        return "$rc"
    fi
}

if [ $# -eq 0 ]; then
    for f in "$SCRIPT_DIR"/src/*.org "$SCRIPT_DIR"/examples/*/readme.org; do
        [ -f "$f" ] && run_file "$f"
    done
else
    for f in "$@"; do
        test -f "$(realpath "$f")" # assert thanks to set -eu
        run_file "$(realpath "$f")"
    done
fi

13. Init examples (host)

Runs :init yes bash blocks in example readme.org files to set up their dagger modules (dagger init, dagger install, local checkout rewrite). The daily-use init-examples.sh wrapper runs this inside a container for reproducibility.

With --from-scratch, first removes everything except readme.org in each example directory. 

# Initialize example dagger modules.
# Usage:
#   ./init-examples.sh                       # init all examples
#   ./init-examples.sh --no-cache            # ignore cached results
#   ./init-examples.sh --from-scratch        # clean + init (implies --no-cache)
#   ./init-examples.sh examples/foo/readme.org  # init a specific example
set -eu

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"

NO_CACHE=
FROM_SCRATCH=
while [ "${1:-}" = "--no-cache" ] || [ "${1:-}" = "--from-scratch" ]; do
    case "$1" in
        --no-cache) NO_CACHE=1; shift ;;
        --from-scratch) FROM_SCRATCH=1; NO_CACHE=1; shift ;;
    esac
done

if [ -n "$FROM_SCRATCH" ]; then
    for d in "$SCRIPT_DIR"/examples/*/; do
        find "$d" -mindepth 1 -not -name readme.org -delete 2>/dev/null || true
        find "$d" -mindepth 1 -type d -empty -delete 2>/dev/null || true
    done
fi

# Ensure pinned org-mode is cloned and up-to-date
ORG_PIN="1025e3b49a98f175b124dbccd774918360fe7e11"
ORG_DIR="$SCRIPT_DIR/.tangle-deps/org"
if [ -d "$ORG_DIR" ] && [ "$(git -C "$ORG_DIR" rev-parse HEAD 2>/dev/null)" != "$ORG_PIN" ]; then
    echo "Org-mode pin changed, re-cloning..."
    rm -rf "$ORG_DIR"
fi
if [ ! -d "$ORG_DIR" ]; then
    echo "Cloning pinned org-mode ($ORG_PIN)..."
    mkdir -p "$SCRIPT_DIR/.tangle-deps"
    git clone --quiet https://git.savannah.gnu.org/git/emacs/org-mode.git "$ORG_DIR"
    git -C "$ORG_DIR" checkout --quiet "$ORG_PIN"
    # Generate org-loaddefs.el and org-version.el (needed for org to load properly)
    emacs --batch --no-init-file \
        --eval "(progn
                  (push \"$ORG_DIR/lisp\" load-path)
                  (require 'autoload)
                  (setq generated-autoload-file \"$ORG_DIR/lisp/org-loaddefs.el\")
                  (update-directory-autoloads \"$ORG_DIR/lisp\"))" 2>/dev/null
    ORG_GIT_VERSION=$(git -C "$ORG_DIR" describe --tags --match "release_*" 2>/dev/null || echo "N/A")
    ORG_RELEASE=$(echo "$ORG_GIT_VERSION" | sed 's/^release_//;s/-.*//')
    (cd "$ORG_DIR/lisp" && emacs --batch --no-init-file \
        --eval "(progn
                  (push \"$ORG_DIR/lisp\" load-path)
                  (load \"$ORG_DIR/mk/org-fixup.el\")
                  (org-make-org-version \"$ORG_RELEASE\"
                                        \"$ORG_GIT_VERSION\"))") 2>/dev/null || true
fi

init_file() {
    local orgfile="$1"
    local example_dir
    example_dir="$(dirname "$orgfile")"
    echo "Init $orgfile..."
    local raw_output rc=0 no_cache_arg=""
    if [ -n "$NO_CACHE" ]; then
        no_cache_arg="t"
    fi
    raw_output=$(emacs --batch --no-init-file \
        -l "$SCRIPT_DIR/tangle.el" \
        -l "$SCRIPT_DIR/run.el" \
        --eval "(dagger-init-file \"$orgfile\" $no_cache_arg)" 2>&1) || rc=$?
    if [ "$rc" -ne 0 ]; then
        echo "ERROR: init failed (exit $rc):" >&2
        echo "$raw_output" >&2
        return "$rc"
    fi
    local djson="$example_dir/dagger.json"
    if [ -f "$djson" ]; then
        local dir_name
        dir_name="$(basename "$example_dir")"
        python3 -c "
import json, sys
p, want = sys.argv[1], sys.argv[2]
d = json.load(open(p))
if d.get('name') != want:
    print(f'Setting module name to {want!r} in {p}')
    d.pop('name', None)
    d = {'name': want, **d}
    json.dump(d, open(p, 'w'), indent=2)
    print()
" "$djson" "$dir_name"
    fi
}

if [ $# -eq 0 ]; then
    for f in "$SCRIPT_DIR"/examples/*/readme.org; do
        [ -f "$f" ] && init_file "$f"
    done
else
    for f in "$@"; do
        test -f "$(realpath "$f")"
        init_file "$(realpath "$f")"
    done
fi

14. Export HTML (host)

Exports all publishable org files to HTML with noweb expansion. GitHub's org-mode renderer cannot expand noweb references, so we export to HTML via Emacs and publish to GitHub Pages.

Prose styling comes from water.css — a classless drop-in with auto light/dark mode, pulled at export time and cached in .tangle-deps/. Syntax highlighting comes from htmlize emitting class-tagged <span> elements; we inline a short palette for those classes since batch Emacs has no face colors.

14.1. Export config (Emacs Lisp)

Loaded after tangle.el, so the pinned org-mode is already on load-path. Reads .tangle-deps/water.min.css (fetched by the host script) and inlines it into every exported page via org-html-head — no separate stylesheet file in _site/, so pages stay self-contained regardless of the deployment path. 

;;; export-html.el --- ox-html configuration for publishing -*- lexical-binding: t; -*-

(require 'ox-html)
(require 'htmlize)

;; Class-tagged spans so our <style> block below drives colors.
(setq org-html-htmlize-output-type 'css)

;; Rewrite file:foo.org links to foo.html so cross-page navigation
;; works on the published site.
(setq org-html-link-org-files-as-html t)

;; Don't treat "_" and "^" as subscript/superscript in prose — we
;; have plenty of identifiers like =.with_exec= and =DOCKER_HOST=
;; that would otherwise render as =.with<sub>exec</sub>=.  Explicit
;; =_{foo}= / =^{foo}= still work if anyone ever needs them.
(setq org-export-with-sub-superscripts '{})

;; water.css replaces the default prose theme; drop the built-in one.
(setq org-html-head-include-default-style nil)
(setq org-html-head-include-scripts nil)

(defun daggerlib--water-css ()
  "Return the contents of the water.css bundle in =.tangle-deps/=.
The file is fetched by =export-html-host.sh= before Emacs runs; the
exact filename encodes the pinned version and variant, so we glob
for =water-*.min.css= rather than hard-coding that here."
  (let* ((root (file-name-directory (or load-file-name buffer-file-name)))
         (hits (file-expand-wildcards
                (expand-file-name ".tangle-deps/water-*.min.css" root))))
    (unless hits
      (error "water.css missing in .tangle-deps/ (did export-html-host.sh fetch it?)"))
    (with-temp-buffer
      (insert-file-contents (car hits))
      (buffer-string))))

(defconst daggerlib--syntax-css "
.org-keyword,.org-builtin,.org-preprocessor{color:#d2a8ff}
.org-string{color:#a5d6ff}
.org-comment,.org-comment-delimiter,.org-doc{color:#8b949e;font-style:italic}
.org-function-name{color:#79c0ff}
.org-type,.org-constant{color:#ffa657}
.org-variable-name{color:#ff7b72}")

(setq org-html-head
      (concat "<style>"
              (daggerlib--water-css)
              daggerlib--syntax-css
              "</style>"))

;; "← Home" breadcrumb pointing back to the root readme.  Rendered
;; as a preamble so every page (except the home page itself) shows
;; a way back.  The href is computed relative to the page being
;; exported so deployment sub-paths (e.g. /daggerlib/) don't matter.
(defun daggerlib--home-preamble (info)
  (let* ((src (expand-file-name (plist-get info :input-file)))
         (root (locate-dominating-file src "dagger.json"))
         (root-readme (expand-file-name "readme.org" root))
         (src-dir (file-name-directory src))
         (rel (file-relative-name
               (expand-file-name "index.html" root) src-dir)))
    (if (file-equal-p src root-readme)
        ""
      (format "<nav><a href=\"%s\">← Home</a></nav>" rel))))

(setq org-html-preamble #'daggerlib--home-preamble)

(defun daggerlib/export-file (orgfile)
  "Export ORGFILE to HTML next to it (ox-html writes alongside the source)."
  (find-file orgfile)
  (org-html-export-to-html)
  (kill-buffer))

(provide 'export-html)
;;; export-html.el ends here

14.2. Host script

Walks the publishable org files, fetches water.css once into the shared .tangle-deps/ cache, invokes batch Emacs with both tangle.el and export-html.el, and moves each resulting .html into _site/ mirroring the source layout.

The water.css version is pinned in jsdelivr' URL path; bump the version here to refresh. The file is committed nowhere — Dagger's tangle-deps cache volume keeps it across runs inside the container, and locally it lives next to the pinned org-mode clone. 

# Export org files to HTML for GitHub Pages.
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"

# Ensure pinned org-mode is cloned and up-to-date
ORG_PIN="1025e3b49a98f175b124dbccd774918360fe7e11"
ORG_DIR="$SCRIPT_DIR/.tangle-deps/org"
if [ -d "$ORG_DIR" ] && [ "$(git -C "$ORG_DIR" rev-parse HEAD 2>/dev/null)" != "$ORG_PIN" ]; then
    echo "Org-mode pin changed, re-cloning..."
    rm -rf "$ORG_DIR"
fi
if [ ! -d "$ORG_DIR" ]; then
    echo "Cloning pinned org-mode ($ORG_PIN)..."
    mkdir -p "$SCRIPT_DIR/.tangle-deps"
    git clone --quiet https://git.savannah.gnu.org/git/emacs/org-mode.git "$ORG_DIR"
    git -C "$ORG_DIR" checkout --quiet "$ORG_PIN"
    # Generate org-loaddefs.el and org-version.el (needed for org to load properly)
    emacs --batch --no-init-file \
        --eval "(progn
                  (push \"$ORG_DIR/lisp\" load-path)
                  (require 'autoload)
                  (setq generated-autoload-file \"$ORG_DIR/lisp/org-loaddefs.el\")
                  (update-directory-autoloads \"$ORG_DIR/lisp\"))" 2>/dev/null
    ORG_GIT_VERSION=$(git -C "$ORG_DIR" describe --tags --match "release_*" 2>/dev/null || echo "N/A")
    ORG_RELEASE=$(echo "$ORG_GIT_VERSION" | sed 's/^release_//;s/-.*//')
    (cd "$ORG_DIR/lisp" && emacs --batch --no-init-file \
        --eval "(progn
                  (push \"$ORG_DIR/lisp\" load-path)
                  (load \"$ORG_DIR/mk/org-fixup.el\")
                  (org-make-org-version \"$ORG_RELEASE\"
                                        \"$ORG_GIT_VERSION\"))") 2>/dev/null || true
fi

WATER_CSS_VERSION="2.1.1"
WATER_CSS_VARIANT="dark"
WATER_CSS_URL="https://cdn.jsdelivr.net/npm/water.css@${WATER_CSS_VERSION}/out/${WATER_CSS_VARIANT}.min.css"
WATER_CSS_FILE="$SCRIPT_DIR/.tangle-deps/water-${WATER_CSS_VERSION}-${WATER_CSS_VARIANT}.min.css"
if [ ! -s "$WATER_CSS_FILE" ]; then
    echo "Fetching water.css@${WATER_CSS_VERSION}..."
    mkdir -p "$(dirname "$WATER_CSS_FILE")"
    curl -fsSL "$WATER_CSS_URL" -o "$WATER_CSS_FILE"
fi

SITE_DIR="$SCRIPT_DIR/_site"
rm -rf "$SITE_DIR"
mkdir -p "$SITE_DIR"

export_file() {
    local orgfile="$1"
    local relpath
    relpath="$(realpath --relative-to="$SCRIPT_DIR" "$orgfile")"
    local outdir="$SITE_DIR/$(dirname "$relpath")"
    mkdir -p "$outdir"
    echo "Exporting $relpath..."
    local rc=0
    emacs --batch --no-init-file \
        -l "$SCRIPT_DIR/tangle.el" \
        -l "$SCRIPT_DIR/export-html.el" \
        --eval "(daggerlib/export-file \"$orgfile\")" 2>&1 || rc=$?
    if [ "$rc" -ne 0 ]; then
        echo "ERROR: export failed for $relpath (exit $rc)" >&2
        return "$rc"
    fi
    local htmlfile="${orgfile%.org}.html"
    if [ -f "$htmlfile" ]; then
        mv "$htmlfile" "$outdir/"
    fi
}

for f in "$SCRIPT_DIR"/readme.org "$SCRIPT_DIR"/TECHNICAL.org \
         "$SCRIPT_DIR"/src/*.org "$SCRIPT_DIR"/tests/*.org \
         "$SCRIPT_DIR"/examples/*/readme.org; do
    [ -f "$f" ] && export_file "$f"
done

# Root readme becomes index.html
if [ -f "$SITE_DIR/readme.html" ]; then
    cp "$SITE_DIR/readme.html" "$SITE_DIR/index.html"
fi

15. Containerized wrappers

The daily-use scripts run their host counterparts inside a Dagger container for reproducible emacs output. Each wrapper calls the corresponding dind-* function and exports the result back to the working tree with -o ..

15.1. run.sh

set -eu
cd "$(dirname "$0")"
args=""
if [ "${1:-}" = "--no-cache" ]; then
    args="--no-cache"
    shift
fi
for f in "$@"; do
    args="$args --files=$f"
done
exec dagger ${DAGGER_EXTRA_ARGS:-} call dind-run-org $args -o .

15.2. tangle.sh

set -eu
cd "$(dirname "$0")"
exec dagger ${DAGGER_EXTRA_ARGS:-} call dind-tangle -o .

15.3. init-examples.sh

set -eu
cd "$(dirname "$0")"
args=""
if [ "${1:-}" = "--from-scratch" ]; then
    args="--from-scratch"
    shift
elif [ "${1:-}" = "--no-cache" ]; then
    args="--no-cache"
    shift
fi
exec dagger ${DAGGER_EXTRA_ARGS:-} call dind-init-examples $args -o .

15.4. export-html.sh

set -eu
cd "$(dirname "$0")"
exec dagger ${DAGGER_EXTRA_ARGS:-} call export-html export --path _site

Author: root

Created: 2026-04-18 Sat 21:16

Validate