Konubinix' opinionated web of thoughts

Trying the Zsa-Voyager With Ergo-L

Fleeting

authorship
this note is edited using Claude/default, governed by the note how to do literate programming
tl;dr
ergo-l on a voyager, the firmware staying qwerty while xkb owns the characters, the leds telling what keys do and the computer able to put them out.
rule
this note carries no promise driven development.
why
we found no way to test those promises to our taste.

tl;dr, building and flashing my firmware

how
one nix call takes the keymap and hands back the bin it put in the store.

KEYMAP="${KEYMAP_TAR%%\?*}" nix build --impure ~/.cache/voyager-ergol \
    --print-out-paths --no-link
https://ipfs.konubinix.eu/p/bafkreidzesuuaq5euzfoffuhzij6ulwseq5jurnrdpbkfvtr34oon4tkfe?ms57294nywbf16kj40kr7hwhbrr4kpd1_zsa_voyager_konubinix.bin
therefore
with the keyboard in bootloader, the second call puts that bin on it.

lsusb -d 3297:0791 > /dev/null || {
    echo "not in bootloader: press the boot key of L2,"
    echo "or plug it back in holding esc, which erases what it remembers too"
    exit 1
}

dfu-util --device 3297:0791 --alt 0 \
    --dfuse-address 0x08002000:leave \
    --download "${firmware%%\?*}" | sed '/\r/d'
dfu-util 0.11

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2021 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

Opening DFU capable USB device...
Device ID 3297:0791
Device DFU version 011a
Claiming USB DFU Interface...
Setting Alternate Interface #0 ...
Determining device status...
DFU state(2) = dfuIDLE, status(0) = No error condition is present
DFU mode device DFU version 011a
Device returned transfer size 2048
DfuSe interface name: "Internal Flash   "
Downloading element to address = 0x08002000, size = 56868
Erase    done.
Download done.
File downloaded successfully
Submitting leave request...
Transitioning to dfuMANIFEST state

tl;dr, the cheatsheet I learn from

what
a drawing of my layout carrying the ergol characters on each key.
why
I learn the keyboard off it.
how
it is read off the keymap qmk compiled, so it cannot drift from what I flashed.

what ergol prints on each key

from keymap_drawer.config import ParseConfig
from xkbcommon import xkb

# The alphanumeric keys. Above them the keypad repeats the same glyphs.
SPACE = 65
MAIN_BLOCK = [*range(9, 62), SPACE]

# The spaces ergol puts on that key are invisible, so name them.
VISIBLE = {" ": "␣", " ": "⍽", " ": "⎵"}

# ergol reaches its accented letters through the ★ dead key, which is level 5.
ONE_DEAD_KEY = 4

# Level 4: the maths and the superscripts, plus the non-breaking space.
ALTGR_SHIFT = 3

context = xkb.Context()


def glyphs(layout, variant, held):
    """What each key of the main block produces, with `held` down."""
    keymap = context.keymap_new_from_names(layout=layout, variant=variant)
    state = keymap.state_new()
    if held:
        state.update_key(keymap.key_by_name(held), xkb.XKB_KEY_DOWN)
    return {code: state.key_get_string(code) for code in MAIN_BLOCK}


def level(layout, variant, index):
    """What each key produces at that level, for levels no modifier reaches."""
    keymap = context.keymap_new_from_names(layout=layout, variant=variant)
    out = {}
    for code in MAIN_BLOCK:
        syms = []
        if keymap.num_levels_for_key(code, 0) > index:
            syms = keymap.key_get_syms_by_level(code, 0, index)
        out[code] = (xkb.keysym_to_string(syms[0]) or "") if syms else ""
    return out


def ergol_levels():
    """Every key of ergol, as (base, shifted, altgr, dead key, altgr+shift)."""
    base = glyphs("fr", "ergol", None)
    shifted = glyphs("fr", "ergol", "LFSH")
    altgr = glyphs("fr", "ergol", "RALT")
    dead = level("fr", "ergol", ONE_DEAD_KEY)
    hard = level("fr", "ergol", ALTGR_SHIFT)
    return {
        code: (base[code] or "★", shifted[code], altgr[code], dead[code], hard[code])
        for code in MAIN_BLOCK
    }


def keycode_map():
    """keymap-drawer's qmk_keycode_map, with the ergol levels of that same key."""
    plain = glyphs("us", None, None)
    caps = glyphs("us", None, "LFSH")
    ergol = ergol_levels()

    # A keycode that already means "shift + something" lands on one glyph only.
    unshifted = {plain[code]: ergol[code] for code in MAIN_BLOCK if plain[code]}
    shifted = {
        caps[code]: (ergol[code][1], "", "", "", "")
        for code in MAIN_BLOCK
        if caps[code]
    }

    # Letters are absent from that map: it falls back to the keycode name.
    out = {glyph.upper(): levels for glyph, levels in unshifted.items() if glyph.isalnum()}

    for keycode, glyph in ParseConfig().qmk_keycode_map.items():
        out[keycode] = unshifted.get(glyph) or shifted.get(glyph) or (glyph, "", "", "", "")

    out["SPACE"] = out["SPC"] = ergol[SPACE]
    return out


for keycode, (base, shifted, altgr, dead, hard) in keycode_map().items():
    # Showing "q Q" on every letter is noise.
    shifted = "" if shifted == base.upper() else shifted
    # Keycaps are printed in majuscule.
    base = base.upper() if base.isalpha() else base
    # A cell cannot hold a bare pipe.
    cells = [
        VISIBLE.get(cell, cell).replace("|", r"\vert")
        for cell in (base, shifted, altgr, dead, hard)
    ]
    print("%s|%s|%s|%s|%s|%s" % (keycode, *cells))

# noweb lays the python out under the line it stands on, hence the dedent.
DEDENT='import sys, textwrap; exec(textwrap.dedent(sys.stdin.read()))'
nix shell --impure --expr 'with import <nixpkgs> {};
    python3.withPackages (ps: [ ps.xkbcommon ps.keymap-drawer ])' \
    --command python3 -c "$DEDENT" <<'GLYPHS' # the ergol levels
<<ergol_glyphs_py>>
GLYPHS
1 1 € ₁ „ ¹
2 2 « ₂ “ ²
3 3 » ₃ ” ³
4 4 $ ₄ ¢ ⁴
5 5 % ₅ ‰ ⁵
6 6 ^ ₆ ⁶
7 7 & ₇ ⁷
8 8 * ₈ § ⁸
9 9 # ₉ ⁹
0 0 @ ₀ ° ⁰
Q Q ^ â
W C < ç ≤
E O > œ ≥
R P $ ô
T W % ‰
Y J @
U M & µ
I D * _ ×
O ★ ! '
P Y ` û
A A { à
S S ( é
D E ) è
F N } ê
G F = ñ ≠
H L \ (
J R + ) ±
K T - î
L I / ï ÷
Z Z ~ æ
X X [ ß
C - ? ] ‑
V V _ – –
B B # —
N . : | … ¦
M H ! ¬
XXXXXXX
NO
MINUS / _ ÷
MINS / _ ÷
EQUAL = + ≠
EQL = + ≠
LEFT_BRACKET [ {
LBRC [ {
RIGHT_BRACKET ] }
RBRC ] }
BACKSLASH \ |
BSLS \ |
NONUS_HASH »
NUHS »
SEMICOLON U " ù
SCLN U " ù
QUOTE ' "
QUOT ' "
GRAVE ` ~
GRV ` ~
COMMA G ;
COMM G ;
DOT , ; : · :
SLASH K ?
SLSH K ?
TILDE ~
TILD ~
EXCLAIM €
EXLM €
AT «
HASH »
DOLLAR $
DLR $
PERCENT %
PERC %
CIRCUMFLEX ^
CIRC ^
AMPERSAND &
AMPR &
ASTERISK *
ASTR *
LEFT_PAREN #
LPRN #
RIGHT_PAREN @
RPRN @
UNDERSCORE _
UNDS _
PLUS +
LEFT_CURLY_BRACE {
LCBR {
RIGHT_CURLY_BRACE }
RCBR }
PIPE |
COLON U
COLN U
DOUBLE_QUOTE "
DQUO "
DQT "
LEFT_ANGLE_BRACKET G
LABK G
LT G
RIGHT_ANGLE_BRACKET ;
RABK ;
GT ;
QUESTION K
QUES K
SPACE ⎵ ⍽ ⎵ ’  
SPC ⎵ ⍽ ⎵ ’  

what the drawer is told

import json
import os
import subprocess
import sys
import tempfile

<<board_py>>

qmk_source(os.path.expanduser("~/.cache/voyager-ergol"))
from qmk.keycodes import load_spec

# Keycodes keymap-drawer leaves as their raw name.
RENAMES = {
    # Shift+backspace is a key override, absent from the keymaps array.
    "BSPC": {"t": "DEL", "s": "suppr"},
    "LEFT_GUI": "super",
    "RIGHT_GUI": "super",
    "CW_TOGG": "caps word",
    "QK_BOOT": "boot",
    "PGUP": "PAGE UP",
    "PGDN": "PAGE DOWN",
    "VOLD": "vol -",
    "VOLU": "vol +",
    "MUTE": "mute",
    "MPRV": "prev",
    "MNXT": "next",
    "MSTP": "stop",
    "MPLY": "play",
    "RM_TOGG": "rgb",
    "RM_NEXT": "rgb anim",
    "RGB_M_P": "rgb solid",
    "RM_VALD": "dimmer",
    "RM_VALU": "brighter",
    "LAYER_COLORS_ON": "layer colors",
    # rgb_matrix_sethsv with those hues, per process_record_user.
    "HUE_RED": "red",
    "HUE_GREEN": "green",
    "HUE_BLUE": "blue",
}

# What a glyph is coloured by is the modifier that reaches it.
INKS = {
    "shift": "#b03a2e",
    "altgr": "#1a5fb4",
    "both": "#7d3c98",
    "dead": "#1e7d32",
    # The modifiers that select no layout level.
    "ctrl": "#c98200",
    "alt": "#0e8a8a",
    "super": "#c0399f",
    "plain": "#6b6f76",
}

STYLE = """
rect.homerow { fill: #d7e3f2; }
rect.thumbrest { fill: #eaf0f8; }
/* Restated last: the thumb that reaches the layer stays the held pink. */
rect.held { fill: #fdd; }
text.glyph.tl { font-size: 24px; }
text.tr, text.left, text.right { font-size: 18px; }
text.glyph.tap { font-size: 15px; }
text.hold { font-size: 12px; }
text.left, text.right { dominant-baseline: alphabetic; }
text.left, text.right { transform: translateY(14px); }
text.hold { transform: translateY(11px); }
text.glyph.tap { transform: translateY(9px); }
text.tr, text.shifted, text.shiftmod.hold { fill: %(shift)s; }
text.right, text.altgrmod.hold { fill: %(altgr)s; }
text.glyph.tap { fill: %(both)s; }
text.left, text.deadkeymod.tl { fill: %(dead)s; }
text.plainmod.hold, text.modkey.tap { fill: %(plain)s; }
text.ctrlmod.hold { fill: %(ctrl)s; }
text.altmod.hold { fill: %(alt)s; }
text.supermod.hold { fill: %(super)s; }
""" % INKS

# footer_text lands raw inside a <text>, so it can carry its own colours.
LEGEND = (
    'base' +
    ' · <tspan fill="%(shift)s">shift</tspan>' +
    ' · <tspan fill="%(altgr)s">altgr</tspan>' +
    ' · <tspan fill="%(both)s">altgr+shift</tspan>' +
    ' · <tspan fill="%(dead)s">★ then key</tspan>'
) % INKS

# The config leaves as its own file, which the drawing block is handed.
OUT = tempfile.mkdtemp()

with open(os.path.join(OUT, "ergol.json"), "w") as f:
    keycode_map = {}
    for keycode, base, shifted, altgr, dead, hard in glyphs:
        spec = {"t": base, "s": shifted, "right": altgr, "left": dead, "br": hard}
        keycode_map[str(keycode)] = {
            field: str(value).replace(r"\vert", "|")
            for field, value in spec.items()
            if value != ""
        }
    keycode_map.update(RENAMES)
    # keymap-drawer shows the name it is given, where the sdk knows the aliases.
    for keycode in load_spec("latest")["keycodes"].values():
        name = keycode["key"].split("KC_")[-1]
        for alias in keycode.get("aliases", []):
            keycode_map.setdefault(alias.split("KC_")[-1], name)
    json.dump(
        {
            "parse_config": {"qmk_keycode_map": keycode_map},
            "draw_config": {
                "svg_extra_style": STYLE,
                "footer_text": LEGEND,
                "shrink_wide_legends": 5,
                "key_w": 76,
                "key_h": 72,
                # Clearance between a corner legend and the key border.
                "small_pad": 15,
            },
        },
        f,
    )

print(os.path.join(OUT, "ergol.json"))
https://ipfs.konubinix.eu/p/bafkreig65fjfdkuz57na67wfr25oxs4mahbyxvlycexn2erk672lffzpqy?ergol.json

import json
import sys

import yaml

<<modifiers_py>>

# A key that is only a modifier, rather than a modifier held on a letter.
MOD_NAMES = set(MODIFIERS.values())

# Third of the four rows, minus the outer pinky column and the inner index
# column, which are stretches: four fingers per hand actually rest here.
HOME_ROW = (25, 26, 27, 28, 31, 32, 33, 34)

# Of the two keys per thumb, the layer-tap is the one under the thumb at rest.
THUMBS = (48, 51)

path, config_path = sys.argv[1], sys.argv[2]
keymap = yaml.safe_load(open(path, encoding="utf-8"))
config = json.load(open(config_path, encoding="utf-8"))

# keymap-drawer builds mod-taps as tap/hold/shifted only, dropping left and
# right, which is exactly where the accents and the symbols live.
levels = {
    spec["t"]: spec
    for spec in config["parse_config"]["qmk_keycode_map"].values()
    if isinstance(spec, dict) and "t" in spec
}

for layer in keymap["layers"].values():
    for index, key in enumerate(layer):
        # A key with nothing but a tap is a bare string, and cannot carry a class.
        if isinstance(key, str):
            key = layer[index] = {"t": key}
        if not isinstance(key, dict):
            continue
        # keymap-drawer already typed the thumb that reaches the layer as
        # held, and the transparent keys as trans. Keep what it found.
        classes = key.get("type", "").split()
        if index in HOME_ROW:
            classes.append("homerow")
        elif index in THUMBS:
            classes.append("thumbrest")
        if key.get("h") in MODIFIERS:
            key["h"] = MODIFIERS[key["h"]]
            classes.append(key["h"] + "mod")
        # The altgr backspace is left out of the override, to keep its level.
        if key.get("t") == "DEL" and key.get("h") == "altgr":
            key.pop("s", None)
        if key.get("t") == "★":
            classes.append("deadkeymod")
        if key.get("t") in MOD_NAMES:
            classes.append("modkey")
        for side in ("left", "right", "br"):
            if side not in key and side in levels.get(key.get("t"), {}):
                key[side] = levels[key["t"]][side]

        # A layout glyph goes to the top corners, leaving the sides for the
        # other levels and the bottom middle for the modifier name. A word
        # label needs the whole key and stays in the middle.
        if len(str(key.get("t", ""))) == 1:
            classes.append("glyph")
            key["tl"] = key.pop("t")
            if "s" in key:
                key["tr"] = key.pop("s")
                # The middle is free once the letter moved to the corner, and it is
                # the only spot altgr+shift does not land on the altgr glyph.
            if "br" in key:
                key["t"] = key.pop("br")
        if classes:
            key["type"] = " ".join(classes)

with open(path, "w", encoding="utf-8") as f:
    yaml.safe_dump(keymap, f, allow_unicode=True, sort_keys=False)

the drawing itself

how
the drawing reads the keymap qmk compiled, so it shows what the keyboard does.

ergol="${ERGOL%%\?*}"
keymap=$(KEYMAP="${KEYMAP_TAR%%\?*}" nix build --impure \
    ~/.cache/voyager-ergol#keymap-json --print-out-paths --no-link)

# noweb lays the python out under the line it stands on, hence the dedent.
DEDENT='import sys, textwrap; exec(textwrap.dedent(sys.stdin.read()))'
# The drawing is made in a scratch, only its last file being kept.
cd "$(mktemp -d)"
nix shell nixpkgs#keymap-drawer --command \
    keymap -c "$ergol" parse -q "$keymap" -o voyager.yaml
nix shell nixpkgs#python3Packages.pyyaml --command \
    python3 -c "$DEDENT" voyager.yaml "$ergol" <<'MARKS' # the mod marks
<<mark_mods_py>>
MARKS
nix shell nixpkgs#keymap-drawer --command \
    keymap -c "$ergol" draw voyager.yaml -o cheatsheet.svg
nix shell nixpkgs#librsvg --command \
    rsvg-convert -z 1.5 -b white cheatsheet.svg -o cheatsheet.png
printf '%s' "$PWD/cheatsheet.png"

tl;dr, the leds

what
on the base layer, each home row key lights in the colour of the modifier it holds.
  • example:
what
with layer 1 held, the function row goes yellow and the numpad takes the altgr blue.
  • example:
what
with layer 2 held, the led keys go white, the media green, the arrows blue, boot red.
  • example:
how
from the computer, clk voyager lights --off puts them out, and --on gives them back.

I want to type ergol on the voyager

what
ergo-l on a 4x6 compact is the layout, drawn for the shape the voyager has.
example
that drawing.
rule
ergol itself sends the firmware away from the job1, 2.
  • why: everything around a keyboard reads it as qwerty, so a firmware that spells ergol out fights them all.
therefore
the firmware stays qwerty and xkb is what owns the characters.
scope
the only thing the hardware owes ergol is an altgr under a thumb3.

only the voyager should speak ergol

what
setxkbmap fr ergol and setxkbmap fr bepo switch the whole session, which is one layout too few.
therefore
xorg is told to reach one device, so the voyager takes ergol and the typematrix keeps bépo.
{ ... }:
{
  services.xserver.inputClassSections = [
    ''
      Identifier "voyager-ergol"
      MatchIsKeyboard "on"
      MatchUSBID "3297:1977"
      Option "XkbLayout" "fr"
      Option "XkbVariant" "ergol"
    ''
  ];
}

first, the layout the keyboard comes with

what
the default voyager keymap is what ZSA ships, and where every layout below starts.
therefore
it is drawn here, so what I move is read against it.

<<imports_py>>

<<board_py>>

<<layers_py>>

qmk_home = qmk_source(FLAKE)
from qmk.keymap import generate_json

default = default_of(qmk_home)
out = os.path.join(tempfile.mkdtemp(), "default.json")
with open(out, "w") as f:
    json.dump(generate_json(
        keymap="default",
        keyboard="zsa/voyager",
        layout=default[0]["layout"],
        layers=[layer["keycodes"] for layer in default],
    ), f)

print(out)
what
that drawing names the keycodes, which is not what my fingers will meet.
therefore
the same layout is drawn again with what ergol prints on each of them.
therefore
reading those two is what tells me what is already there, and what is not.

after some time on it, I converged to my own setting

rule
only what I changed from the default keymap is said here.
why
the fork already carries the rest, and the build reads it from there.

first, a way to name a key

therefore
the board is drawn once with a number on each key, and the changes below point at it.

import sys

import yaml

path = sys.argv[1]
keymap = yaml.safe_load(open(path, encoding="utf-8"))

one = next(iter(keymap["layers"]))
keymap["layers"] = {
    "positions": [{"t": str(index)}
                  for index, _ in enumerate(keymap["layers"][one])]
}
# The levels of ergol are not what this one is about.
keymap["draw_config"] = {"footer_text": ""}

with open(path, "w", encoding="utf-8") as f:
    yaml.safe_dump(keymap, f, allow_unicode=True, sort_keys=False)

the modifiers belong under my fingers

what
the home row holds the modifiers, which the default leaves out.
why
emacs makes use of the modifiers a lot.
what
the modifiers fall symmetrically on the two hands.
why
it eases remembering them.
therefore
the thumbs and the bottom row give up the ones the default put on them.

HOME_ROW = {
    24: "MOD_RALT", 25: "MOD_LGUI", 26: "MOD_LALT",
    27: "MOD_LCTL", 28: "MOD_LSFT",
    31: "MOD_RSFT", 32: "MOD_RCTL", 33: "MOD_LALT",
    34: "MOD_RGUI", 35: "MOD_RALT",
}

PLAIN = (37, 46, 49, 50)
FREED = (36, 47)

the numpad owes me an ergol minus and dot

what
layer 1 gives the -, +, /, *, . and = the numpad beside it needs.
  • why: layer 1 is the default one, which puts that numpad there.
how
under the hood the keycodes stay qwerty, xkb turning each one into its ergol character.
therefore
the minus and the dot of that row are written KC_C and KC_N, the keys ergol prints them on.

the leds want keys of their own

how
layer 2 takes the keycodes python defines, in place of the layer colour key the default has.
therefore
the three hues sit on the row below, where the default leaves the keys through.

CHANGES = {
    1: {21: "KC_C", 22: "KC_MINS", 45: "KC_N"},
    2: {1: "LAYER_COLORS_ON",
        39: "HUE_RED", 40: "HUE_GREEN", 41: "HUE_BLUE"},
}

there is no suppr, and I want one under my thumb

what
neither the default nor ergol gives me a suppr I can reach without leaving the home row.
therefore
shift on the backspace my thumb rests on gives it, which no layout tool can say.

so my layers are the default plus that

how
the sdk reads the default keymap out of the fork the build compiles against.
therefore
nothing of it is said twice, and a key I left alone stays whatever it becomes there.

def tap_of(keycode):
    held = re.match(r"\w+_T\((.*)\)$", keycode)
    return held.group(1) if held else keycode


def default_of(qmk_home):
    from qmk.keymap import parse_keymap_c

    return parse_keymap_c(os.path.join(
        qmk_home, "keyboards/zsa/voyager/keymaps/default/keymap.c"
    ))["layers"]


def layers_of(qmk_home):
    default = [layer["keycodes"] for layer in default_of(qmk_home)]

    base = default[0]
    for index, modifier in HOME_ROW.items():
        base[index] = "MT(%s, %s)" % (modifier, tap_of(base[index]))
    for index in PLAIN:
        base[index] = tap_of(base[index])
    for index in FREED:
        base[index] = "KC_TRNS"

    for layer, changes in CHANGES.items():
        for index, keycode in changes.items():
            default[layer][index] = keycode
    return default

the low level it takes to get there

what
qmk takes no layout, it takes a folder of c it compiles.
therefore
python writes every file of that folder, and hands it over as one.

the keymap qmk compiles, and the suppr spliced into it

how
the folder carries a keymap.json naming two modules.
  • why: the board’s own code calls LED_LEVEL and TOGGLE_LAYER_COLOR, which zsa/defaults declares.
  • why: zsa/oryx speaks a raw hid, through which the computer drives the leds4.

def modules_json():
    return json.dumps({"modules": ["zsa/defaults", "zsa/oryx"]}, indent=2) + "\n"
how
the keymaps array is the layout with its rows written out as qmk spells them.

def keymap_c(layers):
    from qmk.keymap import generate_c, generate_json

    # The sdk writes the array, out of the format it owns.
    written = generate_c(generate_json(
        keymap="konubinix",
        keyboard="zsa/voyager",
        layout="LAYOUT_voyager",
        layers=layers,
    ))

    keycodes = ["enum custom_keycodes {"]
    for index, name in enumerate(OWN):
        seed = " = QK_USER" if index == 0 else ""
        keycodes.append("    %s%s," % (name, seed))
    keycodes += ["};", ""]

    # The declarations have to stand before the array that names them.
    head, rest = written.split("\n", 1)
    # qmk compiles this file and includes it too, so it guards itself.
    out = ["#pragma once", "", head, "", '#include "ledmap.h"', ""]
    out += keycodes + [rest]
    return "\n".join(out) + "\n" + handlers_c()
how
the override and the cases of my own keycodes close that same file.

def handlers_c():
    out = [
        "const key_override_t shift_backspace =",
        "    ko_make_basic(MOD_MASK_SHIFT, KC_BSPC, KC_DEL);",
        "",
        "const key_override_t *key_overrides[] = {",
        "    &shift_backspace,",
        "    NULL,",
        "};",
        "",
        "bool process_record_user(uint16_t keycode, keyrecord_t *record) {",
        "    if (!record->event.pressed) {",
        "        return true;",
        "    }",
        "    switch (keycode) {",
    ]
    for name, body in OWN.items():
        out.append("        case %s:" % name)
        out.append("            %s" % body)
        out.append("            return false;")
    out += ["    }", "    return true;", "}", ""]
    return "\n".join(out)

one folder, one file

how
the board’s own json says which matrix each key and each led sits on.
therefore
the led of a key is found without the note saying where either sits.

def board_of(flake):
    board = subprocess.run(
        ["nix", "build", flake + "#board", "--print-out-paths", "--no-link"],
        capture_output=True, text=True, check=True,
    ).stdout.strip()
    board = json.load(open(board))
    return ([tuple(m) for m in board["keys"]],
            [tuple(m) for m in board["leds"]])


def qmk_source(flake):
    qmk_home = subprocess.run(
        ["nix", "eval", "--raw", flake + "#qmk_source"],
        capture_output=True, text=True, check=True,
    ).stdout.strip()
    # The sdk reads its own data out of the tree it is standing in.
    os.environ["QMK_HOME"] = qmk_home
    os.environ["ORIG_CWD"] = os.getcwd()
    sys.path.insert(0, os.path.join(qmk_home, "lib", "python"))
    os.chdir(qmk_home)
    return qmk_home
how
five files make the folder qmk is handed, and each one is written here.

def folder(layers, levels, keys, leds, effect):
    return {
        "keymap.json": modules_json(),
        "keymap.c": keymap_c(layers),
        "ledmap.h": ledmap_h(layers, levels, keys, leds),
        "rgb_matrix_user.inc": effect,
        "rules.mk": "\n".join(FEATURES) + "\n",
    }
how
the folder leaves as one file, so the whole keymap travels as the result of one block.

def tarred(written):
    scratch = tempfile.mkdtemp()
    for name, body in written.items():
        with open(os.path.join(scratch, name), "w") as f:
            f.write(body)

    keymap = os.path.join(tempfile.mkdtemp(), "keymap.tar")
    with tarfile.open(keymap, "w") as tar:
        for name in written:
            tar.add(os.path.join(scratch, name), arcname=name)
    return keymap
therefore
one call writes the folder, and nothing of the layout is said twice.

<<imports_py>>

<<colours_py>>
<<modifiers_py>>
<<families_py>>
<<own_py>>
<<home_row_py>>
<<changes_py>>
<<layers_py>>
<<emit_modules_py>>
<<emit_handlers_py>>
<<emit_keymap_py>>
<<classify_py>>
<<emit_ledmap_py>>
<<board_py>>
<<written_py>>
<<tar_py>>

EFFECT = """
<<effect_c>>
"""

keys, leds = board_of(FLAKE)
qmk_home = qmk_source(FLAKE)
levels = {str(row[0]): str(row[1]) for row in glyphs}

print(tarred(folder(
    layers_of(qmk_home), levels, keys, leds, EFFECT.lstrip("\n"))))
https://ipfs.konubinix.eu/p/bafkreiftez3lqzkzib57cvsvhdb3k74gexrmhcnxljwtj6kg762ieemp74?keymap.tar

turning it into a bin

what
the build is against ZSA’s own qmk fork5, an arm toolchain beside it.
  • why: the layout starts from their default, whose rgb keycodes mainline has renamed6.
how
the flake takes the keymap python tarred as its source, so the bin comes out of a derivation.
therefore
no tree is left lying around to drift from what was flashed.

{
  inputs = {
    nixpkgs.url = "nixpkgs";
    qmk_firmware = {
      url = "git+https://github.com/zsa/qmk_firmware?submodules=1&shallow=1";
      flake = false;
    };
  };

  outputs =
    { nixpkgs, qmk_firmware, ... }:
    let
      pkgs = nixpkgs.legacyPackages.x86_64-linux;
      board = "${qmk_firmware}/keyboards/zsa/voyager/keyboard.json";
    in
    {
      <<flake_serves>>

      <<flake_bin>>

      <<flake_keymap_json>>
    };
}

what the flake serves

how
the fork itself is served, so the python reaches qmk’s own keymap writer.
how
a python is served with what the qmk command asks for, lib/python asking the same.
how
the board’s json is boiled down to the matrix of each key and of each led.

qmk_source = qmk_firmware;

packages.x86_64-linux.python =
  pkgs.python3.withPackages (ps: pkgs.qmk.propagatedBuildInputs);

packages.x86_64-linux.board = pkgs.runCommand "board.json" { } ''${pkgs.jq}/bin/jq '
  {keys: [.layouts.LAYOUT.layout[].matrix],
   leds: [.rgb_matrix.layout[].matrix]}
' ${board} > $out
'';

the bin itself

how
qmk builds inside its own tree, so the fork is copied and the folder dropped in it.

packages.x86_64-linux.default = pkgs.stdenv.mkDerivation {
  name = "zsa_voyager_konubinix.bin";
  # The keymap arrives as one file, named by the block that wrote it.
  src = /. + (builtins.getEnv "KEYMAP");
  unpackCmd = "tar xf $curSrc";
  sourceRoot = ".";

  # qmk propagates gcc-arm-embedded, gnumake and dfu-util.
  nativeBuildInputs = [ pkgs.qmk ];

  buildPhase = ''export HOME=$TMPDIR QMK_HOME=$TMPDIR/qmk
    cp -r ${qmk_firmware} $QMK_HOME
    chmod -R u+w $QMK_HOME
    keymap=$QMK_HOME/keyboards/zsa/voyager/keymaps/konubinix
    mkdir -p $keymap && cp *.c *.h *.inc *.json *.mk $keymap
    # version.h is stamped from git describe, the store copy has no .git
    qmk compile -e SKIP_VERSION=yes -kb zsa/voyager -km konubinix
  '';

  installPhase = "cp $QMK_HOME/*.bin $out";
};

the json the drawing needs

how
the cheatsheet reads a json, which qmk makes back out of the keymap it compiled.
  • why: keymap-drawer knows LAYOUT for this board and nothing else.

packages.x86_64-linux.keymap-json = pkgs.stdenv.mkDerivation {
  name = "voyager.json";
  src = /. + (builtins.getEnv "KEYMAP");
  unpackCmd = "tar xf $curSrc";
  sourceRoot = ".";
  nativeBuildInputs = [
    pkgs.qmk
    pkgs.jq
  ];

  buildPhase = ''export HOME=$TMPDIR QMK_HOME=${qmk_firmware}
    qmk c2json -kb zsa/voyager -km konubinix --no-cpp keymap.c -o keymap.json
    jq 'del(.layout)' keymap.json > $out
  '';

  dontInstall = true;
};

getting fancy with the leds

what
the leds should tell me what a key does, rather than run a pattern.
why
they are helpers, and a layer I am still learning is where I need one.

what colour a key deserves

how
each family of keys takes the hue the cheatsheet gives it, saturated.
  • why: the inks of the cheatsheet are mixed for paper, where the leds are not.

COLOURS = {
    "shift": "0xff, 0x17, 0x00",
    "altgr": "0x00, 0x73, 0xff",
    "layer": "0xb2, 0x00, 0xff",
    "command": "0x00, 0xff, 0x37",
    "function": "0xff, 0xf0, 0x00",
    "led": "0xff, 0xff, 0xff",
    "ctrl": "0xff, 0xa5, 0x00",
    "alt": "0x00, 0xff, 0xff",
    "super": "0xff, 0x00, 0xc1",
    "letter": "0x10, 0x14, 0x18",
    "punctuation": "0xb0, 0xb0, 0xb0",
    "off": "0x00, 0x00, 0x00",
}
how
a key held as a modifier falls in the family of that modifier.
therefore
the cheatsheet colours its modifiers off this same table.

MODIFIERS = {
    "MOD_LSFT": "shift",
    "MOD_RSFT": "shift",
    "MOD_RALT": "altgr",
    "MOD_LALT": "alt",
    "MOD_LCTL": "ctrl",
    "MOD_RCTL": "ctrl",
    "MOD_LGUI": "super",
    "MOD_RGUI": "super",
}
how
away from the base layer a key falls by what it does, not by what it types.

FAMILIES = (
    (r"KC_F\d+$", "function"),
    (r"(RM_|RGB_|HUE_|LAYER_COLORS_ON)", "led"),
    (r"KC_(VOLD|VOLU|MUTE|MPRV|MNXT|MSTP|MPLY)$", "command"),
    (r"KC_(UP|DOWN|LEFT|RGHT|HOME|END|PGUP|PGDN)$", "altgr"),
    (r"C\(", "punctuation"),
    (r"QK_BOOT$", "shift"),
)
how
my four led keys each run a line of c, written below.
how
each feature used somewhere above is turned on here.
  • what: my own led effect, which the led keys switch to.
  • what: the override, which makes shift and backspace a suppr.
  • what: caps word, whose key comes with the default layout.
  • what: the keyboard’s own effects, without which oryx does not paint what the computer sends7.

OWN = {
    "LAYER_COLORS_ON": "rgb_matrix_mode(RGB_MATRIX_CUSTOM_LAYER_COLORS);",
    "HUE_RED": "rgb_matrix_sethsv(0, 255, 255);",
    "HUE_GREEN": "rgb_matrix_sethsv(74, 255, 255);",
    "HUE_BLUE": "rgb_matrix_sethsv(169, 255, 255);",
}

FEATURES = (
    "CAPS_WORD_ENABLE = yes",
    "KEY_OVERRIDE_ENABLE = yes",
    "RGB_MATRIX_CUSTOM_USER = yes",
    "RGB_MATRIX_CUSTOM_KB = yes",
)

the ledmap

how
a key with no family of its own falls by the character ergol prints on it.
how
a letter and a sign are told apart, and the number row of the base layer is left dark.

def classify(keycode, base, levels):
    held = re.match(r"MT\((MOD_\w+),", keycode)
    if held:
        return MODIFIERS[held.group(1)]
    if keycode.startswith("LT("):
        return "layer"
    for pattern, name in FAMILIES:
        if re.match(pattern, keycode):
            return name
    glyph = levels.get(keycode.split("KC_")[-1].rstrip(")"), "")
    if len(glyph) != 1:
        return "command" if keycode == "CW_TOGG" else "off"
    if glyph.isdigit():
        return "off" if base else "altgr"
    return "letter" if glyph.isalpha() else "punctuation"
how
the leds run down one hand then the other, so a key is found by its matrix.

def ledmap_h(layers, levels, keys, leds):
    to_led = {keys.index(m): i for i, m in enumerate(leds)}
    out = ["const uint8_t PROGMEM ledmap[][RGB_MATRIX_LED_COUNT][3] = {"]
    for index, layer in enumerate(layers):
        paint = ["off"] * len(leds)
        for position, keycode in enumerate(layer):
            paint[to_led[position]] = classify(keycode, index == 0, levels)
        out.append("    [%d] = {" % index)
        for name in paint:
            out.append("        {%s}," % COLOURS[name])
        out.append("    },")
    out += ["};", ""]
    return "\n".join(out)

the effect that paints it

how
the effect reads that table, so the leds follow the layer and dim with the rest.
rule
this one stays c, python having nothing to say about it.

RGB_MATRIX_EFFECT(LAYER_COLORS)

#ifdef RGB_MATRIX_CUSTOM_EFFECT_IMPLS

extern const uint8_t PROGMEM ledmap[][RGB_MATRIX_LED_COUNT][3];

static bool LAYER_COLORS(effect_params_t *params) {
    RGB_MATRIX_USE_LIMITS(led_min, led_max);
    uint8_t layer = get_highest_layer(layer_state);

    for (uint8_t i = led_min; i < led_max; i++) {
        RGB_MATRIX_TEST_LED_FLAGS();
        uint8_t v = rgb_matrix_config.hsv.v;
        rgb_matrix_set_color(i,
            scale8(pgm_read_byte(&ledmap[layer][i][0]), v),
            scale8(pgm_read_byte(&ledmap[layer][i][1]), v),
            scale8(pgm_read_byte(&ledmap[layer][i][2]), v));
    }
    return rgb_matrix_check_finished_leds(led_max);
}

#endif
how
once flashed, layer colors of L2 selects the mode, which the keyboard remembers.
scope
that same key brings the mode back after a stray rgb anim.

switching them from the computer

why
I want to be able to shut them off.
how
--off paints them all black, and --on hands them back to the mode the keyboard remembers8.
  • scope: off lasts until the keyboard is unplugged9.

set -eu

source "_clk.sh"

clk_usage () {
    cat<<EOF
$0

Light the voyager leds up, or put them out
--
F:--on/--off:Give the leds back to the firmware, or put them out:{"default": true}
EOF
}

clk_help_handler "$@"

if clk_true on
then
    voyager-disco reset
else
    voyager-disco set-color 000000
fi
how
voyager-disco speaks the oryx raw hid to the keyboard, with nothing running in between10.
how
nix builds it from github.
  • scope: it wants oryx protocol 4, and zsa/oryx speaks 5, which only adds two automouse orders11, 12.
    • example: unpatched, it answers « Unsupported Oryx protocol version 0x05 (expected 0x04) ».
  • therefore: the version it expects is patched to 5.

{ pkgs, ... }:
{
  environment.systemPackages = [
    (pkgs.rustPlatform.buildRustPackage {
      pname = "voyager-disco";
      version = "1753694";
      src = pkgs.fetchFromGitHub {
        owner = "monorkin";
        repo = "voyager-disco";
        rev = "1753694a0d9d43b4bfb56ad5ae362d47beb8538b";
        hash = "sha256-odtKWWb6cfkCCZma/Vpt+o2yuVLFgH2rto3TitnDWqM=";
      };
      cargoHash = "sha256-+AOUfeOvfNzQjqdJjxwHvLgSn4deddaHnEPB5WGCWrY=";
      postPatch = ''
        substituteInPlace src/device.rs \
          --replace-fail "EXPECTED_PROTOCOL_VERSION: u8 = 0x04" "EXPECTED_PROTOCOL_VERSION: u8 = 0x05"
      '';
      nativeBuildInputs = [ pkgs.pkg-config ];
      buildInputs = [ pkgs.udev ];
    })
  ];
}

and the bin has to reach the keyboard

how
the boot key of L2 enters the bootloader, and the keyboard stops typing.
scope
plugging it back in while esc is held enters it too, and erases what it remembers.
what
lsusb then shows 3297:0791 in place of 3297:1977, saying it is in bootloader mode.
lsusb | grep --ignore-case zsa
Bus 001 Device 020: ID 3297:0791 ZSA Technology Labs Voyager Bootloader
  • what: talking to the bootloader needs udev rules.
{ ... }:
{
  hardware.keyboard.zsa.enable = true;
}

footnotes


  1. « Il y a deux façons d’utiliser Ergo‑L sur un clavier programmable : soit on laisse le clavier en Qwerty dans QMK/ZMK/Vial, et c’est le pilote côté ordi qui s’occupe de la disposition […] ; soit on cherche à émuler Ergo‑L côté QMK/ZMK/Vial, pour avoir la disposition sur un ordi qui reste configuré en Azerty. La première méthode est vivement recommandée. » — Installation, Ergo‑L, note aux utilisateurices de claviers programmables. ↩︎

  2. « L’émulation est beaucoup plus compliquée, et n’est pertinente qu’en dernier ressort : ordinateur totalement verrouillé (même pour le pilote nomade), administration de nombreuses machines, ou autre besoin spécifique. » — Installation, Ergo‑L. ↩︎

  3. « On peut alors configurer son clavier très facilement, il faut juste avoir AltGr sur une touche de pouce pour accéder aux symboles. » — Installation, Ergo‑L. ↩︎

  4. “The Voyager’s firmware includes a QMK module called “Oryx” […] that accepts commands over USB Raw HID.” — README.md, voyager-disco. ↩︎

  5. “Choose whether to build your layout against ZSA’s QMK fork or mainline QMK. Using ZSA’s fork means your firmware will compile easily, but it is slower to update and does not pull all the new features of mainline QMK. […] If you would like to compile against ZSA’s fork, make sure to manually set the path when going through the setup: https://github.com/zsa/qmk_firmware/ (ZSA’s QMK fork). Otherwise, the setup process will default to mainline QMK (qmk/qmk_firmware).” — README.md, Oryx source export. ↩︎

  6. “RGB keycodes have been reworked […] for rgblight we have UG_xxx, and RM_xxx for rgb_matrix. Longer-term, RGB_xxx style keycodes will be removed, but for now they act as if they’re UG_xxx.” — QMK Breaking Changes, 2024 November 24. ↩︎

  7. #if defined(RGB_MATRIX_ENABLE) && !defined(PROTOCOL_LUFA) && defined(RGB_MATRIX_CUSTOM_KB) guards set_webhid_effect, which switches to RGB_MATRIX_CUSTOM_oryx_webhid_effect — oryx.c, zsa/qmk_modules. ↩︎

  8. “Restores the keyboard’s own lighting (whatever you configured in Oryx).” — README.md, voyager-disco. ↩︎

  9. “Colors are stored in keyboard RAM. They survive the tool exiting but not a power cycle.” — README.md, voyager-disco. ↩︎

  10. “A small stand-alone CLI tool to control the RGB LEDs on ZSA keyboards (Voyager, Moonlander, ErgoDox EZ)” […] “Keymapp must not be running” — README.md, voyager-disco. ↩︎

  11. “Oryx protocol (v0x04)” […] “0x09 | SET_RGB_LED_ALL | Set all LEDs to one color”, “0x0A | STATUS_LED_CONTROL | Status LED control mode”, “0xFE | GET_PROTOCOL_VERSION | Get protocol version” — README.md, voyager-disco. ↩︎

  12. #define ORYX_PROTOCOL_VERSION 0x05, its command list ending on ORYX_SET_AUTOMOUSE, ORYX_GET_AUTOMOUSE — oryx.h, zsa/qmk_modules. ↩︎