The tree the node was building

Everything in this series rests on one assumption: the copy of the checkout on the build node is the checkout. The analyzer answers questions about that copy, cargo test runs in it, and a refactoring is type-checked against it. When the copy drifts, every answer is about a different program, and nothing says so.

Three bug reports arrived within one minute on September 23, 2026, describing problems with a Rust checkout, a SwiftPM package and a mixed Rust/Swift repository. All three were filed under the alex09x account; the issues do not establish whether an agent wrote them or whether they came from the same checkout. Each one describes a way the copy had drifted.

This account follows the three fixes at the revisions merged that day: PR #129 (revision 1477baa3) for issue #124, PR #130 (revision 6fb3b54f) for issue #123, and PR #131 (revision ff711ff1) for issue #125. The transcripts and the one timing below are the ones reported in those issues and PRs and in the follow-up comments on PR #129 and issue #124; none of them was repeated for this post. All three parts changed afterwards: PR #730 of September 29, 2026 widened the directories counted as code and raised the filter version again, PR #1052 of October 8 split the sync filter into modules and reworked the reconciliation of tracked files, and PR #1067 of October 11 reworked the engine-aware routing. What they do is not covered here, and the present tense below describes the implementation at the September revisions, not the current release.

Files cut by extension

A test embedded a fixture with include_bytes!("fixtures/capture.bin"). Locally it built; remotely:

$ prod-code exec -- cargo test -q
error: couldn't read `tests/fixtures/capture.bin`: No such file or directory (os error 2)
error: couldn't read `tests/fixtures/case/snapshot.txt`: No such file or directory (os error 2)

The sync sent only files whose extension was on an allowlist: source files, manifests, lock files, configuration. The list was written early on, to keep repositories full of datasets and logs from pouring gigabytes onto the node, and at that time it was right. A directory walk cannot tell a fixture from a dataset. A .bin is a .bin.

But the sync does not walk directories when it does not have to. In a git checkout every path it considers comes from git: ls-files on first contact, diff since the last synced commit, and status for what is uncommitted. Git already makes the distinction the allowlist was guessing at. A dataset that should stay local is in .gitignore, and an ignored untracked file is never listed. So a file git lists is now a candidate whatever its extension: fixtures, snapshots, recordings, .github/. What stays out is what was kept out by directory, not by extension: data trees, build output, vendored dependencies, .git; and two size limits still apply, 5 MiB per file and 256 KiB for a .json, the second there to keep datasets with that extension out. After the fix:

$ prod-code exec -- cargo test -q
test result: ok. 1 passed; 0 failed; ...
$ prod-code exec -- find tests .github -type f
.github/workflows/ci.yml
tests/fixtures/capture.bin
tests/fixtures/case/snapshot.txt
tests/fixtures_test.rs

The report also named the worse failure: a test that passes remotely against a stale fixture left over from an earlier copy. That one produces no error at all.

Directories that never went away

A renamed directory stayed on the node next to its new name, and so did a deleted one. The report came from a SwiftPM package, which treats an unexpected directory under Sources/ as an error. The files inside were being deleted correctly. The directories were not:

$ git mv extra/OldDir extra/NewDir && git rm -r extra/Unused && git commit -m move
$ prod-code exec -- find extra
extra
extra/Unused
extra/Unused/deep
extra/OldDir
extra/NewDir
extra/NewDir/a.rs

Both deletion paths on the gateway, the incremental sync and the manifest reconciliation on first contact, called remove_file and nothing else. Now a deletion also removes the directories it leaves empty, up to the workspace root. With that deployed to the Linux nodes, one more rename on the same checkout, as reported in the comment on PR #129:

$ git mv extra/NewDir extra/Third && git commit -m third
$ prod-code exec -- find extra
extra
extra/Unused
extra/Unused/deep
extra/OldDir
extra/Third
extra/Third/a.rs

NewDir went with its last file. Unused and OldDir are still there. They were emptied before the fix, and the incremental sync only sends what changes, so it will never touch them again. For those, the reconciliation now also removes every empty directory it finds, except in the node’s own caches such as target/ and .build/. The allowlist change above raises the sync’s filter version, and a client with a watermark from an older version treats its next sync as a first contact. So the reconciliation ran, and the leftovers went, as the last comment on issue #124 reports:

$ prod-code exec -- find extra
extra
extra/Third
extra/Third/a.rs

The Swift half that never reached the Mac

The repository in the third report has a Cargo.toml at the root and a SwiftPM package under swift/. The cluster has Linux nodes and one macOS node that serves only Swift:

$ prod-code mcp  ← code_test {"path": "swift"}
remote test failed to start: failed to start swift: No such file or directory (os error 2)
$ prod-code -r <macOS node>:9400 exec -- uname -s
Linux

The MCP server is placed once, when it starts, by the engine of the repository root, which is Rust. Every call after that went to the same Linux node, whatever path it named. And naming the Mac explicitly did not help: the address was treated as a seed, the seed was expanded into the whole cluster by gossip, and the remembered placement won.

There were three fixes:

• A call that names a path is routed by that path. If the path is in a nested project of another language and the current node does not serve that language, the call goes to a node that does.
• That placement has its own key, <workspace>#swift. Otherwise the Swift query would move the checkout’s Rust placement to the Mac, and the next Rust query would move it back.
• An address given with --remote on the command line is used as given.

In that order, from the repository root:

$ prod-code exec -- uname -s
Linux
$ prod-code -r <macOS node>:9400 exec -- uname -s
Darwin
$ prod-code exec -- uname -s
Linux
$ prod-code mcp  ← code_test {"path": "swift"}
$ swift test
swift test: OK in 10.1s; 1 passed, 0 failed

The timing is the one PR #131 reports for that run, not a benchmark. The separate key is checked by a test rather than by this run: with a fake Rust node and a fake Swift node, the Swift path is remembered as mixed#swift, and mixed is left alone.

How they are pinned

The allowlist fix is tested on a real git repository with half of its files committed and half untracked. The scan must give exactly what git lists minus a data tree, with no .gitignored log, and a binary fixture must arrive byte for byte. With the old rule restored, PR #130 reports both new tests failing and the other nineteen sync tests still passing, which says something about what the old tests were checking. The directory fix is tested on a manifest that drops two directories and must leave the caches alone. The routing fix is tested against two fake nodes, one per engine. The end-to-end proof is the run above, on a real macOS node, because a fake node cannot say what uname prints.

What it does not do

• Non-git checkouts keep the old allowlist: without git there is nothing to ask.
• Command-line runs route by the directory they are run from, not by a path argument. prod-code test from the root stays on Linux. Run from swift/, the same command already went to the Mac before this change; what changed is that it now keeps its own placement key.
• Two nodes. In the run above, the Rust calls went to a Linux node and the Swift call to the Mac. Whether each node then keeps its own synced copy of the checkout is not something these PRs state, and it is not claimed here.

Alexander Panasenko (2026-10-27). The tree the node was building. https://prod.codes/blog/the-tree-the-node-was-building/