Salsa has no branches

Two agents on two worktrees of one repository, sharing one analyzer database, answer each other's questions. A benchmark that forks four worktrees and asserts what each should see, and why we stopped trying to share anything writable.

Git gives every worktree its own tree and its own branch, and that is exactly why we hand agents worktrees: a worker can change a signature without the resident session seeing half of it. The analyzer underneath has no such notion. rust-analyzer’s database, Salsa, holds one state of one workspace, keyed by file path. Two worktrees of the same repository have the same relative paths, so if both are served from one database, the file src/metrics.rs has exactly one content: whichever session last opened it.

We found this the way you find these things, with a benchmark that asserts what should be true. It clones a real repository, makes four worktrees, and gives each a different divergence: master untouched, A with an extra parameter on a real function, B with a touched manifest, C with an untracked file that declares a new symbol. Then twelve workers hover the same symbol concurrently, through the real wire protocol, with the same pre-flight sync a live editor session does, and the run fails unless master and B show the base signature, A shows its new parameter, and C resolves its untracked symbol.

Four worktrees against one shared Salsa database, where B answers with A's signature, versus four worktrees each with their own database — One database keyed by file path cannot hold four trees. The bleed is not a race; it is the design working as specified.

On a 532-file Rust workspace, one shared database across the four worktrees:

	one shared database	one per worktree
p50	242 ms	253 ms
p95	3135 ms	1674 ms
errors in 60 queries	19	6
correctness	FAIL	4/4 pass

The failures are the interesting column. A returned the base signature twice, B returned A’s new parameter once, and C’s untracked symbol came back null fifteen times out of fifteen.

B returning A’s parameter is the whole argument in one line. Nothing crashed, nothing logged an error, and the agent in worktree B got a signature that does not exist in its tree, with the same confidence as a correct answer. The untracked file in C failed for the same root cause: the module that declares it is owned by another session’s buffer.

Two fixes followed, in order. First, a session’s unsaved buffer became private to that session: overlays per session on top of the base inputs, so an open document in one session no longer rewrites the file for everyone. Second, and this is the part we will not trade away, every git worktree gets its own server workspace and its own database, named <origin>--wt-<hash of its path>. The main checkout keeps its own. The shared mode still exists, in the benchmark, as the thing we compare against.

What that looks like from an agent, asking the same question twice in the same second, once from the main checkout and once from a worktree where that function took an extra parameter:

# in the main checkout
$ code_hover { "symbol": "default_root" }
pub fn default_root(storage_root: &Path) -> PathBuf

# in a worktree of the same repository, same second
$ code_hover { "symbol": "default_root" }
pub fn default_root(storage_root: &Path, tmpfs: bool) -> PathBuf

Two answers, both correct, because there are two databases. The worktree’s change is not committed and not pushed; it exists as a dirty file on the laptop and as a file in that worktree’s own mirror on the node.

While reading those traces we also found the null hovers. About one query in ten, with three concurrent sessions on one file, came back empty. Salsa cancels in-flight queries when a write bumps the revision, and our hover path turned the cancellation into Ok(None), which the client printed as “no result”. rust-analyzer’s own server answers ContentModified and the editor retries. We made queries run under the engine lock together with the view activation, so a concurrent edit can no longer cancel a query into a plausible-looking nothing.

What isolation costs, and the trap in paying it

Isolation is not free. A new worktree is a new database, and the first load of a Rust workspace runs its build scripts and expands its proc macros before the first hover answers: about 45 s for our own gateway repository, once, after which the engine stays resident until it has been idle for thirty minutes and reloads in about two seconds.

The obvious way to cut that is the tempting one. Every worktree copy on the node has its own target/, and they are nearly identical, so we pointed them all at the origin repository’s target/. It worked beautifully: cold load of a new worktree went from 45 s to 2.1 s, four at once in 20 s. We shipped it in the morning and reverted it the same day, because of a question with one right answer: what happens when ten agents in ten worktrees all run cargo test? Cargo takes an exclusive lock on the target directory. Nine of them wait. The thing we bought with a shared cache was a queue, and a queue is worse than a cold start, because a cold start is paid once per worktree and the queue is paid on every build for as long as the fleet runs.

The honest alternative for that 45 s is a compiler cache, and we tried that too: sccache installed as rustc-wrapper on the nodes. It never hits across worktrees, because the command line it hashes contains the target directory path (-L dependency=…), and every worktree has its own. So we pay the 45 s.

The rule

Things that run at the same time do not share writable state. A database is writable state; two worktrees cannot share one. A target/ directory is writable state behind a lock; ten workers cannot share one. What they can share is anything read-only: the git objects, the cargo registry, the origin copy of the files that a new worktree is seeded from, the pages of the files themselves.

That is also the shape of the newest thing we built on top of this, where each candidate fix runs against an overlay of the same workspace directory: read through the shared tree, write into your own layer, disappear. The isolation argument and the speed argument stopped being opposites the moment we stopped sharing anything that could be written.

What it does not do yet

The 45 s first load is per worktree and shared with nothing. A worktree evicted after thirty idle minutes reloads in about two seconds; a new one pays the full load.
Eviction is time-based only (--idle-evict-secs, default 1800), with no memory pressure signal: twenty extra workspaces on one node took its resident memory from 774 MB to 1042 MB and nothing pushed back.
Worktree directories on the node are pruned after seven days unused (--prune-worktree-days), not when the worktree disappears from your laptop.
The shared-database mode is still in the benchmark and still fails correctness. It exists to be measured against, not to be switched on.