Offloading the language server to a cluster

A Rust repository can contain a Swift desktop client. Place the repository on a Linux build node, then ask about a directory under swift/Tests/: the root project’s placement does not tell us where that request belongs. The directory needs its own engine decision, and a macOS target may need an SDK the default node does not have.

This is one of the boundaries addressed in prod-code v0.3.29. Moving analysis away from the laptop leaves us responsible for choosing a compatible host, preserving the workspace a query sees, and accounting for the work a proposed edit causes. The client still tracks and synchronizes files and handles requests and responses.

The implementation references below pin release commit e323a0a318191002fd57642b55d80a460e0cb3d5. They describe routing and resource policy; they are not a new benchmark of the release.

Place the request by its target

In the routing implementation, the target path matters alongside the repository root. Engine detection considers nested projects, manifests, file extensions and directory contents, then falls back to the root’s expected engine.

This excerpt from route_in shows the placement decision after its arguments have been supplied:

let engine = nested_engine(root, path).or_else(|| engine_for_path(root, path));
let Some(engine) = engine else {
    return Ok(default);
};
if default_serves(default, engine).await {
    return Ok(default);
}
let key = format!("{workspace}#{engine}");
pick_node_with(nodes, &key, Some(engine), None, placement_file)
    .await
    .with_context(|| format!("`{path}` is in a {engine} project"))

An already compatible default node stays the default. Otherwise, selection asks for a node that serves the detected engine and remembers that placement under an engine-specific key. A secondary Swift session need not overwrite the Rust workspace’s placement.

Compatibility does not mean the engine is already warm or the project has finished loading. Nor does Swift itself imply macOS: SourceKit-LSP supports Linux. Xcode and Apple SDK requirements are a separate constraint of the target being built. For our macOS client, the host also needs the target SDK and build tools; an engine label alone does not establish those requirements.

Share the Rust base, keep worktree views separate

A separate working directory need not require a wholly independent Rust engine. The gateway’s workspace loader recognizes the mirrored worktree naming convention and takes a Rust-specific attachment path. This is not a claim that every language server supports shared worktree analysis.

That path loads the base workspace, attaches the worktree to its mirrored Rust engines, and constructs a workspace holding a reference to the base. Here is the constructor call; the surrounding loading, attachment and error handling are omitted:

let ws = Arc::new(SharedWorkspace::with_base(
    workspace_root.to_path_buf(),
    "rust".to_string(),
    Some(Arc::clone(base_engine_arc)),
    None,
    None,
    None,
    Some(Arc::clone(&base_ws)),
));

The Arc shares an engine within the gateway process. It does not demonstrate zero memory overhead, a fixed query latency, or how memory scales across arbitrary projects. The useful source-level distinction is that a worktree gets its own workspace identity while retaining the base engine and workspace references.

Proposed edits have their own cost

A diagnostic request on proposed text can cause analysis work even when it never applies that text to the repository working tree. Opening an overlay, asking for diagnostics and closing it can make the analyzer reconsider dependent code. A navigation query using the same engine may then pay for work caused by that validation.

The Rust validation view uses a second engine so proposed documents can be handled separately from ordinary queries. That engine consumes additional memory. If loading or admission fails, the implementation falls back to the main engine; the separation is conditional.

For an agent, analyzer diagnostics are feedback about a candidate. They do not establish that the candidate compiles, that all references were found, or that its behavior is preserved. When compiler verification is requested, its result is a separate check. Command-based shadow execution also has its own results and storage costs. A clean analyzer response should not be relabeled a successful compiler run, and no working-tree edits should not be relabeled zero disk activity.

This also changes how we describe performance. The release README contains recorded workload timings, but that table does not identify the measured revisions, repetitions or every timer boundary. It cannot establish the latency of this release’s overlay path or a comparable laptop speedup.

Bound the wait and the retained state

An execution budget must reach the layer that waits for the tool. The release’s timeout resolver defaults to 150 seconds. A positive timeout_secs gets a five-second margin, capped at 24 hours; a positive timeout_millis takes precedence and is capped without that margin. Thus a seconds-only request for 2,400 seconds gives this wrapper 2,405 seconds. The command and transport can still fail for other reasons.

Retention is another boundary. The gateway defaults allow engine eviction after 1,800 seconds without a session and disk-worktree pruning after 3,600 seconds of disuse. These are configurable thresholds, not promises of deletion at an exact instant. The pruner skips loaded workspaces.

The rule for an offloaded request is to carry its engine, workspace identity, validation mode and execution budget across the boundary. A remote machine can supply compute; it cannot repair a request that was sent to the wrong environment or turn incomplete diagnostics into a proof.

Alexander Panasenko (2026-10-11). Offloading the language server to a cluster. https://prod.codes/blog/three-hundred-and-fifty-five-cores/