Moby Under the Microscope: What 67 AST Tools Found Inside Docker's Engine
Selected results from the 67-tool prod-code AST suite on moby/moby, as reported: 387K lines of Go, 224 Protobuf varint clone occurrences, 99.9% slicing reduction on ContainerStart, 858 structural error returns in 1.9s, and a reported rejection during boolean inversion.
On this page · 8 sections
- 1. Protobuf Varint Decoding Clone Clusters: 224 Occurrences
- 2. Meaning-Based Semantic Search: Container Start & Attach in 212 ms
- 3. Program Slicing: Reducing 387K Lines to 250 Lines (99.9% Reduction)
- 4. Boolean Inversion: Inverting IsRunning Across 19 Files with a Reported Rejection
- 5. Structural Pattern Matching: 858 Error Propagations in 1.9s
- 6. Remote In-Memory Validation (2.00s)
- 7. Compatibility Matrix for the Named Tools: Moby (Docker)
- Summary: What This Means for Systems Engineering
First published on 2026-09-30. The measurements reported in this article are historical and have not been re-run for this revision. The article reports using moby/moby at commit 367ff5729a24.
moby/moby is the foundational open-source component library behind Docker. Spanning the Go API client (client/), daemon orchestration, storage graph drivers (overlay2), libnetwork container networking, and containerd runtime integration, Moby defines modern containerization. The docker command-line application itself is maintained separately, in docker/cli.
Building and maintaining a system that interfaces with Linux kernel cgroups, namespaces, OCI runtimes, and distributed image registries requires high reliability. A faulty state transition in container lifecycle management can leave orphan processes, locked mount points, or uncollected resources on host nodes.
The article reports running the 67-tool prod-code AST suite against moby/moby (commit 367ff5729a24): 387,617 lines of Go across 2,269 source files and 20,508 declarations. The sections below show selected operations from that suite; no per-tool record of the rest was published. The evaluation ran on LAN cluster nodes (node-a node-a:9400 and node-b node-b:9400); the article reported 0% local laptop CPU, a figure for which no local CPU measurement was published.
Here is what deep code intelligence discovered.
1. Protobuf Varint Decoding Clone Clusters: 224 Occurrences
Moby integrates BuildKit and gRPC transport protocols for container build sessions, secret mounting, and exporter streams.
Running prod-code duplicates -r node-a:9400 --min-lines 14 revealed a massive clone cluster: 224 occurrences of an identical 14-line varint decoding loop:
// Repeated 224 times across vendor/github.com/moby/buildkit and fsutil
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return protohelpers.ErrIntOverflow
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
wire |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
Generated code generators (such as vtprotobuf) inline varint decoding into the generated message structs; the article did not establish why each generator does so. As reported, prod-code duplicates grouped these generated clone clusters apart from handwritten daemon logic; no timing was published for that run.
2. Meaning-Based Semantic Search: Container Start & Attach in 212 ms
In a repository with 2,269 Go files, finding where standard I/O streams are attached to running containers using simple text search returns thousands of log messages and test assertions.
We queried prod-code search:
$ prod-code -r node-a:9400 search "container start and attach streams"
In 212 ms, across 20,508 declarations indexed on the cluster node, the engine ranked the exact daemon streaming machinery:
(*Daemon).containerAttach(daemon/attach.go:126) - attaches stdio streams to container(*Daemon).ContainerAttachRaw(daemon/attach.go:99) - raw stream forwardingContainerAttachConfig(daemon/server/backend/backend.go:33) - stream configuration descriptor(*Client).ContainerStart(client/container_start.go:20) - client start request(*Config).AttachStreams(daemon/internal/stream/attach.go:45) - attaches container streams to AttachConfig(*Daemon).ContainerStart(daemon/start.go:49) - starts container process
Graph centrality and vector embeddings surfaced the core daemon streaming logic immediately.
3. Program Slicing: Reducing 387K Lines to 250 Lines (99.9% Reduction)
Starting a container involves validating checkpoints, checking experimental flags, verifying host cgroup settings, acquiring container locks, initializing network sandboxes, creating OCI specs, and launching containerd tasks.
We seeded a slice on (*Daemon).ContainerStart:
$ prod-code -r node-a:9400 slice daemon/start.go --line 49 --depth 2
In 420 ms, prod-code slice extracted the exact dependency closure:
- Seed:
(*Daemon).ContainerStart(lines 49-76) - Depth 1:
(*Daemon).containerStart, OpenTelemetry tracer spans,errdefs.InvalidParameter,errors.New - Depth 2: containerd task lifecycle, sandbox cleanup hooks, and container state transitions
The slice reduced 387,617 lines of code down to 250 relevant lines, stripping away all unrelated image pruning, Swarm clustering, and volume management routines.
4. Boolean Inversion: Inverting IsRunning Across 19 Files with a Reported Rejection
Containers transition through states (Running, Paused, Restarting, Dead, Created). The daemon queries State.IsRunning() across nearly every subsystem:
// daemon/container/state.go:226
func (s *State) IsRunning() bool {
s.Lock()
defer s.Unlock()
return s.Running
}
We tested prod-code invert-boolean:
$ prod-code -r node-a:9400 invert-boolean \
--to IsStopped \
--line 226 --character 17 \
daemon/container/state.go
The tool computed:
- Body negation:
return !(s.Running) - Call site inversion: 15 calls gained a
!, 15 calls lost their existing!across 64 lines in 19 files (daemon/daemon.go,daemon/delete.go,daemon/commit.go,daemon/changes.go,daemon/exec.go, etc.). - Reported rejection: the article reports that the patch was not written and names
daemon/internal/plugin/executor/containerd/containerd.goanddaemon/pkg/plugin/manager.go. Those files concern the plugin executor’sIsRunning, which takes a plugin ID and is a different method from the zero-argumentState.IsRunning()targeted here; inverting theStatemethod alone does not touch that interface. The diff and diagnostics behind the rejection were not published, so whether the proposal had also rewritten the unrelated method is not shown.
Correction, 2026-10-27. As published on 2026-09-30, this item said the analyzer detected that changing
IsRunning()broke plugin executor interface compliance. In the pinnedstate.go,State.IsRunning()takes no argument, while the plugin executor’s method takes an ID;Statedoes not implement that interface. What was rejected, and why, is not established by the article. The interface-compliance explanation is withdrawn here, in the section heading (published as “with Interface Guard”), in the results table (published as “interface safety rejection”) and in the summary (published as “Interface-safe refactoring: Prevents breaking subtle plugin executor contracts across package boundaries”); the report that no patch was written stands as published.
5. Structural Pattern Matching: 858 Error Propagations in 1.9s
In systems software communicating with the Linux kernel and containerd over gRPC sockets, proper error forwarding is mandatory.
We ran structural AST search:
$ prod-code -r node-a:9400 structural-search 'if err != nil { return nil, err }'
In 1,928 ms, across 2,268 source files, the engine mapped 858 exact structural occurrences across 285 distinct files:
- Systematically captured across
client/client.go,client/container_logs.go,client/image_pull.go, andclient/hijack.go. - As reported, matched multi-line returns and nested error wraps; the article lists files rather than the matches, so it does not show the match set’s precision against non-nil variable returns.
6. Remote In-Memory Validation (2.00s)
When modifying daemon dispatch logic, waiting for local go test runs or docker builds slows down feedback loops.
Testing prod-code validate with unclosed syntax:
$ cat << 'EOF' | prod-code -r node-a:9400 validate daemon/start.go
package daemon
func invalidMobySyntax(
EOF
Returned:
daemon/start.go: 1 error(s), 0 warning(s)
error: expected ')', found 'EOF' (daemon/start.go:4:1)
[prod-code] analysed in 2.00s
As reported, the error came back in 2.00 seconds. It is a parse error in the proposed contents of daemon/start.go, the one buffer the output shows was checked; it is not a validation of the 387K-line repository.
7. Compatibility Matrix for the Named Tools: Moby (Docker)
The table, as published, names 21 of the 67 tools; its results are the article’s reported outcomes, not per-tool run records.
| Category | Tools Tested | Result | Latency / Metric |
|---|---|---|---|
| Diagnostics & Outline | outline, symbols, type_at, hover |
✅ Passed | 20,508 declarations indexed |
| Navigation | definition, references, callers, callees |
✅ Passed | containerd shim & daemon traversal |
| Search & Discovery | search, structural_search, duplicates |
✅ Passed | 224 varint clones found; 858 error guards in 1.9s |
| Program Slicing | slice |
✅ Passed | 99.9% reduction on ContainerStart (387K → 250 lines) |
| Architecture | dependencies |
✅ Passed | Zero cyclic dependencies across daemon modules |
| Refactoring (Functions) | extract_function, change_signature, rename |
✅ Passed | Container state refactoring |
| Refactoring (Types/Params) | parameter_object, invert_boolean |
✅ Passed | 19-file boolean inversion; reported rejection, cause not published |
| Validation & Safety | validate_edit, validate_edits, shadow_run |
✅ Passed | 2.00s remote in-memory syntax validation |
Correction, 2026-10-27. As published on 2026-09-30, this section was headed “Full 67-Tool Compatibility Matrix”, and the opening said all 67 tools were run. The table names 21 tools, and the article shows output for a subset of those; it does not establish an outcome for every tool. The Architecture row reports zero cyclic dependencies across daemon modules, but no dependency output was published, so the article does not establish acyclicity. The full-suite and acyclicity claims are narrowed to what the article reports; the table itself is kept as published.
Summary: What This Means for Systems Engineering
Moby’s scale-387K lines of Go orchestrating low-level Linux primitives-demands precision:
- Offloaded analysis: builds and language server indexes ran on the LAN cluster nodes rather than the laptop; the article reported 0% local CPU, without a published local measurement.
- Millisecond-scale slicing: the
prod-code slicerun reported above took 420 ms, as published on 2026-09-30; that one reported call was a matter of hundreds of milliseconds, not microseconds, and no other slice timings are on record. - Reported refactoring rejection: the article reports that the 19-file inversion was not written; the cause was not published, so no claim is made that the tool protected plugin executor contracts.
Editorial note, 2026-10-11: the host names, addresses and home-directory paths of our build nodes in this post were replaced with the placeholders node-a, node-b and <user>. No other part of the quoted output was changed.
Cite this article
Alexander Panasenko (2026-09-30). Moby Under the Microscope: What 67 AST Tools Found Inside Docker's Engine. https://prod.codes/blog/moby-under-the-microscope-67-ast-tools/