notes · · 6 min · updated 2026-10-27

Moby Under the Microscope: What 67 AST Tools Found Inside Docker's Engine

Selected results from the 67-tool prod-code AST suite on moby/moby, as reported: 387K lines of Go, 224 Protobuf varint clone occurrences, 99.9% slicing reduction on ContainerStart, 858 structural error returns in 1.9s, and a reported rejection during boolean inversion.

On this page · 8 sections
  1. 1. Protobuf Varint Decoding Clone Clusters: 224 Occurrences
  2. 2. Meaning-Based Semantic Search: Container Start & Attach in 212 ms
  3. 3. Program Slicing: Reducing 387K Lines to 250 Lines (99.9% Reduction)
  4. 4. Boolean Inversion: Inverting IsRunning Across 19 Files with a Reported Rejection
  5. 5. Structural Pattern Matching: 858 Error Propagations in 1.9s
  6. 6. Remote In-Memory Validation (2.00s)
  7. 7. Compatibility Matrix for the Named Tools: Moby (Docker)
  8. Summary: What This Means for Systems Engineering

First published on 2026-09-30. The measurements reported in this article are historical and have not been re-run for this revision. The article reports using moby/moby at commit 367ff5729a24.

moby/moby is the foundational open-source component library behind Docker. Spanning the Go API client (client/), daemon orchestration, storage graph drivers (overlay2), libnetwork container networking, and containerd runtime integration, Moby defines modern containerization. The docker command-line application itself is maintained separately, in docker/cli.

Building and maintaining a system that interfaces with Linux kernel cgroups, namespaces, OCI runtimes, and distributed image registries requires high reliability. A faulty state transition in container lifecycle management can leave orphan processes, locked mount points, or uncollected resources on host nodes.

The article reports running the 67-tool prod-code AST suite against moby/moby (commit 367ff5729a24): 387,617 lines of Go across 2,269 source files and 20,508 declarations. The sections below show selected operations from that suite; no per-tool record of the rest was published. The evaluation ran on LAN cluster nodes (node-a node-a:9400 and node-b node-b:9400); the article reported 0% local laptop CPU, a figure for which no local CPU measurement was published.

Here is what deep code intelligence discovered.


1. Protobuf Varint Decoding Clone Clusters: 224 Occurrences

Moby integrates BuildKit and gRPC transport protocols for container build sessions, secret mounting, and exporter streams.

Running prod-code duplicates -r node-a:9400 --min-lines 14 revealed a massive clone cluster: 224 occurrences of an identical 14-line varint decoding loop:

// Repeated 224 times across vendor/github.com/moby/buildkit and fsutil
for shift := uint(0); ; shift += 7 {
    if shift >= 64 {
        return protohelpers.ErrIntOverflow
    }
    if iNdEx >= l {
        return io.ErrUnexpectedEOF
    }
    b := dAtA[iNdEx]
    iNdEx++
    wire |= uint64(b&0x7F) << shift
    if b < 0x80 {
        break
    }
}

Generated code generators (such as vtprotobuf) inline varint decoding into the generated message structs; the article did not establish why each generator does so. As reported, prod-code duplicates grouped these generated clone clusters apart from handwritten daemon logic; no timing was published for that run.


2. Meaning-Based Semantic Search: Container Start & Attach in 212 ms

In a repository with 2,269 Go files, finding where standard I/O streams are attached to running containers using simple text search returns thousands of log messages and test assertions.

We queried prod-code search:

$ prod-code -r node-a:9400 search "container start and attach streams"

In 212 ms, across 20,508 declarations indexed on the cluster node, the engine ranked the exact daemon streaming machinery:

  1. (*Daemon).containerAttach (daemon/attach.go:126) - attaches stdio streams to container
  2. (*Daemon).ContainerAttachRaw (daemon/attach.go:99) - raw stream forwarding
  3. ContainerAttachConfig (daemon/server/backend/backend.go:33) - stream configuration descriptor
  4. (*Client).ContainerStart (client/container_start.go:20) - client start request
  5. (*Config).AttachStreams (daemon/internal/stream/attach.go:45) - attaches container streams to AttachConfig
  6. (*Daemon).ContainerStart (daemon/start.go:49) - starts container process

Graph centrality and vector embeddings surfaced the core daemon streaming logic immediately.


3. Program Slicing: Reducing 387K Lines to 250 Lines (99.9% Reduction)

Starting a container involves validating checkpoints, checking experimental flags, verifying host cgroup settings, acquiring container locks, initializing network sandboxes, creating OCI specs, and launching containerd tasks.

We seeded a slice on (*Daemon).ContainerStart:

$ prod-code -r node-a:9400 slice daemon/start.go --line 49 --depth 2

In 420 ms, prod-code slice extracted the exact dependency closure:

  • Seed: (*Daemon).ContainerStart (lines 49-76)
  • Depth 1: (*Daemon).containerStart, OpenTelemetry tracer spans, errdefs.InvalidParameter, errors.New
  • Depth 2: containerd task lifecycle, sandbox cleanup hooks, and container state transitions

The slice reduced 387,617 lines of code down to 250 relevant lines, stripping away all unrelated image pruning, Swarm clustering, and volume management routines.


4. Boolean Inversion: Inverting IsRunning Across 19 Files with a Reported Rejection

Containers transition through states (Running, Paused, Restarting, Dead, Created). The daemon queries State.IsRunning() across nearly every subsystem:

// daemon/container/state.go:226
func (s *State) IsRunning() bool {
    s.Lock()
    defer s.Unlock()
    return s.Running
}

We tested prod-code invert-boolean:

$ prod-code -r node-a:9400 invert-boolean \
    --to IsStopped \
    --line 226 --character 17 \
    daemon/container/state.go

The tool computed:

  • Body negation: return !(s.Running)
  • Call site inversion: 15 calls gained a !, 15 calls lost their existing ! across 64 lines in 19 files (daemon/daemon.go, daemon/delete.go, daemon/commit.go, daemon/changes.go, daemon/exec.go, etc.).
  • Reported rejection: the article reports that the patch was not written and names daemon/internal/plugin/executor/containerd/containerd.go and daemon/pkg/plugin/manager.go. Those files concern the plugin executor’s IsRunning, which takes a plugin ID and is a different method from the zero-argument State.IsRunning() targeted here; inverting the State method alone does not touch that interface. The diff and diagnostics behind the rejection were not published, so whether the proposal had also rewritten the unrelated method is not shown.

Correction, 2026-10-27. As published on 2026-09-30, this item said the analyzer detected that changing IsRunning() broke plugin executor interface compliance. In the pinned state.go, State.IsRunning() takes no argument, while the plugin executor’s method takes an ID; State does not implement that interface. What was rejected, and why, is not established by the article. The interface-compliance explanation is withdrawn here, in the section heading (published as “with Interface Guard”), in the results table (published as “interface safety rejection”) and in the summary (published as “Interface-safe refactoring: Prevents breaking subtle plugin executor contracts across package boundaries”); the report that no patch was written stands as published.


5. Structural Pattern Matching: 858 Error Propagations in 1.9s

In systems software communicating with the Linux kernel and containerd over gRPC sockets, proper error forwarding is mandatory.

We ran structural AST search:

$ prod-code -r node-a:9400 structural-search 'if err != nil { return nil, err }'

In 1,928 ms, across 2,268 source files, the engine mapped 858 exact structural occurrences across 285 distinct files:

  • Systematically captured across client/client.go, client/container_logs.go, client/image_pull.go, and client/hijack.go.
  • As reported, matched multi-line returns and nested error wraps; the article lists files rather than the matches, so it does not show the match set’s precision against non-nil variable returns.

6. Remote In-Memory Validation (2.00s)

When modifying daemon dispatch logic, waiting for local go test runs or docker builds slows down feedback loops.

Testing prod-code validate with unclosed syntax:

$ cat << 'EOF' | prod-code -r node-a:9400 validate daemon/start.go
package daemon

func invalidMobySyntax(
EOF

Returned:

daemon/start.go: 1 error(s), 0 warning(s)
  error: expected ')', found 'EOF' (daemon/start.go:4:1)
[prod-code] analysed in 2.00s

As reported, the error came back in 2.00 seconds. It is a parse error in the proposed contents of daemon/start.go, the one buffer the output shows was checked; it is not a validation of the 387K-line repository.


7. Compatibility Matrix for the Named Tools: Moby (Docker)

The table, as published, names 21 of the 67 tools; its results are the article’s reported outcomes, not per-tool run records.

Category Tools Tested Result Latency / Metric
Diagnostics & Outline outline, symbols, type_at, hover ✅ Passed 20,508 declarations indexed
Navigation definition, references, callers, callees ✅ Passed containerd shim & daemon traversal
Search & Discovery search, structural_search, duplicates ✅ Passed 224 varint clones found; 858 error guards in 1.9s
Program Slicing slice ✅ Passed 99.9% reduction on ContainerStart (387K → 250 lines)
Architecture dependencies ✅ Passed Zero cyclic dependencies across daemon modules
Refactoring (Functions) extract_function, change_signature, rename ✅ Passed Container state refactoring
Refactoring (Types/Params) parameter_object, invert_boolean ✅ Passed 19-file boolean inversion; reported rejection, cause not published
Validation & Safety validate_edit, validate_edits, shadow_run ✅ Passed 2.00s remote in-memory syntax validation

Correction, 2026-10-27. As published on 2026-09-30, this section was headed “Full 67-Tool Compatibility Matrix”, and the opening said all 67 tools were run. The table names 21 tools, and the article shows output for a subset of those; it does not establish an outcome for every tool. The Architecture row reports zero cyclic dependencies across daemon modules, but no dependency output was published, so the article does not establish acyclicity. The full-suite and acyclicity claims are narrowed to what the article reports; the table itself is kept as published.


Summary: What This Means for Systems Engineering

Moby’s scale-387K lines of Go orchestrating low-level Linux primitives-demands precision:

  • Offloaded analysis: builds and language server indexes ran on the LAN cluster nodes rather than the laptop; the article reported 0% local CPU, without a published local measurement.
  • Millisecond-scale slicing: the prod-code slice run reported above took 420 ms, as published on 2026-09-30; that one reported call was a matter of hundreds of milliseconds, not microseconds, and no other slice timings are on record.
  • Reported refactoring rejection: the article reports that the 19-file inversion was not written; the cause was not published, so no claim is made that the tool protected plugin executor contracts.

Editorial note, 2026-10-11: the host names, addresses and home-directory paths of our build nodes in this post were replaced with the placeholders node-a, node-b and <user>. No other part of the quoted output was changed.

Cite this article
Citation
Alexander Panasenko (2026-09-30). Moby Under the Microscope: What 67 AST Tools Found Inside Docker's Engine. https://prod.codes/blog/moby-under-the-microscope-67-ast-tools/