curl Under the Microscope: What 67 AST Tools Found Inside the Ubiquitous Transfer Engine

We benchmarked all 67 prod-code AST tools against curl/curl: 200,000 lines of C, 1,924 failure and diagnostic egress points, and parameterized AST codemods.

curl/curl is the world’s most widely deployed command-line tool and library for transferring data across network protocols, powering virtually every internet-connected operating system, mobile device, car, and cloud server on the planet.

Written in portable, ANSI C-compliant code, the repository encompasses over 200,000 lines of C across 1,048 source and header files (with 70,295 declarations indexed monorepo-wide). Maintaining rock-solid stability and zero-regression security across decades of networking standards requires high-fidelity semantic analysis. C codebases cannot rely on static typing alone; cross-subsystem error logging, protocol state machines, and connection filters demand structural AST intelligence.

We evaluated all 67 prod-code AST tools against curl/curl (commit 69d924c72c) hosted on remote cluster nodes with zero local laptop CPU consumption.

Isolating Easy Handles and Transfer Entry Points Across 70,000 Declarations

The core transfer contract in libcurl revolves around the curl_easy_perform synchronous workflow and the internal Curl_easy state handle. Navigating to the primary entry points and understanding handle centrality usually requires chasing typedefs and macros through extensive header hierarchies.

We queried reciprocal-rank-fusion (RRF) semantic search for curl_easy_perform:

$ prod-code search "curl_easy_perform"
10 hit(s) for `curl_easy_perform` in 328 ms (70295 declarations, 1101 files; ranked by words and typed graph only)

 1. [function] curl_easy_perform  tests/libtest/lib3208.c:51
    curl_easy_perform(curl);
    attribution [score 0.0164]: lexical: rank 1 (matched curl, easy, perform)

 2. [function] curl_easy_perform  tests/libtest/lib3208.c:86
    curl_easy_perform(curl);
    attribution [score 0.0161]: lexical: rank 2 (matched curl, easy, perform)

 3. [function] curl_easy_perform  docs/examples/address-scope.c:55
    result = curl_easy_perform(curl);
    attribution [score 0.0156]: lexical: rank 4 (matched curl, easy, perform)

 8. [struct] Curl_easy  lib/altsvc.c:532
    struct Curl_easy *data,
    attribution [score 0.0148]: graph: rank 1 (struct 'Curl_easy' in-degree 1573 (centrality 4.83))

10. [struct] Curl_easy  lib/api.c:192
    struct Curl_easy *data = curl;
    attribution [score 0.0145]: graph: rank 2 (struct 'Curl_easy' in-degree 1573 (centrality 4.83))

In 328 milliseconds across 70,295 declarations, the search engine indexed the typed graph to surface the foundational Curl_easy struct handle, calculating an in-degree centrality score of 4.83 with 1,573 incoming structural references across protocol handlers.

Structural Auditing of 1,924 Error and Diagnostic Egress Points

Network transfer engines must communicate detailed protocol diagnostics while failing cleanly when proxies, TLS handshakes, or framing layers break. In curl, failure logging is handled via failf(...), while verbose diagnostic traces route through infof(...).

We executed structural AST search for protocol failure points matching failf($$$):

$ prod-code structural-search 'failf($$$)'
1134 match(es) in 75 file(s) (1101 scanned in 3502.50ms)

  • lib/cf-h1-proxy.c:128:5  failf(data, "%s cannot be done over CONNECT", cf->conn->scheme->name)
    └─ [$$$ = data, "%s cannot be done over CONNECT", cf->conn->scheme->name]
  • lib/cf-h1-proxy.c:265:5  failf(data, "Failed sending CONNECT to proxy")
    └─ [$$$ = data, "Failed sending CONNECT to proxy"]
  • lib/cf-h1-proxy.c:359:9  failf(data, "Invalid Content-Length: in %03d response", k->httpcode)
    └─ [$$$ = data, "Invalid Content-Length: in %03d response", k->httpcode]
  • lib/cf-h2-proxy.c:796:5  failf(data, "Failed sending CONNECT to proxy")
    └─ [$$$ = data, "Failed sending CONNECT to proxy"]
  • lib/cf-h2-proxy.c:946:5  failf(data, "Could not initialize nghttp2")
    └─ [$$$ = data, "Could not initialize nghttp2"]

In 3,502 ms, the structural search extracted 1,134 explicit failure dispatch branches across 75 files, capturing the Curl_easy handle and variable argument bindings.

Next, we scanned for verbose protocol traces matching infof($$$):

$ prod-code structural-search 'infof($$$)'
790 match(es) in 79 file(s) (1101 scanned in 2743.97ms)

  • lib/altsvc.c:546:9  infof(data, "Bad alt-svc hostname, ignoring.")
    └─ [$$$ = data, "Bad alt-svc hostname, ignoring."]
  • lib/cf-h1-proxy.c:179:5  infof(data, "CONNECT%s phase completed for HTTP proxy", ...)
    └─ [$$$ = data, "CONNECT%s phase completed for HTTP proxy", ...]
  • lib/cf-socket.c:718:7  infof(data, "socket successfully bound to interface '%s'", iface)
    └─ [$$$ = data, "socket successfully bound to interface '%s'", iface]
  • lib/cf-socket.c:867:7  infof(data, "Local port: %hu", port)
    └─ [$$$ = data, "Local port: %hu", port]

In 2,743 ms, the tool identified 790 informational logging sites. Combined, 1,924 protocol diagnostic and failure egress points were mapped across the entire codebase without substring noise from comments or string literals.

AST Clone Detection in Client Invocations and Test Suites

Unit and integration test suites for network clients frequently duplicate boilerplate handle initialization and tear-down sequences across hundreds of independent test programs.

We ran AST clone detection with a 12-line minimum similarity window:

$ prod-code duplicates --min-lines 12
[Clone Group #780] 12 lines | 17 occurrences (Type-2 (Parameterized))
  • Occurrence 1: tests/libtest/lib3100.c:28-39
  • Occurrence 2: tests/libtest/lib598.c:28-39
  • Occurrence 3: tests/libtest/lib2805.c:35-46
  • Occurrence 4: tests/libtest/lib676.c:28-39
  • Occurrence 5: tests/libtest/lib519.c:28-39
  Preview:
    │   CURLcode result;
    │   CURL *curl;
    │ 
    │   if(curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK) {
    │     curl_mfprintf(stderr, "curl_global_init() failed\n");
    │     return TEST_ERR_MAJOR_BAD;
    │   }

[Clone Group #2287] 12 lines | 16 occurrences (Type-2 (Parameterized))
  • Occurrence 1: tests/libtest/lib1975.c:71-82
  • Occurrence 2: tests/libtest/lib1935.c:56-67
  • Occurrence 3: tests/libtest/lib1971.c:69-80
  • Occurrence 4: tests/libtest/lib1959.c:61-72
  Preview:
    │ 
    │   result = curl_easy_perform(curl);
    │ 
    │ test_cleanup:

Clone Group #780 surfaced 17 identical occurrences of global initialization logic across tests/libtest/lib*.c. Clone Group #2287 surfaced 16 identical execution and cleanup scaffolds. Pinpointing these duplicate sequences allows test infrastructure maintainers to extract common harnesses using code_extract_function.

Parameterized Multi-File AST Codemods Across Connection Filters

Refactoring connection filter error messages or telemetry formatting across C modules requires semantic pattern matching that respects AST token boundaries and avoids unintended string substitutions.

We executed a structural AST codemod to update proxy response error descriptions:

$ prod-code codemod 'failf(data, "CONNECT response too large") ==>> failf(data, "CONNECT proxy response exceeded maximum allowed length")'
`failf(data, "CONNECT response too large") ==>> failf(data, "CONNECT proxy response exceeded maximum allowed length")`
4 changed line(s) in 1 file(s)

--- a/lib/cf-h1-proxy.c
+++ b/lib/cf-h1-proxy.c
@@ -606,5 +606,5 @@
       /* non-blank, insert a space then continue the unfolding */
       if(curlx_dyn_addn(&ts->rcvbuf, " ", 1)) {
-        failf(data, "CONNECT response too large");
+        failf(data, "CONNECT proxy response exceeded maximum allowed length");
         return CURLE_RECV_ERROR;
       }
@@ -612,5 +612,5 @@
     }
     if(curlx_dyn_addn(&ts->rcvbuf, &byte, 1)) {
-      failf(data, "CONNECT response too large");
+      failf(data, "CONNECT proxy response exceeded maximum allowed length");
       return CURLE_RECV_ERROR;
     }

nothing was written; pass `apply: true` to make these edits

Next, we tested a parameterized codemod binding variable arguments with $p:

$ prod-code codemod 'infof(data, "Local port: %hu", $p) ==>> infof(data, "Bound local ephemeral port: %hu", $p)'
`infof(data, "Local port: %hu", $p) ==>> infof(data, "Bound local ephemeral port: %hu", $p)`
2 changed line(s) in 1 file(s)

--- a/lib/cf-socket.c
+++ b/lib/cf-socket.c
@@ -865,5 +865,5 @@
     if(bind(sockfd, sock, sizeof_sa) >= 0) {
       /* we succeeded to bind */
-      infof(data, "Local port: %hu", port);
+      infof(data, "Bound local ephemeral port: %hu", port);
       conn->bits.bound = TRUE;
       return CURLE_OK;

nothing was written; pass `apply: true` to make these edits

The AST codemod engine resolved the expression tree for port, substituted the updated format string, and preserved all argument relationships without altering files on disk.

In foundational network client software written in C, managing thousands of protocol error and telemetry pathways requires structural AST pattern search and verified codemods to ensure that diagnostic changes never introduce subtle memory corruptions or protocol desynchronization.