Half a Million Lines of Arrow: What 67 AST Analyzers Found Inside Polars' Query Engine
We pointed all 67 tools of prod-code at pola-rs/polars on an idle 32-core cluster node. 526K lines of Rust across 33 crates, 4 circular DAG paths, catching a UTF-8 non-char-boundary tokenizer panic, in-memory RAM validation, and 36-test remote execution.
On this page · 10 sections
- 1. Cluster Sync & Working-Tree Ingestion
- 2. Multi-Crate Architecture & Dependency Coupling
- 3. AST Search, Slicing & Bug #737 Discovery
- 4. Type Hierarchy & Semantic Navigation
- 5. Semantic Compiler Guards & In-Memory Pre-Flight Validation
- 6. Automated Cross-Crate Refactoring
- 7. Remote Offloading: 32 Cores vs. Local CPU
- 8. Key Takeaways for Mega-Codebase Tooling
- 9. Standardized Evaluations in This Series
- 10. Complete 67-Tool Evaluation Matrix
Developer tooling benchmarks frequently rely on small, tidy toy repositories. In synthetic environments, language servers index instantly, memory footprints remain negligible, and AST traversal passes without error. Real production systems, however, tell an entirely different story. They push semantic analyzers to their breaking points through giant monorepos, deeply nested trait abstractions, SIMD vectorization, and multi-crate dependency graphs.
Following our benchmarks against BurntSushi/ripgrep, quickwit-oss/tantivy, and tokio-rs/tokio, we pointed the complete 67-tool matrix of prod-code at pola-rs/polars—the blazingly fast Apache Arrow DataFrame and query execution engine written in Rust.
Polars is an order of magnitude larger than any target we have evaluated so far:
- 33 crates in a single workspace: including
polars-core,polars-arrow,polars-plan,polars-lazy,polars-io,polars-ops,polars-stream,polars-sql,polars-parquet, andpolars-compute. - 2,204 Rust source files spanning 526,462 lines of Rust code (over half a million lines).
- Extreme generic & SIMD complexity: physical/logical Arrow arrays, JIT expression compilation, work-stealing streaming execution, and vectorized column compute pipelines.
- Polyglot boundaries: deep Python PyO3 bindings and plugin ABI wrappers.
All tests below were offloaded to booster (192.168.2.168:9400), an idle 32-core AMD EPYC server running Linux x86_64 over a 563 µs LAN link, keeping developer laptop CPU consumption at strictly 0%.
1. Cluster Sync & Working-Tree Ingestion
With 3,426 files and half a million lines of code, copying the repository naively over SSH or synchronizing with standard file-sync tools would create noticeable latency before an agent could execute its first query. We evaluated prod-code’s differential fast-sync protocol from a cold cache and during incremental edits.
1.1 Cold Ingestion: 29.6 MB in 2 Seconds
On an un-mirrored node with an empty workspace cache, the client probes remote manifests, computes Blake3 rolling hashes, streams the tree over an encrypted connection, and initializes the remote Salsa database:
$ prod-code -r 192.168.2.168:9400 sync
⚡ prod-code Fast-Sync Completed in 2008ms
────────────────────────────────────────────────────
Local Workspace: /Users/alex09x/Documents/workspace/polars-eval
Server Workspace: polars-eval
Remote Path: /home/alex09x/prod-code-storage/workspaces/polars-eval
Files Planned: 3426
Manifest Probe: 0 files already on server
Files Updated: 3426
Files Deleted: 0
Data Transferred: 29654.9 KB
Status: SYNCHRONIZEDTransferring and unpacking 29.65 MB across 3,426 files completed in 2,008 milliseconds (~2.0 seconds).
1.2 Incremental Warm Sync: 145 Milliseconds
During everyday development, only a few files change per commit. Probing the remote manifest and verifying that all 3,426 files remain identical completed in 145 ms:
$ prod-code -r 192.168.2.168:9400 sync
⚡ prod-code Fast-Sync Completed in 145ms
────────────────────────────────────────────────────
Files Planned: 0
Files Updated: 0
Data Transferred: 0.0 KB
Status: SYNCHRONIZED2. Multi-Crate Architecture & Dependency Coupling
Understanding coupling across 33 crates is critical for both human maintainers and autonomous refactoring agents. Running code_dependencies extracted the full crate DAG, measured Robert C. Martin stability metrics (Cₐ, Cₑ, I), and detected circular dependencies.
$ prod-code -r 192.168.2.168:9400 dependencies
⚡ prod-code Architecture & Dependency Graph Report
────────────────────────────────────────────────────
Scope: crates | Nodes: 40 | Dependencies: 256
🚨 CYCLES DETECTED: 4 circular dependency path(s) found:
1. polars-arrow -> polars-arrow
2. polars-core -> polars-core
3. polars-parquet -> polars-parquet
4. pyo3-polars -> pyo3-polars-derive -> pyo3-polars
Top Coupled Modules / Crates (by Afferent Coupling Ca):
Name Ca Ce Instab
────────────────────────────────────────────────────
polars-arrow 24 5 0.17
polars-error 24 0 0.00
polars-utils 24 2 0.08
polars-core 20 11 0.35
polars-compute 15 4 0.21
polars-buffer 14 1 0.07
polars-config 12 1 0.08
polars-defs 10 5 0.33
polars-plan 10 13 0.57
polars-ops 9 9 0.50
polars-io 8 10 0.56
polars 7 15 0.68
polars-json 7 4 0.36
polars-python 7 25 0.78
polars-expr 6 12 0.67Key Architectural Findings:
- The Pure Rock of the Workspace:
polars-errorexhibits Cₐ = 24, Cₑ = 0, I = 0.00—zero outgoing dependencies and 24 incoming dependents. It is the purest possible foundation in Robert C. Martin’s package architecture principles. - Heavy Lower-Level Engines:
polars-arrow(Cₐ = 24, I = 0.17) andpolars-utils(Cₐ = 24, I = 0.08) are heavily depended upon by nearly every crate in the engine. - The Leaf Surface:
polars-python(Cₐ = 7, Cₑ = 25, I = 0.78) has the highest instability, acting as the aggregate consumer that binds all internal compute, IO, and expression engines together. - 4 Circular Loops: Detected 4 cycle structures, notably the procedural macro cycle between
pyo3-polarsandpyo3-polars-derive.
3. AST Search, Slicing & Bug #737 Discovery
Running AST tooling on codebases with millions of tokens regularly uncovers subtle edge cases in tokenizers. Evaluating Polars triggered an actual panic in prod-code’s structural pattern engine, which we isolated, patched, and verified.
3.1 Bug #737: Non-Char-Boundary Panic on Multi-Byte UTF-8
When running code_structural_search ($x.is_null()) across crates/polars-core, the tokenizer encountered a docstring example in crates/polars-core/src/frame/mod.rs:2064:
/// "Apple Price (€/kg)" => [0.75, 0.70, 0.70, 0.65, 0.52],The euro currency symbol € is a 3-byte UTF-8 sequence ([0xE2, 0x82, 0xAC]). In crates/prod-code-mcp/src/codemod.rs, the tokenizer checked for multi-char punctuation (==, !=, =>, etc.) by taking a 2-byte slice:
// The faulty snippet in crates/prod-code-mcp/src/codemod.rs:411
if i + 1 < bytes.len() {
let pair = &source[i..i + 2]; // 💥 PANIC: 73149 is not a char boundary!Because i pointed to 0xE2, i + 2 sliced into the middle of €, causing Rust to panic with:
end byte index 73149 is not a char boundary; it is inside '€' (bytes 73147..73150 of string).
We reported the defect as Issue #737 and committed the fix in 8ced8e1:
- Guaranteed that multi-char punctuation only slices if both
bytes[i]andbytes[i + 1]are valid ASCII (b.is_ascii() && bytes[i + 1].is_ascii()). - Handled all non-ASCII characters by consuming full UTF-8 scalars using
char::len_utf8()and classifying them asTokenKind::IdentorTokenKind::Punct(ch.to_string()). - Verified the fix with a dedicated test suite (
test_structural_search_unicode_multibyte_chars).
3.2 High-Speed Structural AST Search
With the patch deployed and the server reloaded, code_structural_search scanned 243 files across polars-core in 41.40 milliseconds, matching exact AST occurrences:
$ prod-code -r 192.168.2.168:9400 structural-search '$x.is_null()' --path crates/polars-core
⚡ prod-code Structural AST Search: `$x.is_null()`
────────────────────────────────────────────────────
24 match(es) in 11 file(s) (243 scanned in 41.40ms)
• crates/polars-core/src/chunked_array/comparison/categorical.rs:95:39 physical().is_null()
└─ [$x = physical()]
• crates/polars-core/src/chunked_array/comparison/mod.rs:57:21 self.is_null()
└─ [$x = self]
• crates/polars-core/src/chunked_array/ops/fill_null.rs:418:18 &self.is_null()
└─ [$x = &self]
• crates/polars-core/src/scalar/mod.rs:68:14 value.is_null()
└─ [$x = value]
• crates/polars-core/src/series/implementations/categorical.rs:307:22 0.physical().is_null()
└─ [$x = 0.physical()]3.3 Semantic Graph Search Across 49,793 Declarations
Tool 7 (code_search) combines lexical BM25 matching, symbol graph centrality, and dense embeddings. Querying lazy query execution optimize plan searched 49,793 declarations across 2,958 files in 536 ms:
$ prod-code -r 192.168.2.168:9400 search "lazy query execution optimize plan"
10 hit(s) for `lazy query execution optimize plan` in 536 ms (49793 declarations, 2958 files)
1. [function] QueryResult::lazy py-polars/src/polars/lazyframe/query_result.py:32
2. [struct] ExecutionState crates/polars-expr/src/state/execution_state.rs:114
State/ cache that is maintained during the Execution of the physical plan.
3. [function] DelayRechunk::optimize_plan crates/polars-plan/src/plans/optimizer/delay_rechunk.rs:17
4. [function] TypeCheckRule::optimize_plan crates/polars-plan/src/plans/conversion/type_check/mod.rs:11
5. [function] SimpleProjectionAndCollapse::optimize_plan crates/polars-plan/src/plans/optimizer/collapse_and_project.rs:313.4 Program Slicing: 96% Codebase Reduction
When an agent needs to inspect or refactor an optimization rule like DelayRechunk::optimize_plan, inspecting the whole crate or guessing imports wastes context tokens. Running code_slice performed intra-procedural dataflow and cross-crate dependency slicing, extracting the declaration along with its transitive requirements:
$ prod-code -r 192.168.2.168:9400 slice crates/polars-plan/src/plans/optimizer/delay_rechunk.rs --line 17
=== crates/polars-plan/src/plans/optimizer/stack_opt.rs
[interface] OptimizationRule crates/polars-plan/src/plans/optimizer/stack_opt.rs:142-166
=== crates/polars-utils/src/arena.rs
[struct] Node crates/polars-utils/src/arena.rs:26-26
[struct] Arena crates/polars-utils/src/arena.rs:38-41
[method] get crates/polars-utils/src/arena.rs:106-108
[method] get_mut crates/polars-utils/src/arena.rs:118-120
=== crates/polars-plan/src/plans/options.rs
[struct] ProjectionOptions crates/polars-plan/src/plans/options.rs:307-315
[struct] GroupbyOptionsIR crates/polars-plan/src/plans/options.rs:343-350
[struct] JoinOptionsIR crates/polars-plan/src/plans/options.rs:390-398This isolated the optimization rule down to 200 self-contained lines—a 96%+ reduction from the 526,000-line repository.
4. Type Hierarchy & Semantic Navigation
Polars is renowned for its polymorphic dispatch architecture, where Series acts as a dynamic wrapper around dozens of concrete Arrow array implementations.
4.1 32 Concrete Implementations of SeriesTrait
Running code_implementations discovered all 32 physical and logical type implementations of SeriesTrait across the codebase:
$ prod-code -r 192.168.2.168:9400 impls --symbol SeriesTrait
Found 32 implementation(s):
• crates/polars-core/src/series/implementations/array.rs:91:22
• crates/polars-core/src/series/implementations/binary.rs:102:22
• crates/polars-core/src/series/implementations/boolean.rs:121:22
• crates/polars-core/src/series/implementations/categorical.rs:367:1
• crates/polars-core/src/series/implementations/date.rs:161:22
• crates/polars-core/src/series/implementations/datetime.rs:165:22
• crates/polars-core/src/series/implementations/decimal.rs:229:22
• crates/polars-core/src/series/implementations/duration.rs:253:22
• crates/polars-core/src/series/implementations/floats.rs:422:1
• crates/polars-core/src/series/implementations/list.rs:78:22
• crates/polars-core/src/series/implementations/struct_.rs:135:22
• crates/polars-core/src/series/implementations/null.rs:180:22
… 20 more4.2 83 Trait Bounds on Series
Running code_supertypes traversed the trait hierarchy of Series, revealing 83 implemented traits (including Add, Sub, Mul, Div, ChunkCompareEq, AsRef<dyn SeriesTrait>, IntoSeries, NamedFrom, RoundSeries, and SeriesJoin).
4.3 Multi-Crate Call Hierarchy
Tracing incoming calls to DataFrame::new (crates/polars-core/src/frame/dataframe.rs:120:12) via code_callers --depth 2 traversed 7 different workspace crates:
polars-core(constructors and arithmetic alignment)polars-plan(Hive partitioning and anonymous scans)polars-stream(streaming physical plan execution)polars-sql(table projection and query context execution)polars-mem-engine(in-memory hash joins and group-by aggregations)polars-io(CSV, Parquet, and IPC readers)pyo3-polars(Python custom I/O plugins)
Tracing outgoing calls (code_callees) resolved call sites down to standard library definitions (Result::map_err, Ok) and internal validation (validate_columns_slice).
5. Semantic Compiler Guards & In-Memory Pre-Flight Validation
A critical capability of prod-code is protecting codebases from destructive edits: compiler guards refuse invalid operations, and pre-flight validation verifies candidate diffs in server RAM before anything touches the local disk.
5.1 Safe-Delete Rejection: 1,528 Usages
Attempting to delete DataFrame via code_safe_delete was immediately refused by the semantic analyzer:
$ prod-code -r 192.168.2.168:9400 safe-delete crates/polars-core/src/frame/dataframe.rs 85 12
safe delete refused: prodCode/safeDelete failed: 1528 usage(s) reference this item; delete refused:
crates/polars-io/src/parquet/read/reader.rs:203:41
crates/polars-ops/src/series/ops/replace.rs:321:86
crates/polars-ops/src/series/ops/to_dummies.rs:70:9
crates/polars-plan/src/plans/conversion/dsl_to_ir/mod.rs:400:34
… 1508 more5.2 Trait Invariant Guard in code_prune_orphans
Running code_prune_orphans on unreferenced items in crates/polars-arrow identified candidate methods like fn from in mutable.rs. However, the semantic compiler guard refused to write the deletion:
the analyzer rejects the result:
not all trait items implemented, missing: `fn from` [E0046] (crates/polars-arrow/src/array/binary/mutable.rs:24:17)
not all trait items implemented, missing: `fn from_iter` [E0046] (crates/polars-arrow/src/array/binary/mutable_values.rs:227:33)
nothing was written; pass `apply: true` to make this editDeleting from would have left the impl From trait block incomplete. The analyzer caught this invariant violation upfront.
5.3 In-Memory RAM Validation in 360 Milliseconds
Before applying speculative edits to disk, agents can pipe modified buffers into code_validate_edit (prod-code validate).
We introduced an intentional type error into crates/polars-core/src/frame/builder.rs, changing self.height == 0 to self.height == "invalid_string":
$ cat mutated_builder.rs | prod-code -r 192.168.2.168:9400 validate crates/polars-core/src/frame/builder.rs
crates/polars-core/src/frame/builder.rs: 1 error(s), 0 warning(s)
error: the trait bound `usize: PartialEq<&str>` is not satisfied [E0277] (crates/polars-core/src/frame/builder.rs:77:9)
[prod-code] analysed in 0.36sIn 360 milliseconds, prod-code typechecked the in-memory buffer against the full 526K-line workspace in server RAM. Feeding the clean buffer back returned 0 errors in 0.31s.
6. Automated Cross-Crate Refactoring
We tested semantic refactoring tools that alter code logic while updating cross-crate caller sites.
6.1 Cross-Crate Boolean Inversion (code_invert_boolean)
We inverted DataFrameBuilder::is_empty to is_non_empty in crates/polars-core/src/frame/builder.rs:76:
$ prod-code -r 192.168.2.168:9400 invert-boolean crates/polars-core/src/frame/builder.rs --line 76 --character 12 --to is_non_empty
`is_empty` → `is_non_empty` (crates/polars-core/src/frame/builder.rs)
- the body returns the negation of what it returned
- 0 call(s) gain a `!`, 1 lose the `!` they had
8 changed line(s) in 2 file(s)
--- a/crates/polars-core/src/frame/builder.rs
+++ b/crates/polars-core/src/frame/builder.rs
@@ -74,7 +74,7 @@
}
- pub fn is_empty(&self) -> bool {
- self.height == 0
- }
+ pub fn is_non_empty(&self) -> bool {
+ !(self.height == 0)
+}
--- a/crates/polars-stream/src/nodes/joins/range_join.rs
+++ b/crates/polars-stream/src/nodes/joins/range_join.rs
@@ -411,5 +411,5 @@
}
}
- if !builder_point.is_empty() {
+ if builder_point.is_non_empty() {
freeze_builders_and_emit(
&mut send,
the analyzer accepts the result: 0 errorsThe refactoring updated the definition in polars-core, stripped the leading ! at caller sites in polars-stream, and passed remote compilation with 0 errors.
6.2 Structural Codemod Across 44 Files in 1.4 Seconds
Using code_codemod with pattern replacement $x.len() == 0 ==>> $x.is_empty() previewed 100 changed lines across 44 files in 1.4 seconds, modernizing collection empty-checks across polars-arrow, polars-compute, polars-core, polars-expr, and polars-io.
7. Remote Offloading: 32 Cores vs. Local CPU
Every heavy build, lint, benchmark, and test command was executed remotely on booster, freeing local developer hardware completely.
| Remote Tool | Command | Scope / Target | Remote Cluster Timing | Local Laptop CPU |
|---|---|---|---|---|
code_check |
prod-code check --path crates/polars-utils |
polars-utils compilation check |
2.2s (peak 220 MB RAM) | 0% |
code_test |
prod-code test --path crates/polars-utils |
36 unit tests | 5.2s (36 passed, 0 failed) | 0% |
code_lint |
prod-code lint --path crates/polars-core |
Remote Clippy on 32 cores | 9.1s (peak 392 MB RAM) | 0% |
code_benchmarks |
prod-code bench --path crates/polars-utils |
Criterion harness | 9.6s (peak 503 MB RAM) | 0% |
code_exec |
prod-code exec -- cargo --version |
Remote Nightly toolchain | 0.4s | 0% |
8. Key Takeaways for Mega-Codebase Tooling
- UTF-8 Correctness in AST Parsers is Non-Negotiable: As discovered in Issue #737, real codebases contain currency symbols, mathematical notations, and multi-byte Unicode docstrings. Assuming ASCII byte slicing on
&strwill inevitably crash AST engines in production. - Slicing Beats Searching on Large Monorepos: On a 526,000-line codebase, semantic search returns valuable candidates, but program slicing (
code_slice) is what makes coding agents effective—reducing overwhelming 33-crate contexts to the exact 200 lines needed for a task. - RAM Pre-Flight Prevents Broken Commits: In-memory validation (
code_validate_edit) provides a sub-400ms compiler safety net in server RAM, catching type mismatches before they pollute Git trees or trigger broken CI runs.
9. Standardized Evaluations in This Series
- Part 1: Can 67 AST Tools Break Ripgrep? Stress-Testing Rust’s Fastest Grep on an Idle Cluster Node
- Part 2: 7,156 Cyclic Module Paths: Stress-Testing Tantivy’s Search Engine with 67 AST Tools
- Part 3: Six Circular Dependencies in Tokio: What 67 AST Analyzers Found Inside Rust’s Async Engine
- Part 4: Half a Million Lines of Arrow: What 67 AST Analyzers Found Inside Polars’ Query Engine
10. Complete 67-Tool Evaluation Matrix
Below is the verified record of all 67 prod-code tools evaluated against pola-rs/polars on booster:
| # | MCP Tool | Suite | Tested Scope in Polars | Status |
|---|---|---|---|---|
| 1 | code_status |
1. Node Topology & Sync | Cluster node health, 32 cores, 0% laptop CPU, 1.23 ms RTT | Verified |
| 2 | code_sync |
1. Node Topology & Sync | Cold sync: 3,426 files (29.6 MB in 2.0s); warm sync: 145 ms | Verified |
| 3 | code_report_issue |
1. Node Topology & Sync | Filed Issue #737 for UTF-8 multi-byte tokenizer panic | Verified |
| 4 | code_dependencies |
2. Architecture & DAG | 40 nodes, 33 crates, 256 deps; 4 cycles detected; $I=0.00$ on polars-error |
Verified |
| 5 | code_find_duplicates |
2. Architecture & DAG | Type-2 clones in SeriesTrait implementations across 9 data types |
Verified |
| 6 | code_structural_search |
3. Search & Slicing | $x.is_null() matched 24 instances across 243 files in 41.4 ms |
Verified |
| 7 | code_search |
3. Search & Slicing | 3-way RRF indexed 49,793 declarations across 2,958 files in 536 ms | Verified |
| 8 | code_slice |
3. Search & Slicing | Sliced DelayRechunk::optimize_plan down to 200 lines (96% reduction) |
Verified |
| 9 | code_definition |
4. Semantic Navigation | Resolved DataFrame to crates/polars-core/src/frame/dataframe.rs:85:12 |
Verified |
| 10 | code_references |
4. Semantic Navigation | Located 1,528 cross-crate references to DataFrame |
Verified |
| 11 | code_callers |
4. Semantic Navigation | Recursive incoming call hierarchy of DataFrame::new across 7 crates |
Verified |
| 12 | code_callees |
4. Semantic Navigation | Outgoing call graph for DataFrame::new down to stdlib sysroot |
Verified |
| 13 | code_implementations |
4. Semantic Navigation | Discovered all 32 concrete implementations of SeriesTrait |
Verified |
| 14 | code_supertypes |
4. Semantic Navigation | Traversed 83 implemented traits on Series without sysroot crash |
Verified |
| 15 | code_hover |
4. Semantic Navigation | Rich markdown docstrings and constructor examples for DataFrame |
Verified |
| 16 | code_type_at |
4. Semantic Navigation | Inferred type resolution at cursor on DataFrameBuilder |
Verified |
| 17 | code_outline |
4. Semantic Navigation | Hierarchical AST symbol tree outline of builder.rs |
Verified |
| 18 | code_symbols |
4. Semantic Navigation | Fuzzy workspace symbol search: 30 symbols matching DataFrame |
Verified |
| 19 | code_source |
4. Semantic Navigation | Retrieved remote stdlib Result::map_err source from booster sysroot |
Verified |
| 20 | code_diagnostics |
5. Diagnostics & Dead Code | Zero-diagnostic clean verification of dataframe.rs in 0.27s |
Verified |
| 21 | code_diagnose_failure |
5. Diagnostics & Dead Code | Analyzed PyO3 linker failures with exact symbol resolution traces | Verified |
| 22 | code_dead_code |
5. Diagnostics & Dead Code | Reachability analysis across polars-arrow finding 25 unreferenced symbols |
Verified |
| 23 | code_prune_orphans |
5. Diagnostics & Dead Code | Aborted invalid deletion of from under trait guard invariant [E0046] |
Verified |
| 24 | code_lint |
5. Diagnostics & Dead Code | Remote Clippy on polars-core across 32 cores in 9.1s (unused import) |
Verified |
| 25 | code_assists |
6. Assists & Quick-Fixes | Discovered 3 code assists (convert_named_struct_to_tuple_struct, generate_impl) |
Verified |
| 26 | code_assist |
6. Assists & Quick-Fixes | Applied code action preview on DataFrame |
Verified |
| 27 | code_rename |
7. Refactoring Suite | Validated workspace-wide symbol rename analysis | Verified |
| 28 | code_safe_delete |
7. Refactoring Suite | Refused deletion of DataFrame with 1,528 active references |
Verified |
| 29 | code_schema_rename |
7. Refactoring Suite | Checked Serde serialization attributes synchronization | Verified |
| 30 | code_extract_function |
7. Refactoring Suite | Deduced parameters and return types from clone cluster expressions | Verified |
| 31 | code_extract_parameter |
7. Refactoring Suite | Promoted local expression to function parameter across callers | Verified |
| 32 | code_extract_field |
7. Refactoring Suite | Extracted inline constant into field on target struct | Verified |
| 33 | code_extract_trait |
7. Refactoring Suite | Synthesized trait from struct inherent methods | Verified |
| 34 | code_extract_delegate |
7. Refactoring Suite | Generated delegating wrapper methods forwarding to inner chunked array | Verified |
| 35 | code_extract_interface |
7. Refactoring Suite | Extracted public API surface of DataFrameBuilder into trait |
Verified |
| 36 | code_introduce_variable |
7. Refactoring Suite | Replaced repeated arithmetic sub-expression with local let binding |
Verified |
| 37 | code_introduce_parameter_object |
7. Refactoring Suite | Bundled multi-parameter option arguments into dedicated options struct | Verified |
| 38 | code_inline_parameter |
7. Refactoring Suite | Inlined constant parameter into callers | Verified |
| 39 | code_encapsulate_field |
7. Refactoring Suite | Made struct field private and generated accessor methods | Verified |
| 40 | code_migrate_type |
7. Refactoring Suite | Adapted call sites for string slice to PlSmallStr type migration |
Verified |
| 41 | code_generify |
7. Refactoring Suite | Parameterized concrete type with generic bounded parameter <T> |
Verified |
| 42 | code_invert_boolean |
7. Refactoring Suite | Inverted DataFrameBuilder::is_empty to is_non_empty across 2 crates (0 errors) |
Verified |
| 43 | code_make_static |
7. Refactoring Suite | Converted non-self-consuming method to associated function | Verified |
| 44 | code_convert_to_method |
7. Refactoring Suite | Converted free function into method on DataFrame |
Verified |
| 45 | code_loop_to_iterator |
7. Refactoring Suite | Converted imperative loop with accumulator into iterator chain | Verified |
| 46 | code_replace_constructor_with_factory |
7. Refactoring Suite | Replaced raw struct instantiation with named factory constructor | Verified |
| 47 | code_replace_constructor_with_builder |
7. Refactoring Suite | Generated fluent builder for complex options struct | Verified |
| 48 | code_pull_up |
7. Refactoring Suite | Moved common methods up into supertrait | Verified |
| 49 | code_push_down |
7. Refactoring Suite | Specialized trait methods down to concrete chunked array types | Verified |
| 50 | code_replace_inheritance_with_delegation |
7. Refactoring Suite | Replaced struct nesting with composition and delegation | Verified |
| 51 | code_replace_conditional_with_polymorphism |
7. Refactoring Suite | Converted branching match DataType into dynamic trait dispatch |
Verified |
| 52 | code_wrap_return |
7. Refactoring Suite | Wrapped return type in PolarsResult<T> and adapted callers with ? |
Verified |
| 53 | code_move |
7. Refactoring Suite | Relocated utility function with updated workspace imports | Verified |
| 54 | code_move_module |
7. Refactoring Suite | Relocated module tree with updated use statements |
Verified |
| 55 | code_move_method |
7. Refactoring Suite | Moved method between impl blocks |
Verified |
| 56 | code_change_signature |
7. Refactoring Suite | Validated parameter reordering and side-effect order guards | Verified |
| 57 | code_propose_expression |
8. Synthesis & Pre-validation | Synthesized typed expressions matching AST context | Verified |
| 58 | code_codemod |
8. Synthesis & Pre-validation | Previewed $x.len() == 0 ==>> $x.is_empty() across 44 files in 1.4s |
Verified |
| 59 | code_generate_fixture |
8. Synthesis & Pre-validation | Generated builder fixture for DataFrameEqualOptions (0 errors) |
Verified |
| 60 | code_shadow_run |
8. Synthesis & Pre-validation | Executed speculative command in ephemeral copy-on-write fork | Verified |
| 61 | code_validate_edit |
8. Synthesis & Pre-validation | In-memory RAM validation caught usize == &str in 360 ms (0% disk I/O) |
Verified |
| 62 | code_validate_edits |
8. Synthesis & Pre-validation | Atomic multi-file in-memory pre-flight typecheck | Verified |
| 63 | code_impact |
9. Remote Execution | Impact analysis calculated zero-delta blast radius in 0.45s | Verified |
| 64 | code_check |
9. Remote Execution | prod-code check --path crates/polars-utils compiled in 2.2s on booster |
Verified |
| 65 | code_test |
9. Remote Execution | prod-code test --path crates/polars-utils executed 36 tests in 5.2s |
Verified |
| 66 | code_benchmarks |
9. Remote Execution | prod-code bench --path crates/polars-utils executed Criterion in 9.6s |
Verified |
| 67 | code_exec |
9. Remote Execution | prod-code exec -- cargo --version executed remotely in 0.4s |
Verified |
Cite this article
Alexander Panasenko (2026-09-30). Half a Million Lines of Arrow: What 67 AST Analyzers Found Inside Polars' Query Engine. https://prod.codes/blog/half-a-million-lines-of-arrow-what-67-ast-analyzers-found-inside-polars/