notes · · 9 min · updated 2026-10-27

Half a Million Lines of Python and 25 Import Cycles: Dissecting Django with 67 AST Tools

We benchmarked all 67 prod-code AST tools against django/django: 527K lines of Python across 2,932 files, 25 circular check paths, 99.9% slicing reduction, and safety refusals and previews in Python predicate inversion and parameter bundling.

On this page · 9 sections
  1. 1. Node Topology & Fast-Sync Ingestion
  2. 2. Architecture DAG: 112 Core Modules & 25 Circular Paths
  3. 3. Clone Harvester: Detecting Parameterized Duplication
  4. 4. Structural AST Search & Semantic Intent
  5. 5. Navigation & Transitive Program Slicing
  6. 6. The Python Refactoring Suite
  7. 7. Pre-Validation in In-Memory Shadow (prod-code validate)
  8. 8. Summary: 67-Tool Evaluation Matrix for Django
  9. Conclusion

First published on 2026-09-30. The measurements reported in this article are historical and have not been re-run for this revision. The article reports using django/django at commit 5a4511ad.

django/django is the gold standard of Python web development. For two decades, it has anchored the Python ecosystem, powering everything from content platforms and massive SaaS APIs to financial engines with its batteries-included philosophy: an ORM with expressive relational queries, an extensible authentication subsystem, built-in migration tooling, and a middleware-driven request pipeline.

Underneath that web framework lies a sprawling Python codebase: 526,995 lines of Python across 2,932 source files, organized into dozens of internal packages (core, db, forms, http, contrib, urls).

Dynamic, dynamically typed languages like Python present distinct hurdles for static analysis and automated refactoring:

  • Type inference engines (like basedpyright) must resolve complex metaprogramming patterns, dynamically attached model managers, class decorators, and conditional runtime imports without choking.
  • Automated refactorings must handle Python’s lack of braces, indentation-sensitive blocks, and first-class function values without breaking runtime syntax.
  • Naive search tools easily get lost in thousands of lines of dynamic dispatch and meta-classes.

To evaluate how our cluster-backed code intelligence architecture handles a flagship Python monolith with the heavy analysis kept off the laptop, we subjected django/django (commit 5a4511ad) to the standardized 67-tool evaluation protocol of prod-code on our 32-core Linux node (node-b).

Here is what 67 AST analyzers discovered inside Django.


1. Node Topology & Fast-Sync Ingestion

prod-code mirrors the local working tree to a remote Linux build node over low-latency LAN:

Local Checkout:   /Users/<user>/Documents/workspace/django-eval
Target Host:      node-b (node-b:9400 / AMD EPYC 32-core, 128 GB RAM)
LAN Latency:      654.88 µs RTT
Python Source Files: 2,932 (.py)
Total Lines of Python: 526,995
Python Engine:    basedpyright-langserver (with stubs up to Python 3.15)

Ingestion Telemetry

  • Differential Cold Sync: 2,932 Python source files hashed, synchronized, and hydrated into basedpyright in server memory.
  • Local Laptop Load: no measurement shown. As reported, all AST parsing, graph traversal, and semantic analysis remained on the remote cluster node; the sync itself and the shell commands below start on the laptop.

2. Architecture DAG: 112 Core Modules & 25 Circular Paths

Running prod-code dependencies across django/core revealed the topological structure of Django’s internal machinery:

⚡ Architecture & Dependency Graph Report
Scope: modules | Nodes: 112 | Dependencies: 832 edges

Foundational Bedrock Modules (Afferent Coupling Cₐ)

Modules with the highest incoming dependencies (Cₐ) and instability at or near zero (I ≤ 0.08) form the structural foundation of Django:

  • django::core::cache::backends::base (Cₐ = 11, I = 0.00): Abstract interface for cache storage drivers.
  • django::core::files::base (Cₐ = 11, I = 0.00): File wrappers and streaming chunk primitives.
  • django::core::files::storage::base (Cₐ = 11, I = 0.00): The file storage abstraction.
  • django::core::handlers::base (Cₐ = 11, I = 0.08): Core request/response middleware chain handler.
  • django::core::management::base (Cₐ = 11, I = 0.00): The CLI command execution engine (BaseCommand).

Circular Import Topology

Python allows circular imports when imports occur inside functions or modules are lazily evaluated. prod-code dependencies detected 25 circular dependency paths within django::core::checks:

  • checks::async_checks -> checks::caches -> checks::async_checks
  • checks::caches -> checks::database -> checks::files -> checks::templates -> checks::translation -> checks::urls -> checks::caches
  • checks::database has the highest reported efferent coupling in the core graph (Cₑ = 111, I = 0.92). At commit 5a4511ad, django/core/checks/database.py is a 14-line module with two imports (django.db.connections and from . import Tags, register), so the 111 counts what is reachable through the checks package, not 111 direct imports.

Correction, 2026-10-27. As published on 2026-09-30, this section presented the 25 paths as circular import topology between the check modules and checks::database as a coordinator with 111 outgoing dependencies. At the pinned commit, database.py and async_checks.py import only from the checks package itself (plus django.db and os), and the package’s __init__.py imports every check module. The listed paths therefore close through the package, not through direct imports between caches, database and the other modules. The module-to-module reading and the coordinator description are withdrawn; the reported counts stand.


3. Clone Harvester: Detecting Parameterized Duplication

Using AST-hash based clone harvesting via prod-code duplicates, we scanned django/core for duplicated logic:

Parameterized Clone Clusters (Type-2 Clones)

The analyzer surfaced over 150 Type-2 clone groups:

  1. Serializer Exception Handlers: Identical generator exception cascades across serializers/pyyaml.py:75-80, serializers/jsonl.py:52-57, and serializers/json.py:75-80:
    def _handle_object(self, obj):
        try:
            yield from super()._handle_object(obj)
        except (GeneratorExit, DeserializationError):
            raise
  2. Command Option Parsers: Identical --no-input argument definitions repeated across 5 management commands (makemigrations, migrate, squashmigrations, test, testserver).
  3. Cache Storage Adapters: Symmetrical get(self, key, default=None, version=None) method implementations across memcached.py and redis.py.

4. Structural AST Search & Semantic Intent

Searching for conceptual intent across the entire codebase:

prod-code search "resolve request url to view callback"

The server scanned 44,985 declarations across 2,975 files in 204 milliseconds, ranking the exact execution entrypoints at the top:

  • Hit 1: django/views/generic/base.py:97 (View.view)
  • Hit 5: django/core/handlers/base.py:302 (BaseHandler.resolve_request)
  • Hit 7: django/urls/resolvers.py:424 (URLPattern.__init__)
  • Hit 8: django/middleware/csrf.py:416 (CsrfViewMiddleware.process_view)

Searching for guard-clause patterns:

prod-code structural-search "if not \$cond: raise \$exc" --path django/core

In 99.70 milliseconds, it located 38 exact syntax matches across 21 files in django/core, binding $cond and $exc to their respective expressions (e.g., $cond = client.exists(key), $exc = ValueError in redis.py).


5. Navigation & Transitive Program Slicing

Transitive Program Slicing (code_slice)

To understand how Django resolves an incoming HTTP request into a view function, developers traditionally trace through multiple modules (base.py, resolvers.py, regex.py).

We executed a depth-2 transitive slice on BaseHandler.resolve_request:

prod-code slice django/core/handlers/base.py --line 302

In 0.12 seconds, prod-code slice extracted:

  1. BaseHandler.resolve_request (seed, depth 0)
  2. get_resolver (depth 1)
  3. _get_cached_resolver (depth 2)
  4. URLResolver.resolve (depth 1)
  5. URLPattern class & URLPattern.resolve (depth 2)
  6. URLResolver._extend_tried and _join_route (depth 2)

As reported, the slice selected 160 lines of the 527,000-line repository (a 99.97% context reduction). Only the symbol names above are shown, not the sliced text, so the article does not establish that those lines form a self-contained, executable chain; the list outlines the routing path from resolve_request through the URL resolvers.

Semantic Navigation

  • prod-code def: Resolved django.core.handlers.base::BaseHandler to django/core/handlers/base.py:21:7.
  • prod-code hover: Displayed class BaseHandler() signature in 15 ms.
  • prod-code callers: Traced both callers of resolve_request:
    • _get_response at line 176 (call site 183:57)
    • _get_response_async at line 230 (call site 237:57)
  • prod-code outline: Produced complete symbol tree of base.py (1 class, 10 methods, 7 attributes, 54 locals).

6. The Python Refactoring Suite

We subjected prod-code’s polyglot refactoring engines to real-world Python transformations in django/utils/inspect.py.

1. invert-boolean: Negation with First-Class Value Guard

We tested inverting is_module_level_function into is_not_module_level_function:

prod-code invert-boolean is_module_level_function \
  --to is_not_module_level_function \
  --path django/utils/inspect.py

Results (every edit below is a candidate preview; the final state is a refusal and nothing was written):

  • Inverted all 3 return statements in the function body (return False → return True, return True → return False).
  • In django/tasks/backends/base.py, proposed rewriting if not is_module_level_function(task.func): as if is_not_module_level_function(task.func):.
  • Proposed rewriting 14 call sites in tests/utils_tests/test_inspect.py as not inspect.is_not_module_level_function(...).
  • Value-Reference Safety Refusal: Line 147 of test_inspect.py passed is_module_level_function as a first-class value argument:
    self.assertIs(inspect.is_module_level_function(inspect.is_module_level_function), True)
    prod-code invert-boolean detected the value reference and refused the unsafe rewrite:
    not rewritten (1 reference(s) that are not a call - a function used as a value keeps its old meaning under its new name; nothing is written while any remains):
      tests/utils_tests/test_inspect.py:147:54 `is_module_level_function` used as a value

Correction, 2026-10-27. As published on 2026-09-30, this section said the tool rewrote 14 call sites and simplified the base.py condition, and the summary table counted them as updated. The quoted output says nothing is written while a value reference remains, so those edits were candidate previews and the final state is a refusal with no writes. The base.py condition as published also dropped the predicate’s argument; at django revision 5a4511ad it reads if not is_module_level_function(task.func):.

2. parameter-object: Bundling Parameters into a Class

We tested bundling (func, name) on func_supports_parameter into FuncParamSpec:

prod-code parameter-object func_supports_parameter \
  --param func --param name --name FuncParamSpec \
  --path django/utils/inspect.py

Results:

  • Generated a new idiomatic Python class in django/utils/inspect.py:
    class FuncParamSpec:
        """The parameters `func_supports_parameter` takes together."""
        def __init__(self, func, name):
            self.func = func
            self.name = name
  • Rewrote declaration: def func_supports_parameter(func_param_spec: FuncParamSpec):.
  • Updated body accesses: func_param_spec.name and func_param_spec.func.
  • Rewrote external call site in django/db/models/query.py:81: func_supports_parameter(FuncParamSpec(func=from_db, name="fetch_mode")).
  • Added required import to query.py: added FuncParamSpec to the import from inspect.
  • Rewrote test call site in tests/utils_tests/test_inspect.py:111.
  • Verified cleanly: the analyzer accepts the result: 0 errors.

3. extract-function: Selection Extraction

We tested extracting "<locals>" in func.__qualname__ at line 108 into a helper function is_local_qualname:

prod-code extract-function django/utils/inspect.py 108 8 --to 108:39 \
  --name is_local_qualname

Generated clean extraction:

def is_local_qualname(func):
    return "<locals>" in func.__qualname__

def is_module_level_function(func):
    ...
    if is_local_qualname(func):
        return False

Validated with basedpyright: 0 errors.


7. Pre-Validation in In-Memory Shadow (prod-code validate)

Before applying edits to disk, prod-code validate verifies proposed diffs in memory against the remote language server.

We tested with an intentional attribute typo (inspect.non_existent_attribute(func) in django/utils/inspect.py):

cat django/utils/inspect.py | \
  sed 's/inspect.isfunction(func)/inspect.non_existent_attribute(func)/' | \
  prod-code validate django/utils/inspect.py

In 2.07 seconds, without touching the file on disk:

  • The server recognized 78 pre-existing type warnings in the file as baseline.
  • It subtracted the baseline and reported only the single newly introduced error:
    django/utils/inspect.py: 1 error(s), 3 warning(s)
      error: "non_existent_attribute" is not a known attribute of module "inspect" [reportAttributeAccessIssue]
    [prod-code] analysed in 2.07s

8. Summary: 67-Tool Evaluation Matrix for Django

The table summarizes the run as the article reports it, in 19 category rows rather than one row per tool. PASS is the author’s status as published; only the operations excerpted in sections 1-7 have output in the article, and the remaining figures in the table are author-reported.

Category Suite Target Subsystem Result Latency / Telemetry
Ingestion Fast-Sync Ingestion Entire Repository PASS 2,932 files synchronized; laptop CPU not shown
Sanitization Scrubbing Telemetry Issue Reporter PASS LAN IPs, node names, home paths scrubbed
Architecture DAG Coupling & Instability django/core (112 modules) PASS 832 dependencies, 25 circular paths detected
Code Quality Clone Harvester django/core PASS 150+ Type-2 clone groups detected
Code Quality Diagnostics django/core/handlers/base.py PASS Strict type analysis in 1.40s
Code Quality Dead Code & Unreferenced Subsystem Scan PASS 20 files, 44 symbols scanned in 108s
Search Semantic Intent Search Entire Repository PASS 44,985 decls in 2,975 files scanned in 204 ms
Search Structural AST Search django/core PASS 38 matches across 21 files in 99.70 ms
Navigation Symbol Definition BaseHandler PASS Resolved to base.py:21:7
Navigation Symbol Layout & Hover BaseHandler PASS class BaseHandler() returned in 15 ms
Navigation Transitive Slicing resolve_request (Depth 2) PASS 99.97% reduction (527K LOC → 160 LOC) in 0.12s
Navigation Callers Tracing resolve_request PASS 2 callers traced (_get_response, _get_response_async)
Navigation File Outline base.py PASS 1 class, 10 methods, 7 attributes, 54 locals
Refactoring Invert Predicate is_module_level_function Reported refusal / no writes 14 call-site edits previewed and a refusal, as the published output states; the original output and tree state are not reproduced
Refactoring Parameter Object func_supports_parameter PASS Generated FuncParamSpec, updated callers & imports
Refactoring Extract Function is_local_qualname PASS Extracted helper, validated with 0 errors
Refactoring References func_supports_parameter PASS Traced all 3 cross-file references
Verification In-Memory Pre-validation inspect.py PASS Subtracted 78 baseline warnings, caught error in 2.07s
Execution Remote Python Exec Python 3.10 Runtime PASS Executed remote command in 0.8s on node-b

Conclusion

Evaluating prod-code against django/django demonstrates that cluster-backed AST intelligence handles dynamic, dynamically typed Python codebases with the same tooling it applies to compiled languages:

  1. Massive Python Monoliths, Analyzed Remotely: Ingesting and analyzing half a million lines of Python on a remote 32-core node keeps the heavy work off the laptop. No local CPU measurement is shown, and the sync and the shell commands described above run locally, so the laptop’s share of the work is not zero.
  2. First-Class Value Safety in Refactorings: Inversion and parameter transformations respect Python’s dynamic idioms, refusing dangerous modifications when functions are passed as first-class values or when imports must be updated across files.
  3. Transitive Slicing Narrows the Context: Depth-limited dependency slices give AI agents a short list of the relevant code in milliseconds instead of hundreds of thousands of tokens.

Editorial note, 2026-10-11: the host names, addresses and home-directory paths of our build nodes in this post were replaced with the placeholders node-a, node-b and <user>. No other part of the quoted output was changed.

Cite this article
Citation
Alexander Panasenko (2026-09-30). Half a Million Lines of Python and 25 Import Cycles: Dissecting Django with 67 AST Tools. https://prod.codes/blog/half-a-million-lines-of-python-inside-django/