Scalameta Under the Microscope: What 67 Remote AST Tools Found Inside the Scala Metaprogramming Platform

Historical Scalameta Scala metaprogramming analysis with prod-code: 106,270 Scala and Java lines, 24 extracted SBT modules, 202 reported @ast declarations, and a remote Metals refactoring example.

Evaluating code intelligence tools against a syntax library exercises the structures used by other development tools. Scalameta provides libraries to read, analyze, transform, and generate Scala programs. Its Tree Guide documents lossless syntax trees, parsing, quasiquotes, traversal, and transformation.

Scalameta’s code generation, macro annotations, and language dialects complicate analysis. Parser dialect support is distinct from the Scala versions on which the library is built; the current Tree Guide lists Scala 2.12, 2.13, and Scala 3, plus Scala.js and Scala Native. We used selected operations from prod-code’s 67-tool suite against a Scalameta checkout, running AST traversal, dependency harvesting, clone detection, structural search, and Metals diagnostics remotely.

The command excerpts below are historical observations retained from the original publication. No pinned upstream commit or complete measurement environment is recorded here, so counts, paths, line numbers, and scan durations are snapshot-specific and have not been remeasured for this update. The excerpts cover selected operations from the 67-tool suite, not verification of every tool. A cycle-free extracted module graph does not establish all source-level or external dependency relationships; clone counts do not establish runtime performance. Analyzer diagnostics are not a compiler build or test result.

$ git ls-files '*.scala' | wc -l
694
$ git ls-files '*.java' | wc -l
28
$ git ls-files | wc -l
830
$ git ls-files '*.scala' | xargs wc -l | grep -v 'total$' | awk '{s+=$1} END {print s}'
105866
$ git ls-files '*.java' | xargs wc -l | grep -v 'total$' | awk '{s+=$1} END {print s}'
404

The captured inventory reports 105,866 Scala lines and 404 Java lines, totaling 106,270 lines across 722 source files in 830 tracked files. It includes no overall indexing-time or local CPU measurement.

Architecture, Multi-Matrix SBT Topology, and Metaprogramming Tiers

Compiler platforms require pristine architectural layering: syntax tokens cannot depend on AST trees, AST trees cannot depend on recursive descent parsers, and parsers cannot depend on SemanticDB symbol extractors. Scalameta organizes its codebase across 24 cross-built subprojects via sbt-projectmatrix. We ran prod-code dependencies to verify this hierarchy.

$ prod-code dependencies
⚡ prod-code Architecture & Dependency Graph Report
────────────────────────────────────────────────────
Scope: modules | Nodes: 24 | Dependencies: 38

✓ Zero circular dependencies detected. Architecture graph is a clean DAG.

Top Coupled Modules (by Afferent Coupling Ca):
  Name                                Ca    Ce  Instab
  ────────────────────────────────────────────────────
  scalameta/common2                   18     0    0.00
  scalameta/trees2                    12     2    0.14
  scalameta/io                         9     1    0.10
  scalameta/parsers                    8     1    0.11
  scalameta/dialects2                  7     1    0.12
  scalameta/tokenizers2                6     2    0.25
  scalameta/common                     5     1    0.17
  scalameta/scalameta                  5     4    0.44
  scalameta/testkit                    4     3    0.43
  semanticdb/semanticdb                3     1    0.25
  semanticdb/scalac/library            3     1    0.25
  semanticdb/metac                     1     2    0.67
  tests                                0     8    1.00
  bench/scalameta                      0     6    1.00

Isolated Leaf Endpoints: tests, tests-semanticdb, bench/scalameta, scalameta-docs

The captured scan reports 24 modules and no cycles in the extracted module graph ($C = 0$):

The Common Platform Foundation: scalameta/common2 ($C_a = 18, C_e = 0, I = 0.00$) defines core internal macro helpers, invariant checking annotations, and memory-efficient data structures with zero outbound module edges in this report; this does not count all external dependencies.
The Syntax Tree Subsystem: scalameta/trees2 ($C_a = 12, C_e = 2, I = 0.14$) defines the entire immutable Scala AST node hierarchy (Term, Type, Pat, Stat, Decl, Defn, Mod). It depends exclusively on common2 and io ($C_a = 9, C_e = 1, I = 0.10$), maintaining strict purity from parsing or semantic resolution.
Lexing and Parsing Pipeline: scalameta/dialects2 ($C_a = 7, C_e = 1, I = 0.12$) configures language dialect flags; scalameta/tokenizers2 ($C_a = 6, C_e = 2, I = 0.25$) emits lossless token streams (Tokens); and scalameta/parsers ($C_a = 8, C_e = 1, I = 0.11$) constructs syntax trees from token streams. Each stage depends strictly on the prior stage.
The Public Façade and SemanticDB: scalameta/scalameta ($C_a = 5, C_e = 4, I = 0.44$) exposes the unified user-facing entry points, while semanticdb/semanticdb and compiler plugins consume syntax trees to emit structured symbol indices.

Code Duplication, Position Verification Fixtures, and Syntax Tree Clones

Compiler libraries present unique clone profiles: test suites must meticulously verify source positions, offsets, comments, and whitespace across dozens of syntax permutations and Scala language versions. We ran prod-code duplicates across the repository.

$ prod-code duplicates
Files Scanned: 724 | Lines: 106448 | Clone Groups: 20 | Duplication: 2.7%

Discovered Clone Groups:

[Clone Group #934] 6 lines | 34 occurrences (Type-2 (Parameterized))
  • Occurrence 1: tests/shared/src/test/scala-2.13/scala/meta/tests/tokenizers/TokensPositionSuite.scala:841-846
  • Occurrence 2: tests/shared/src/test/scala-2.13/scala/meta/tests/tokenizers/TokensPositionSuite.scala:851-856
  • Occurrence 3: tests/shared/src/test/scala-2.13/scala/meta/tests/tokenizers/TokensPositionSuite.scala:863-868

[Clone Group #4043] 6 lines | 16 occurrences (Type-2 (Parameterized))
  • Occurrence 1: tests/shared/src/test/scala-2.13/scala/meta/tests/tokenizers/TokensPositionSuite.scala:852-857
  • Occurrence 2: tests/shared/src/test/scala-2.13/scala/meta/tests/parsers/dotty/Scala3PositionSuite.scala:1322-1327
  Preview:
    │   )
    │ 
    │   checkPositions[Term](
    │     """..."""

[Clone Group #2422] 6 lines | 16 occurrences (Type-2 (Parameterized))
  • Occurrence 1: tests/shared/src/test/scala-2.13/scala/meta/tests/parsers/dotty/Scala3PositionSuite.scala:2293-2298
  • Occurrence 2: tests/shared/src/test/scala-2.13/scala/meta/tests/parsers/dotty/Scala3PositionSuite.scala:2311-2316
  Preview:
    │        |    case null =>
    │        |      try
    │        |        val bar = baz
    │        |      catch

Across 106,448 lines scanned, the duplicate harvester identified 20 clone groups accounting for 2.7% duplication. The displayed clone groups contain parser regression test fixtures:

Token and Syntax Position Verification (Clone Groups #934 & #4043): With 34 and 16 occurrences across TokensPositionSuite.scala and Scala3PositionSuite.scala, these test blocks declare parameterized syntax snippets (checkPositions[Term](...)) to ensure that every AST node retains exact start and end column offsets.
Dotty Indentation and Pattern Syntax Permutations (Clone Group #2422): Contains 16 occurrences testing significant indentation syntax (if cond then ..., try ... catch ...) across different Scala 3 dialect settings.

Because these test fixtures intentionally vary single syntactic constructs to prevent regression bugs in editor syntax highlighting, keeping them explicit ensures complete test isolation.

Structural AST Invariants, The 202 Syntax Nodes, and Dialect Modernization

In Scalameta, the queried concrete syntax classes are declared with the @ast macro annotation. We executed prod-code structural-search to query the syntax tree definition catalog across the entire library.

$ prod-code structural-search '@ast class $A'
202 match(es) in 1 file(s) (scanned in 749.69ms)

  • scalameta/trees2/shared/src/main/scala/scala/meta/Trees.scala:195:3   @ast class Comment
  • scalameta/trees2/shared/src/main/scala/scala/meta/Trees.scala:250:3   @ast class Block
  • scalameta/trees2/shared/src/main/scala/scala/meta/Trees.scala:279:3   @ast class This
  • scalameta/trees2/shared/src/main/scala/scala/meta/Trees.scala:305:3   @ast class Int
  • scalameta/trees2/shared/src/main/scala/scala/meta/Trees.scala:381:3   @ast class ArgClause
  • scalameta/trees2/shared/src/main/scala/scala/meta/Trees.scala:383:3   @ast class CasesBlock
  • scalameta/trees2/shared/src/main/scala/scala/meta/Trees.scala:385:3   @ast class EnumeratorsBlock

$ prod-code structural-search 'require($A)'
48 match(es) in 24 file(s) (scanned in 256.01ms)

  • scalameta/io/shared/src/main/scala/scala/meta/io/AbsolutePath.scala:12:3 require(toNIO.isAbsolute, s"$toNIO is not absolute!")
  • scalameta/io/shared/src/main/scala/scala/meta/io/RelativePath.scala:12:3 require(!toNIO.isAbsolute, s"$toNIO is not relative!")
  • scalameta/tokens2/shared/src/main/scala/scala/meta/tokens/Token.scala:265:7 require(dialect.allowXmlLiterals, "doesn't support xml")

The captured structural search reports 202 @ast class matches in Trees.scala and 749.69 ms for that scan. This pattern-specific count is not proof of coverage of the entire Scala grammar. The require query reports 48 matches, including path and dialect checks.

In Dialect.scala, unquoting operations contained legacy Yoda-condition style null comparisons (require(null eq unquoteParentDialect)). We executed a dry-run rule using prod-code codemod to modernize these clauses:

$ prod-code codemod 'require(null eq $A) ==>> require($A eq null)'
`require(null eq $A) ==>> require($A eq null)`
2 changed line(s) in 1 file(s)

--- a/scalameta/dialects2/shared/src/main/scala/scala/meta/Dialect.scala
+++ b/scalameta/dialects2/shared/src/main/scala/scala/meta/Dialect.scala
@@ -681,5 +681,5 @@
 
   private[meta] def unquote(unquoteType: UnquoteType): Dialect = {
-    require(null eq unquoteParentDialect)
+    require(unquoteParentDialect eq null)
     privateCopy(unquoteType = unquoteType, allowTypeLambdas = true)
   }

nothing was written; pass `apply: true` to make these edits

The captured dry run emits a proposed 2-line diff. Its output includes no elapsed time.

Remote Semantic Refactoring and Language Server Verification

In scalameta/io/shared/src/main/scala/scala/meta/io/AbsolutePath.scala, relative path resolution is a foundational helper used throughout file reading and SemanticDB generation:

def toRelative: RelativePath = toRelative(PathIO.workingDirectory)
def toRelative(prefix: AbsolutePath): RelativePath = RelativePath(prefix.toNIO.relativize(toNIO))

We used prod-code extract-function to isolate the path relativization expression into a dedicated relativizeAgainst helper method:

$ prod-code extract-function --to 34:100 --name relativizeAgainst \
    scalameta/io/shared/src/main/scala/scala/meta/io/AbsolutePath.scala 34 56
`fn relativizeAgainst` extracted (scalameta/io/shared/src/main/scala/scala/meta/io/AbsolutePath.scala); the selection now reads `relativizeAgainst(RelativePath, prefix, toNIO)`
- no other place in the file has the selection's text

--- a/scalameta/io/shared/src/main/scala/scala/meta/io/AbsolutePath.scala
+++ b/scalameta/io/shared/src/main/scala/scala/meta/io/AbsolutePath.scala
@@ -10,2 +10,6 @@
 /** Wrapper around an absolute nio.Path. */
+private def relativizeAgainst(factory: RelativePath.type, prefix: AbsolutePath, toNIO: nio.Path): RelativePath = {
+  RelativePath(prefix.toNIO.relativize(toNIO))
+}
+
 sealed abstract case class AbsolutePath(toNIO: nio.Path) {
@@ -33,3 +37,3 @@
   def toRelative: RelativePath = toRelative(PathIO.workingDirectory)
-  def toRelative(prefix: AbsolutePath): RelativePath = RelativePath(prefix.toNIO.relativize(toNIO))
+  def toRelative(prefix: AbsolutePath): RelativePath = relativizeAgainst(RelativePath, prefix, toNIO)

the analyzer accepts the result: 0 errors

The captured Metals 1.6.9 response reports 0 errors. The proposal introduces a top-level helper; without a pinned dialect and compiler build, this does not establish compatibility across Scalameta’s cross-built Scala versions.

The selected examples extend the series from Spark, Pekko, and Play to Scalameta’s syntax library. They are not evidence that all 67 tools or the entire Scala ecosystem were verified.

Keep syntax-pattern counts separate from grammar coverage, and analyzer feedback separate from compiler verification.