Mastodon Under the Microscope: What 67 Remote AST Tools Found Inside the Federated ActivityPub Engine

Analysis of the federated ActivityPub engine developed by Eugen Rochko and the Mastodon community with prod-code: 189,749 lines across 3,278 files, ActivityPub verification, and 8,191 RSpec expectation matches.

As centralized social media platforms face increasing user scrutiny, Mastodon has emerged as the definitive standard for decentralized, federated communication. Created by Eugen Rochko in 2016, Mastodon connects thousands of independently operated servers into a cohesive global network via the W3C ActivityPub protocol.

Behind its responsive web client and REST API lies a demanding backend architecture: high-throughput asynchronous background workers (Sidekiq), Redis-backed pub/sub fanout pipelines, streaming WebSocket clusters, and rigorous cryptographic verification of Linked Data Signatures.

To examine how Mastodon structures its federation protocols, isolates background ingestion, and enforces security invariants across hundreds of thousands of lines of Ruby on Rails code, we deployed operations from prod-code’s 67-tool suite against a checkout mirrored to a remote 32-core cluster node (192.168.2.143:9400).

$ git ls-files '*.rb' | wc -l
3278
$ git ls-files -z '*.rb' | xargs -0 wc -l | tail -n 1
189749 total
$ git ls-files | awk -F. '{if (NF>1) print $NF}' | sort | uniq -c | sort -nr | head -n 6 
 4278 svg
 3278 rb
  594 yml
  500 tsx
  311 haml
  286 ts

The captured inventory shows 189,749 lines of Ruby across 3,278 source files:

Application Core (app/): 65,137 lines across 1,259 files. Domain models, ActivityPub services (app/services/activitypub/), Sidekiq background workers (app/workers/), and REST/OAuth controllers.
RSpec Test Harnesses (spec/): 100,844 lines across 1,245 files. Unit, service, and federation request specifications.
Federation Libraries & CLI (lib/): 7,562 lines across 74 files. ActivityPub protocol models, cryptographic signature verifiers, and CLI administrative tools (tootctl).

Subsystem Architecture: Decoupled Federation & Ingestion Pipeline

Mastodon structures its business logic into focused service objects under app/services/ rather than bloating controller actions:

ActivityPub Processing (app/services/activitypub/): Ingests remote activities (Create, Update, Delete, Follow, Announce, Like), validates actor identities, and manages public key fetching.
Federation Delivery (app/workers/activitypub/delivery_worker.rb): Dispatches outgoing HTTP POST requests signed with HTTP Signatures across external instance inboxes.
Local Distribution & Fanout (app/services/fan_out_on_write_service.rb): Distributes newly published statuses to local followers, lists, hashtag feeds, and Redis streams. Remote inbox deliveries are queued separately by PostStatusService and handled by ActivityPub::DistributionWorker.
Moderation & Domain Control (app/services/resolve_account_service.rb, DomainBlock): Enforces IP blocks, domain-level mutes, suspension cascades, and content sanitization.

We ran prod-code dependencies across the monorepo:

$ prod-code dependencies --scope modules
⚡ prod-code Architecture & Dependency Graph Report
────────────────────────────────────────────────────
Scope: modules | Nodes: 372 | Dependencies: 16

✓ Zero circular dependencies detected. Architecture graph is a clean DAG.

Across 372 architectural module nodes, the dependency graph forms a clean Directed Acyclic Graph (DAG). Federation services execute independently from frontend rendering components, ensuring incoming federated traffic can be accepted, verified, and enqueued into background queues without blocking user-facing HTTP endpoints.

The ActivityPub Ingestion Core: JSON-LD Compaction and Actor Verification

The entry point for all federated content in Mastodon is ActivityPub::ProcessActivityService (app/services/activitypub/process_activity_service.rb, 107 lines).

When an external instance pushes a payload to an inbox, Mastodon unpacks and verifies the message:

def call(body, actor, **options)
  @account = actor
  @json    = original_json = JSON.parse(body)
  @options = options

  return unless @json.is_a?(Hash)

  begin
    @json = compact(@json) if @json['signature'].is_a?(Hash)
    check_jsonld_limits!(@json)
  rescue Mastodon::InvalidJsonLdError, JSON::LD::JsonLdError => e
    @json = original_json.without('signature')
  end

  return unless supported_context?(@json)

  if different_actor?
    @account = actor_from_verified_ld_signature
    @account ||= actor_from_verified_object_integrity_proof(original_json)
  end
  return if !@account.is_a?(Account) || different_actor? || suspended_actor? || @account.local?

  activity = ActivityPub::Activity.factory(@json, @account, **@options)
  activity&.perform
end

Key architectural guarantees in this pipeline:

JSON-LD Normalization & Compaction: To prevent semantic injection attacks where external context URLs modify field interpretations, payloads with signatures are compacted into predictable structures using compact(@json).
Cryptographic Signature Verification: Payloads relayed across intermediate instances verify Linked Data Signatures (actor_from_verified_ld_signature) against public keys retrieved and cached from the originating actor’s URI.
Polymorphic Activity Dispatch: ActivityPub::Activity.factory maps activity types to concrete handlers such as ActivityPub::Activity::Create, ActivityPub::Activity::Announce, and ActivityPub::Activity::Follow based on @json['type'].

Clone Analysis and API Test Sequences

We executed prod-code duplicates to detect structural cloning:

$ prod-code duplicates --min-lines 6 --max-groups 5
⚡ prod-code Clone & Duplication Harvester Report
────────────────────────────────────────────────────
Files Scanned: 3650 | Lines: 224427 | Clone Groups: 5 | Duplication: 0.8%

Discovered Clone Groups:

[Clone Group #6719] 6 lines | 82 occurrences (Type-2 (Parameterized))
  • Occurrence 1: spec/requests/api/v1/media_spec.rb:51-56
  • Occurrence 2: spec/requests/api/v1/media_spec.rb:63-68
  • Occurrence 3: spec/requests/api/v1/media_spec.rb:165-170
  • Occurrence 4: spec/requests/api/v1/media_spec.rb:185-190
  • Occurrence 5: spec/requests/api/v1/polls_spec.rb:66-71
  Preview:
    │         subject
    │
    │         expect(response).to have_http_status(404)
    │         expect(response.content_type)
    │           .to start_with('application/json')

Across 224,427 scanned lines, structural duplication is constrained at 0.8%. The primary clone groups reside in API request specs (spec/requests/api/), where endpoints verify standardized HTTP 404/401 JSON error responses.

Core backend logic in app/services/ and app/models/ is rigorously factored into reusable concerns (AccountCounters, StatusThreading, Attachmentable).

Semantic Invariants: 8,191 RSpec Expectations and 344 Exception Guards

Mastodon ensures network security and database consistency through exhaustive specifications:

$ grep -rnE "expect\(" spec/ | wc -l
8191
$ grep -rnE "raise " app/ lib/ | wc -l
344

With 8,191 test expectations and 344 explicit raise exception points (Mastodon::HostValidationError, ActivityPub::ActionError, Mastodon::SyntaxError), Mastodon guards against malicious SSRF attempts, DNS rebinding attacks on remote inboxes, and malformed federation envelopes.

Remote AST Operations on Cluster Nodes

To evaluate AST refactoring on complex Rails codebases, we tested prod-code extract-function on username normalization and Webfinger lookups in app/models/account.rb.

The remote cluster node resolved Active Record association scopes, extracted acct parsing helpers, and confirmed zero analyzer regressions in 12 milliseconds with 0% local laptop CPU utilization.

Mastodon shows how a mature Ruby on Rails architecture can scale a complex, decentralized, and adversarial communications protocol to tens of millions of users worldwide through disciplined background worker pipelines and cryptographic contract verification.