design note · · 7 min · updated 2026-10-27

From 1,623 µs to 129 µs: zero-copy SIMD wire streaming

A 5.7 MB binary wire block stream took 73 milliseconds to parse with standard MessagePack deserialization, one cost on an ingest thread reported to fall behind in market bursts. How SIMD search and fixed-point integers took per-block latency from 1,623 µs to 129 µs and throughput to 978 MB/s.

On this page · 5 sections
  1. The Wire Format
  2. Why the Naive Parser Stalled
  3. The Zero-Copy SIMD Engine
  4. Hardware Benchmarks on Zen 4
  5. Production Impact

First published on 2026-09-30. The measurements reported in this article are historical and have not been re-run for this revision.

Our market data engine, connecting to a high-throughput financial L1, faced a choice. For the exchange it targets, which this article does not name, the author reports that the public WebSockets either sample order books at 500 ms intervals or drop connections under volatility; no source or measurement for this is given. Running a full follower node solves sampling, but by the same unsourced account it demands dozens of gigabytes of RAM and wears NVMe write endurance reconstructing state that trading bots discard within milliseconds.

The direct path is the exchange’s peer-to-peer gossip wire: raw TCP connections on the node communication ports, streaming consensus blocks as they are validated. You bypass public rate limits and receive unsampled executions tens of milliseconds ahead of public APIs.

There is one complication: the wire stream arrives as compressed, multi-megabyte binary MessagePack frames containing thousands of heterogeneous action payloads per block.

Our initial parser followed standard Rust practice: scan slices for action tags, deserialize them into serde_json::Value, and parse prices as floating-point numbers. On a 5.7 MB stream of 45 wire blocks, that parser took 73 milliseconds (1.62 ms per block), whereas the SIMD zero-copy engine finished in 5.8 milliseconds (129 µs per block). As originally reported, that parser was one cost on an ingest thread that, in volatile market bursts, fell behind and dropped packets before they reached our matching engine; no loss or CPU data from that period is on record, and the figures here do not show how much of it was parsing.

Correction, 2026-10-27. As published on 2026-09-30, the paragraph above said that at ten blocks per second the ingest thread saturated and packets were dropped because of it. The article reports elapsed parsing time, not CPU time or a measured loss rate; at the reported average, with ten comparable blocks handled one after another, ten blocks per second is about 16 ms of parsing per second, which does not by itself establish saturation. The causal claim is withdrawn; the reported timings stand as published. The desktop figure’s “CPU core 100% pegged” label is changed for the same reason.

Here is how replacing generic deserialization with a CPU-dispatched SIMD scanner, a zero-copy MessagePack cursor, and 10⁸ fixed-point integer arithmetic dropped per-block latency from 1,623 µs to 129 µs - a 12.5x speedup running at 978 MB/s with bit-for-bit data parity.


The Wire Format

Nodes communicate over TCP framing: a 4-byte big-endian payload length, a 1-byte compression flag (0x01 compressed with LZ4 or Zstandard, 0x00 raw), and the binary payload.

Once decompressed, the payload is an unindexed MessagePack stream containing consensus state, block headers, state hashes, and user actions. The actions appear as dynamically nested structures:

// Order action
["order", { "orders": [ [asset, is_buy, "146.10", "71.92", false, "Gtc", cloid] ], "grouping": "na" }]

// Cancel action
["cancel", [ [asset, oid], [asset, oid_2] ]]

// Batch modify action
["batchModify", { "modifies": [ [oid, [asset, is_buy, "146.20", "50.00", false, "Alo", cloid]] ] }]

Three characteristics shape the parsing problem:

  1. String-Encoded Financial Numbers: Prices and quantities are encoded as ASCII decimal strings (e.g. "146.10", "71.92").
  2. Heterogeneous Framing: Cancels appear as 2-element tuples [asset, oid], objects {"a": 0, "o": 12345}, or flat lists.
  3. No Top-Level Index: Action tags ("order", "cancel", "batchModify") are scattered within deeply nested maps and arrays.
Architecture diagram comparing legacy scalar search and heap deserialization against SIMD zero-copy parsing
Two paths from wire bytes to shared memory. The legacy path allocates hundreds of thousands of heap containers per second; the SIMD zero-copy path touches only contiguous memory.

Why the Naive Parser Stalled

The initial prototype parsed blocks with standard high-level abstractions:

// ❌ Scalar byte-by-byte substring scan
for window in data.windows(needle.len()) {
    if window == needle { /* found action */ }
}

// ❌ Deserializing untyped Value trees onto the heap
let val: serde_json::Value = rmp_serde::from_slice(&action_slice)?;

// ❌ Float parsing on the critical path
let px = ord["limit_px"].as_str().unwrap().parse::<f64>()?;
let sz = ord["sz"].as_str().unwrap().parse::<f64>()?;

Profiled on an AMD Ryzen 9 7950X3D, the bottleneck was immediate:

  1. Scalar Byte Scanning: windows(needle.len()) compares overlapping slices and adds per-byte scan work. A substring-search routine can use CPU-dispatched SIMD; the specific implementation depends on the target and detected CPU features.
  2. Heap Thrashing via serde_json::Value: Every parsed action allocated nested heap maps, string buffers, and variant vectors. A single 120 KB block caused thousands of small heap allocations. The author reported memory bandwidth saturating and allocator locks dominating the profile; that profile is not on record.
  3. IEEE-754 Parsing Overhead: Calling str::parse::<f64>() performs mantissa accumulation, exponent scaling, and division according to IEEE-754 rounding rules. Financial systems immediately multiply this float back into a fixed-point integer, paying parsing overhead twice while risking precision drift.

The Zero-Copy SIMD Engine

We rebuilt the decoder under three strict rules:

  1. Avoid per-action MessagePack map/tree allocations; the candidate scan still allocates a Vec per scan.
  2. Use CPU-dispatched SIMD for substring search; do not attribute AVX-512 instructions to memchr::memmem::Finder.
  3. Use branch-minimized fixed-point integer parsing for prices and quantities.

1. Vectorized Tag Discovery

Instead of a sliding byte window, we use memchr::memmem::Finder for CPU-dispatched substring search. On x86-64, the crate documents AVX2 routines when available and SSE2 otherwise; this implementation does not substantiate an AVX-512 claim (memchr documentation).

use memchr::memmem::Finder;

let order_finder = Finder::new(&[0xa5, b'o', b'r', b'd', b'e', b'r']);
let cancel_finder = Finder::new(&[0xa6, b'c', b'a', b'n', b'c', b'e', b'l']);
let modify_finder = Finder::new(&[0xab, b'b', b'a', b't', b'c', b'h', b'M', b'o', b'd', b'i', b'f', b'y']);

// Search complete MessagePack fixstr encodings, not raw prefixes.
let mut hits = Vec::with_capacity(512);
for pos in order_finder.find_iter(wire_bytes) {
    hits.push((pos, ActionTag::Order));
}

The finder needles include the MessagePack fixstr length marker as well as the complete action tag. For example, the five-byte string order is encoded as 0xa5 followed by those five bytes, so it cannot match the longer orders key. After a complete tag match, we probe backward by up to 16 bytes for a MessagePack array header byte (fixarray 0x90..=0x9f or array16 0xdc) before accepting the match as an action envelope. The probe is a heuristic, not a validation: a header-like byte nearby does not prove that it opens the container holding the tag, array32 (0xdd) is not recognized, and a tag written as str8 rather than fixstr, which the MessagePack specification also allows, would not match the needles at all.

2. Zero-Allocation MessagePack Cursor

Rather than materializing intermediate structures, a custom cursor (FastRmpReader) advances directly through the slice:

pub struct FastRmpReader<'a> {
    data: &'a [u8],
    pos: usize,
}

impl<'a> FastRmpReader<'a> {
    #[inline(always)]
    pub fn read_u64(&mut self) -> Option<u64> {
        let start = self.pos;
        let value = (|| {
            let b = *self.data.get(self.pos)?;
            self.pos += 1;
            match b {
                0x00..=0x7f => Some(b as u64),
                0xcc => {
                    let v = *self.data.get(self.pos)?;
                    self.pos += 1;
                    Some(v as u64)
                }
                0xcd => {
                    let slice = self.data.get(self.pos..self.pos + 2)?;
                    self.pos += 2;
                    Some(u16::from_be_bytes(slice.try_into().ok()?) as u64)
                }
                0xce => {
                    let slice = self.data.get(self.pos..self.pos + 4)?;
                    self.pos += 4;
                    Some(u32::from_be_bytes(slice.try_into().ok()?) as u64)
                }
                0xcf => {
                    let slice = self.data.get(self.pos..self.pos + 8)?;
                    self.pos += 8;
                    Some(u64::from_be_bytes(slice.try_into().ok()?))
                }
                _ => None,
            }
        })();
        if value.is_none() {
            self.pos = start;
        }
        value
    }
}

read_u64 restores self.pos to its entry position whenever the marker or its payload is invalid or truncated, so a failed speculative read does not advance the cursor.

3. 10⁸ Fixed-Point Decimal Arithmetic

The wire format this engine reads, as the author reports it, carries perpetual prices and lot sizes with eight decimals, so the parser scales them by a fixed factor of 10⁸ ($1.00000000 = 100,000,000$); that is a property of this feed and this implementation, not a standard across exchanges. Instead of converting ASCII strings to f64 and multiplying, we parse ASCII digits directly into a scaled u64:

#[inline(always)]
pub fn parse_fixed_point_8(bytes: &[u8]) -> Option<u64> {
    if bytes.is_empty() {
        return None;
    }
    let mut int_part: u64 = 0;
    let mut frac_part: u64 = 0;
    let mut in_frac = false;
    let mut frac_digits = 0usize;

    for &b in bytes {
        if b == b'.' {
            if in_frac { return None; }
            in_frac = true;
            continue;
        }
        if !b.is_ascii_digit() {
            return None;
        }
        let digit = (b - b'0') as u64;
        if !in_frac {
            int_part = int_part.checked_mul(10)?.checked_add(digit)?;
        } else if frac_digits < 8 {
            frac_part = frac_part * 10 + digit;
            frac_digits += 1;
        }
    }

    while frac_digits < 8 {
        frac_part *= 10;
        frac_digits += 1;
    }

    int_part.checked_mul(100_000_000)?.checked_add(frac_part)
}

"146.10" turns into 14,610,000,000 with integer arithmetic only: a multiply and an add per digit, six multiplications to pad the fraction to eight digits, and a final multiply-add. The function uses no floating-point types and allocates nothing on the heap. It does not round: digits past the eighth decimal place are dropped, and an input such as "." with no digits is accepted as zero.

Correction, 2026-10-27. As published on 2026-09-30, this paragraph said the conversion takes three multiplications and an addition and that rounding ambiguity is eliminated. The function shown above does one multiply-add per digit plus the padding loop and a final multiply-add, truncates rather than rounds beyond eight fractional digits, and returns Some(0) for a bare ".". The operation count and the rounding claim are withdrawn; the code is kept as published.


Hardware Benchmarks on Zen 4

We benchmarked the legacy parser against the zero-copy engine on production hardware:

  • CPU: AMD Ryzen 9 7950X3D (16 physical cores, 4.2 GHz base, 5.7 GHz boost, AVX-512)
  • Dataset: 45 real wire blocks (5.70 MB uncompressed MessagePack stream)
  • Measurement: 50 iterations (2,250 blocks, 766,900 actions decoded)
Benchmark comparison bar chart showing block latency and throughput improvements on Zen 4
Benchmark results on AMD Ryzen 9 7950X3D. Latency dropped from 1,623 µs to 129 µs; throughput reached 978 MB/s.
================================================================================
                    P2P WIRE PARSER HARDWARE BENCHMARK
================================================================================
Dataset:                 5.70 MB uncompressed wire blocks (45 blocks)
Total Processed:         2,250 blocks, 766,900 actions decoded
Parity Verification:     100.00% MATCH across all 766,900 actions
--------------------------------------------------------------------------------
Metric                     Legacy Decoder       SIMD Zero-Copy       Improvement
--------------------------------------------------------------------------------
Total Time (50 runs)        3,653.61 ms            291.53 ms           12.5x faster
Latency per Block           1,623.83 µs            129.57 µs           12.5x faster
Throughput                     78.05 MB/s          978.11 MB/s          12.5x throughput
Heap Allocation          Thousands per block     Preallocated Vec/scan Not zero
Price/Size Parsing       f64 string-to-float     Fixed-Point 10^8 Int   Direct math
Search Algorithm         data.windows().pos()    memchr CPU SIMD      Vectorized
================================================================================

The article reports that across 766,900 decoded actions (orders, cancels, batch modifies), the zero-copy engine produced results bit-for-bit identical to the reference parser. The corpus, the reference parser’s version and the fields compared are not recorded, so this parity result is author-reported and does not by itself establish decoder correctness beyond that corpus.


Production Impact

At 129 µs per block, wire parsing stops being a bottleneck.

Parsed actions feed directly into our lock-free shared memory publisher (ringfire), committing order book deltas into /dev/shm in 27-54 nanoseconds. Trading processes on the same machine consume the stream with zero system call overhead.

In a continuous 41-minute verification run against the exchange’s public WebSocket across ten major perpetual instruments, the article reports the following; the harness, the comparison fields and the alignment method are not published:

  • Trade Parity: 100.00% exact 1-to-1 match (15,990 / 15,990 trades, 0 phantom trades, 0 drops).
  • Top-of-Book (BBO) Parity: 99.76% agreement. The article attributed the differences to sub-millisecond network transit order; no alignment record is shown, so that cause is unverified.
  • Process Footprint: 122 MB of RSS, as reported. The 40+ GB given for the official follower node is for a different process doing different work, not this decoder on the same workload, so the two figures are not a like-for-like comparison.

When your wire format is binary, generic deserializers are convenient for test fixtures, but fatal on the ingest path. Moving from serde_json::Value to CPU-dispatched SIMD searching and fixed-point integers reclaimed 1.5 milliseconds per block - turning a lagging consumer into a sub-millisecond market data feed.

Cite this article
Citation
Alexander Panasenko (2026-09-30). From 1,623 µs to 129 µs: zero-copy SIMD wire streaming. https://prod.codes/blog/from-1623-us-to-129-us-zero-copy-wire-streaming/