Lunary
Open-source Rust parser for Nasdaq TotalView-ITCH 5.0.
Lunary is an open-source Rust parser for Nasdaq TotalView-ITCH 5.0, the binary feed that carries every order event on Nasdaq.
A single trading day holds hundreds of millions of messages. Research and backtests replay these files again and again, so parse speed sets the pace of every experiment.
Lunary memory-maps the input file and decodes messages zero-copy, reading fields in place instead of copying them into new buffers. It uses SIMD when the CPU has it and still runs on scalar-only hardware. Adaptive batching and lock-free parallel processing let it scale from one core to many.
On a full trading day of 293,989,079 messages, full decode runs at 93.9M messages per second on one thread and 499.7M on 8 vCPUs. Framing alone, which walks message lengths without reading fields, runs at 213.7M on one thread and 1,419.0M on 8 vCPUs. These are medians of 10 runs on a Google Cloud c3-standard-8 with the data in memory, on a cloud VM rather than bare metal.
The repo ships the itch-bench binary used for these numbers, so anyone with an ITCH file can run them again. Licensed AGPL-3.0.
Code and benchmarks are on GitHub.