← Live dashboard
The dashboard shows clean numbers. None of them arrive that way. Ship positions come off a radio as packed binary, the operator's departures feed rewrites its own timestamps, and both sometimes carry outright garbage. This page takes real inputs, captured in the last few minutes, and shows each step that turns them into what the dashboard uses. Every example is regenerated from live data, by the same code the collectors run.
Raw feeds go in, and nothing leaves the home server except finished results. The collector machine makes outbound requests only; the public site is a mirror it pushes to, so nothing on the internet can reach the archive. The counts below are live.
An SQLite archive in WAL mode, several independent writer processes under systemd, a stdlib HTTP API, and a push-only mirror on Netlify. No framework, no queue: at this volume a single file database with short write transactions is the simplest thing that holds up, and the checks in section 05 are what keep it honest.
Grey: framing (talker, fragment count, channel). Blue: the payload. Green: the checksum, an XOR over every character between ! and *.
The payload is not text. Each character carries six bits: subtract 48 from its ASCII code, and subtract 8 more if the result is above 40. These are the first eight.
Every field has a "not available" value that looks like a real number (speed 102.3 knots, heading 511 degrees, latitude 91). Read naively, those become impossible readings in the archive; here they become nulls.
Some inputs are not unusual readings but impossible ones, and those are dropped or nulled as they arrive. Each rule below exists because the thing it catches actually came through.
The battery is only worth having if it finds things. These are real, and recent.
A 23:30 Horseshoe Bay to Nanaimo (Departure Bay) sailing left at 00:08. Every timestamp was built from the sailing's scheduled day, so it appeared to leave before the ship had arrived from its previous run. The same mistake sat in four places, one of them a self-healing step that would have erased the real departure time. Fixed everywhere with one rule and a test built from the real sequence.
A corrupted radio message put Seaspan Trader past the North Pole, mid-crossing, six seconds before the pooled feed had it exactly where it should be. Coordinates that cannot exist are now refused on arrival; positions that are merely unusual are still stored, so this check keeps seeing them.
The job that turns tracks into crossings took the database write lock on its first insert and held it while rescanning each vessel's history. Live writers timed out about 2,500 times a day until the nightly checks themselves failed to run. The job now computes first and writes in one short burst: six minutes became 21 seconds.
Positions in section 03 are as stored, after the arrival rules in section 05 and the thinning described in section 02. The raw samples behind sections 02 and 04 are the latest inputs each collector received, kept on the side because the archive itself only holds the cleaned form.