Added an ADSB kinematics data exporter for aircraft tracking via adsb.lol. Here's an Osprey doing loops around LA.

Spent the last few days optimizing performance to support many channels. Here's me real-time tracking/searching every active satellite in orbit via celestrak.org w/ kinematics propagated at 1hz x ~100k channels. UI is super low latency and server is not breaking a sweat at ~6M updates per minute.

For better or worse, the complements of the web were successfully commodified. Linux won the datacenter, then open browser engines/ecosystems won the desktop, and now the process is playing out in AI (especially Chinese labs). Once it happens, the marginal opportunity in an N+1 project is low.

I wrote an exporter from my artificial pancreas loop system to the database. I can monitor glucose readings, automated insulin doses, etc., along with summary stats. I use tailscale to send data from my phone to the server, and can view from any machine connected to the VPN (like my laptop). #t1d

These formats tend to trade core metrics for a long tail of features. I'm not totally convinced WASM is worth it here for most data, but could be useful for large datasets where data-to-header ratio is favorable & future accessibility is paramount (ie: census data or climate data).

I built out a binary format for the database, along with gzip/zstd compression (using gzip for now because safari is slow to adopt). A fully loaded channel is now 7.41KB per request. Can probably trim down slightly more with zstd dictionary training, but it's already very compact before compression.

Vanillajs is a thing, and it works great for me. Installing half the internet is a project-by-project problem and I expect we'll see the same supply chain attacks occur in any ecosystem with similar package governance, once they get big enough and old enough for widespread bitrot, ie: rust/cargo.

Added t-digest to the db, which allow percentiles to be estimated with a fixed memory footprint per channel. This is useful for key stats like medians, but also allows me to generate approximate CDF/PDF plots for understanding value distributions. Aiming for ~4KB total footprint per channel sketch.

I also use uplot, which has an edge in practical performance, but with less features and documentation. I find myself struggling with overlapping labels and value text formatting using either lib. While uplot is faster, the API is still memory heavy (array-of-structs data representation).

First pass at a batch POST endpoint; benchmarks for single batch, single channel, best of 5 samples each:

- 1 week @ 1hz: 12ms
- 1 month @ 1hz: 27ms
- 1 year @ 1hz: 221ms

The implementation is similar in timeseries features/function to Windows Task Manager, which is 5.6MB on Windows 11, vs 37KB (uncompressed) for this application - 2x smaller than the OG Task Manager for Windows NT 4.0 in 1996!

Now I'm adding batch endpoints for high throughput / high cardinality.

Added a search bar and a number of demo feeds I'm using for testing the database. Here's me searching for ISS telemetry, then checking a channel that tracks the volume setting on my macbook, which updates in real-time.

I made a video showing all aircraft within 100 miles of the EC president's aircraft during its flight to Plovdiv. One other aircraft exhibited symptoms of GPS interference at around the same time.

Put it on the pile next to "I need to pay for a fresh prescription for insulin every few months for the last 30 years" I guess.

Beautiful symbology on these! Also, TIL SF was full of horse race tracks.

I have a docker-compose I use everywhere for spinning up and orchestrating prometheus and grafana instances. Those tools feel like magic for understanding systems, however, docker + go server + grafana server is a lot of variable overhead. Hoping this will be a more simple & predictable golden path.