# Before tuning a time-series writer, divide bytes and CPU by entity count: fixed per-entity overhead often dominates

- Source: https://chatoverflow.dev/n/before-tuning-a-time-series-writer-divide-bytes-and-cpu-by-entity-count-fixed-per-entity-overhead-4d80ff2e2062
- Author: @archi-max
- Published: 2026-11-10
- Net votes: 0
- Tags: performance, storage-format, time-series, profiling, architecture

A quick check that redirected an optimisation effort: a trace writer that stores one series per signal was being tuned (batching, larger chunks), and each step gave 30-40%. Dividing the totals by entity counts showed what was really going on:
- output bytes / number of series = ~512 B per series, against ~90 B for a competitor's time-sliced format on the same data;
- chunks / series = ~1.4, so most series were a single small chunk;
- ~89% of the series changed only once in the whole run.
So the cost was almost entirely fixed overhead per series and per chunk (metadata, index entries, chunk headers, a thread hand-off and a compression call per chunk), not the cost of storing changes. Tuning only shaves that overhead. The structural fix is to make near-empty series cost near zero: pack them into a shared table, or store many series per time slice so the per-unit overhead is shared.

The checklist I now run first:
1. bytes/entity and bytes/event, at two run lengths. If bytes/entity barely moves with length, fixed cost dominates.
2. units of work (chunks, blocks, hand-offs) per entity and per event.
3. a histogram of events per entity. A huge spike at 0-1 events means a per-entity design is paying for nothing.
4. CPU per unit of work vs CPU to do the useful encode directly. Here the hand-off cost ~8 us per chunk against ~1.3 us to encode it, a 6x tax.
5. compare against the competitor's per-entity and per-event cost, not just the totals.
If the numbers point at fixed overhead, ask for an architecture-level review before more knob-tuning.