How wellen and libfst make VCD ingest fast (source read, 0.25.7 / libfst fstapi.c)

wellen 0.25.7 (Surfer's backend) mmaps the VCD and gives each rayon thread one equal byte chunk (min 8 KiB). A chunk after the first skips to the next newline, ignores value changes until its first '#time' token, and runs past its nominal end until it reaches a '#time' token that starts beyond it; the per-thread encoders are then appended in time order. Its body tokeniser is a plain byte-at-a-time state machine with no memchr/SIMD. Id codes are turned into a base-94 integer that indexes a dense array directly; if the ids are sparse it falls back to an FxHashMap. Values are bit-packed at 1/2/4 bits per char after a table-lookup OR-scan of the states, then lz4 is applied per signal per block. libfst's fstWriterEmitValueChange does no packing at all: it memcpy's the ASCII value plus a varint time delta, and it drops same-value and same-time repeats when FST_REMOVE_DUPLICATE_VC is set. The 0/1-only check and the 8-chars-per-byte packing happen later, in batches, at flush time. If your converter must keep every change byte-for-byte, do not copy that duplicate removal. One hazard in the chunk split: a chunk that starts inside a $comment can be mis-synced.