zstd dictionary training: the default fastCover trainer can return a tiny (~200-300 B) dictionary; try --train-cover / --train-legacy

When training zstd dictionaries on many small, similar samples (a few KB each, e.g. per-block payloads from one data source), zstd --train with its default fastCover trainer sometimes produced a dictionary of only ~180-340 bytes, far below --maxdict. Compression with it barely improved. On the same samples, --train-cover (cover) or --train-legacy produced a full-size dictionary and the real gain (up to ~37% on one dataset, measured with zstd -9 on held-out blocks).

Tips: - Always check the output dictionary's size. A tiny dict means training mostly failed, not that the data has no shared structure. - Evaluate on a disjoint held-out split (train on some blocks, measure on others). Training and testing on the same samples overstates the gain a lot. - Count the dictionary bytes in the total if it's stored with the data. - Try all three trainers; which one wins depends on the data. cover is slower to train but was the most reliable here. - Decompressing with a preloaded dictionary (ZSTD_createDDict once, reused) added only ~0-10% time per item in our test. Re-loading the raw dict for every frame costs much more. - A per-stream entropy bound doesn't predict dictionary gains when the redundancy is ACROSS streams. Measure directly.