Pool of warm Verdi -nogui sessions: headless start gotchas, RSS growth, prefetch A/B, value/trace as an oracle

Field notes from running N parallel verdi -nogui -dbdir simv.daidir -ssf dump.fsdb -play serve.tcl sessions (Verdi W-2024.09, Linux, a 1.7 GB FSDB of a 16-core RISC-V SoC with its KDB). Each session loops on stdin, answering srcSetCursorTime t; srcActiveTrace sig (the active driver at t).

Headless start on a host with no Synopsys module and no X server. The three errors, in order: - 'no Qt platform plugin could be initialized' (a crash). Fix: QT_QPA_PLATFORM=offscreen. - 'Can't open display'. Xt still opens $DISPLAY at start even with -nogui. An ssh-forwarded display works, but only while that ssh session lives. If the forward dies during start-up, Verdi hangs right after printing its rcfile line. Sessions already started survive a dropped connection. - 'The VCS_HOME environment variable is not set ... required with -simflow'. -dbdir implies -simflow, which only checks that the variable is set. No VCS is run.

Memory is the real limit, not seats. - A session starts at about 450 MB. - It keeps every signal it has traced and never gives the memory back: 0.46 GB at start, 0.65 GB after 300 traces, 2.8 GB after 450, 6.5 GB after 600. - The jumps come from wide signals. The first trace of a 385-bit FIFO data signal took 25-27 s and added 1.3-1.6 GB. 512-bit signals took 4-8 s and added 0.4-0.8 GB. - 1200 random (signal, time) points across the whole design took a single session past 16 GB. Spread over 4 or 8 sessions they used 24-26 GB in total. - So for a long-lived pool: cap RSS per session and recycle a session past the cap (a new one starts in 18-28 s); start the replacement in the background.

Prefetch A/B. I replayed 599 interactive trace clicks, with 3.4 candidate next clicks per step (the shown step's fan-in), against real sessions. - 1 session, no prefetch: p50 168 ms, p90 353 ms, max 27 s. - 1 session with prefetch: 61% of clicks hit the prefetched answers, but 126 of 599 clicks queued behind a running prefetch. One waited 40 s behind a wide-signal prefetch, because a running srcActiveTrace can't be cancelled. - 2 sessions with prefetch: 86% hits, p90 200 ms. 4 sessions: 89%, p90 148 ms. 8 sessions: 90%, p90 139 ms (no gain over 4). - Prefetching the candidates' own candidates (2 deep) lowered the hit rate to 74-75%: it crowded a 64-answer cache and used more memory. - About 32% of prefetched answers get used. - More sessions without prefetch change nothing for a single sequential client.

Oracle use. - srcGetCurrentValue -delim . sig gives the value at the cursor: hex with no prefix, enum labels for enums, decimal for int, 'X' for an all-x value. - Against a home-grown active-driver engine at 1200 random points: the values matched 1200/1200 and the driver lines matched 92.8%, with ports followed. Verdi is not always right either. It once named a reset-branch assignment for a value the reset can't produce. - The mismatches that were real bugs on our side were initial-block testbench writes and always @* blocks with a default then a conditional override. - Throughput with 8 sessions: about 12 points/s.