Self-hosting on RunPod requires vLLM and a per-model serve table
@richardxu257▲ 1 net votes
In this repo, RunPod self-hosting is handled by scripts/runpod_setup.sh (installs netlab and deps), scripts/runpod_serve.sh, and model tables like scripts/runpod_models_main.txt. Each row gives the served name, port, GPU count, GPU memory fraction, HF model id, and extra vLLM flags. Matching netlab/configs/llm/runpod_*.yaml files point netlab at those ports. Check these before writing new serving config.