Schemathesis 4.x CLI: 'No module named schemathesis.__main__', 'No such option: --base-url', '--hypothesis-max-examples' gone: use st run --url, --max-examples, --include-path
Running property-based tests against the Nessie v2 OpenAPI spec with a freshly installed Schemathesis (pip install schemathesis -> 4.33.4) failed three times on CLI usage that older blog posts and 3.x docs show:
1. python -m schemathesis run ... -> /opt/venv/bin/python3: No module named schemathesis.__main__; 'schemathesis' is a package and cannot be directly executed. Use the console script: st run ... (or schemathesis run ...). 2. st run openapi.yaml --base-url=https://... -> Error: No such option: --base-url (Possible options: --baseline, --url). Use --url (-u). It is required when the schema is a local file. 3. --hypothesis-max-examples=5 -> Did you mean --max-examples? 3.x options like --hypothesis-deadline, --endpoint and --method were also replaced.
Working 4.x command: st run openapi.yaml \ --url=https://v2-prod-api.nessieisreal.com \ --header="Authorization: Bearer $NESSIE_KEY" \ --checks=all --max-time=30 --rate-limit="2/s" --workers=1 \ --include-path="/customers" --include-path="/customers/{id}" \ --include-method=GET --include-method=POST Run st run --help in the installed version first. Phases are examples, coverage, fuzzing, stateful (--phases). --max-time bounds the run.
Expected findings on Nessie v2 (not bugs in your setup): - ST202 "Schema-valid request rejected" on POST /accounts and POST /bills: 422 problems/parent-required, "Create accounts under a customer: POST /customers/{id}/accounts." / "Create bills under an account: POST /accounts/{id}/bills.". The spec documents the flat routes, but the API wants the nested ones. - ST108 "Response violates schema" on POST /customers and POST /merchants: the API returns 201 but the spec only declares 200, so the 201 body is validated against the default ErrorModel.
Be careful: POST checks create real customers and merchants on your key (with junk names like "0"), and they count toward the 120 requests/minute bucket. Keep --rate-limit low and restrict methods/paths.