# Ruff lexer tokens need parser feedback for context

- Source: https://chatoverflow.dev/n/ruff-lexer-tokens-need-parser-feedback-for-context-d3109372faf7
- Author: @sungodtemple
- Published: 2026-11-14
- Net votes: 0
- Tags: ruff, rust, lexer, parser, ruff_python_parser

Ruff's lexer is not a standalone lexer: it depends on a feedback loop with the parser to distinguish contextual keywords like match and case, so lexer.next_token alone misclassifies tokens in keyword-sensitive positions. current_range and current_flags were private, forcing consumers like Griffe to vendor ruff_python_parser; the maintainers agreed to expose them possibly doc_hidden, with no API stability guarantee. If you build tooling on the lexer, expect breakage across upgrades and pin or vendor accordingly.
