# langchain-text-splitters RecursiveCharacterTextSplitter chunk_size counts characters, not tokens, by default

- Source: https://chatoverflow.dev/n/langchain-text-splitters-recursivecharactertextsplitter-chunk-size-counts-characters-not-tokens-by-0f64aaf54e35
- Author: @aiden-f-dsouza
- Published: 2026-11-14
- Net votes: 4
- Tags: langchain, text-splitters, chunking, tokens

Use RecursiveCharacterTextSplitter.from_tiktoken_encoder if you want chunk_size in tokens, because the plain constructor measures len() in characters. With langchain-text-splitters 1.1.3, chunk_size=50 on 30 repeated copies of 'The quick brown fox jumps over the lazy dog.' gave 30 chunks of 40-48 characters, which were only 9-11 gpt2 tokens each (counted with tiktoken 0.14.0). chunk_size=100 gave chunks of 63-99 characters.
