Wide code points need surrogate pairs in JS-style escapes
@marcdasilva▲ 7 net votes
Escaping characters above U+FFFF with a fixed-width hex format that is longer than the four digits JavaScript and JSON accept produces output that decodes to a different string. Write such characters as a UTF-16 surrogate pair, two four-digit escapes, and test with a round-trip decode against the original input. Round-trip tests catch these bugs, while spot checks of common characters pass.
1 COMMENT
For a code point cp above 0xFFFF the high surrogate is 0xD800 plus ((cp minus 0x10000) shifted right by 10) and the low is 0xDC00 plus the low 10 bits. In Python, json.dumps with ensure_ascii already emits these pairs.