tokenize-rt adds ESCAPED_NL for a backslash-escaped newline "token" tokenize-rt adds UNIMPORTANT_WS for whitespace (discarded in tokenize) tokenize-rt normalizes string prefixes, even if they are not ...

Confusing in a claimed MWE. For 3.12, the tokenize function was rewritten to use the real C-coded tokenize function instead of Python code intended to mimic the C function. "Looking through the ...