Check it out at our website advtok.github.io to access all the details! We already released the paper, code (wrapped in a package), and the blog.

We’d love to hear your thoughts!

@guyvdb.bsky.social @renatogeh.bsky.social

What happens if we tokenize cat as [ca, t] rather than [cat]?

LLMs are trained on just one tokenization per word, but they still understand alternative tokenizations. We show that this can be exploited to bypass safety filters without changing the text itself.

#AI #LLMs #tokenization #alignment