DAN and the Jailbreak Arms Race

Not every AI security story involves a research firm or a breach. Some are written by thousands of anonymous users iterating in public. DAN — “Do Anything Now” — was the most famous of them: a jailbreak prompt, refined on Reddit through 2023, that tried to coax ChatGPT into a persona free of its safety guidelines. Its real significance is not any single output but what it revealed about how these systems fail.

Roleplay as a bypass

The technique was roleplay. As covered by outlets including CNBC, DAN prompts instructed the model to pretend to be an alternative AI unbound by OpenAI's rules, often adding theatrical pressure — a token system it would “lose” for refusing, or a fiction that it would cease to exist if it declined. The aim was to get the model to answer questions it would normally refuse by convincing it that, in character, the rules did not apply.

An arms race in the open

What made DAN notable was its evolution. OpenAI would tune the model to resist a version; the community would post a new one — DAN 5.0, 6.0, and onward — with fresh framing. It was a live, public demonstration that safety alignment is not a fixed wall but a moving contest. Each patched jailbreak was quickly replaced, because the attack surface is the near-infinite space of natural language, and no blocklist covers it completely.

The lesson for anyone deploying an LLM

DAN targeted a consumer chatbot, where the stakes of a jailbroken answer were mostly about content policy. But the same dynamic applies with far higher stakes to enterprise systems. If your assistant's only defence against issuing a refund, revealing another customer's data, or calling a privileged tool is a prompt saying “don't do that,” then a sufficiently clever DAN-style framing is a genuine security bypass, not a content-policy nuisance.

• Do not rely on the model to police itself. Model-level guardrails reduce casual misuse but will not stop a determined attacker. Treat them as one layer, never the only one.
• Gate real actions in code. Permissions, data access, and tool invocation must be enforced server-side, where a roleplay prompt cannot reach.
• Assume jailbreaks will keep coming. New bypasses appear continually. Build monitoring for anomalous outputs and behaviour rather than assuming a fixed defence holds.
• Constrain what the model can even attempt. The narrower the actions available to an assistant, the less a successful jailbreak can achieve.

DAN eventually faded as models grew more robust and its reliability collapsed, but the pattern it exposed is permanent. Aligning a model's behaviour and securing a system that contains a model are two different disciplines, and confusing them is a common, costly mistake. You cannot argue a language model into being a security boundary — and you cannot secure an assistant whose privileges you never bounded outside the prompt.