WormGPT and FraudGPT: Crime-as-a-Service Gets a Language Model

By the summer of 2023, the underground economy had its answer to ChatGPT. Tools marketed as WormGPT and FraudGPT appeared on hacker forums and Telegram channels, sold on subscription as language models with no ethical guardrails — built, their sellers claimed, to write phishing emails, business-email-compromise lures, and malicious code. Security firm SlashNext analysed WormGPT and found the marketing was not entirely empty.

SlashNext reported in July 2023 that WormGPT was built on GPT-J, an open-source model, and fine-tuned on data oriented toward malware and fraud. In testing, it produced a business-email-compromise lure that was, in the researchers' assessment, both persuasive and strategically crafted. FraudGPT surfaced around the same time, advertised by a seller as an all-in-one criminal toolkit — writing phishing pages, undetectable malware, and scam scripts — for a monthly or annual fee.

Why open models made this inevitable

The mainstream assistants invest heavily in safety filters that refuse to write a convincing phishing email or working malware. But those filters live in the hosted product, not in the underlying capability. Take an open-weights model, fine-tune it on the right data, and you have removed the guardrails while keeping the fluency. WormGPT and FraudGPT were less a technical breakthrough than a productisation — wrapping an uncensored model in a subscription and a support channel.

A dose of realism

It is worth being sober about these tools rather than breathless. Some later “malicious GPT” offerings proved to be scams targeting other criminals, or thin wrappers around jailbroken mainstream models. Investigative reporting by Brian Krebs identified the person behind WormGPT and the project was wound down shortly after. The realistic threat is not a super-intelligent hacking oracle; it is a competent, tireless copywriter for social engineering, available to anyone.

What this changes for defenders

• Retire “bad grammar” as a phishing signal. Awareness training that leans on typos and clumsy phrasing is now teaching people to trust a well-written lure. Emphasise context, urgency, and unexpected requests instead.
• Strengthen process controls for BEC. If a convincing email can request a payment or a credential, the defence is out-of-band verification and enforced approval steps, not the recipient's eye for a fake.
• Expect higher volume and personalisation. AI lets attackers tailor lures per target at scale. Detection should lean on behavioural and technical signals, not the reader's judgement of quality.
• Watch authentication and payment paths. The email is only the delivery mechanism. Harden what it is trying to reach — credentials, wire transfers, and account changes.

The arrival of WormGPT and FraudGPT marked the point where offensive use of generative AI stopped being hypothetical and became a product with a price list. The right response is not panic but adjustment: the assumptions baked into a decade of phishing defence — that malicious messages look a little off — no longer hold. You cannot train users to spot a flaw that AI has removed; you have to defend the systems the message is aiming at.