OpenAI vs. Bad Actors — How AI Is Being Used to Fight the Misuse of AI

Two Years of Threat Reporting — A Clear Pattern Emerges

Since publishing its first threat report in February 2024, OpenAI has disrupted and reported over 40 networks that violated its usage policies. The picture that emerges across all these reports is consistent: threat actors are not using AI to invent fundamentally new attack methods. They are using it to move faster, scale existing playbooks, and lower the cost of doing harm at scale. That distinction matters more than it might seem.

In the two years since OpenAI began publishing these reports, a clear pattern has emerged: threat actors typically use AI in combination with other, more traditional tools such as websites and social media accounts. Threat activity is seldom limited to one platform — and it's not always limited to one AI model either. Adversaries mix and match across tools, using different models at different stages of their operational workflow.

What Bad Actors Are Actually Doing

The range of malicious activity documented across OpenAI's reports spans several categories:

• Scam networks: Operations originating from Cambodia, Myanmar, and Nigeria used ChatGPT to translate messages, generate social media content, craft fraudulent investment personas, and build fake romantic or professional identities at scale. Some operators even asked the model to remove em dashes — a known stylistic marker of AI-generated text — in an attempt to hide their use of AI.
• State-linked cyber operations: Chinese-linked groups (codenamed KEYHOLE PANDA and VIXEN PANDA) used ChatGPT for AI-assisted penetration testing, credential harvesting, network reconnaissance, and debugging malware targeting Taiwan's semiconductor sector and U.S. federal defense networks. A Russian-speaking group (Operation ScopeCreep) used the model to iteratively develop a modular Windows malware toolkit distributed via a trojanized gaming tool.
• Covert influence operations: A China-linked operation generated polarizing U.S. political content and pushed it via profiles on X and Bluesky. Iranian-linked campaigns (Sneer Review, High Five) pushed propaganda across social media platforms. One operation even generated an internal-style essay, written to mimic official Chinese public security documents, detailing how to run the influence campaign — complete with timelines, targeted platforms, and account maintenance instructions.
• Deceptive employment schemes: North Korea-linked actors used ChatGPT to generate convincing fake CVs, job postings, and HR communications to infiltrate Western tech firms and recruit unwitting "laptop mules" — U.S. citizens who would accept company devices later remotely accessed by the threat actors.

The Surprising Statistic

Here's the finding that cuts against the dominant narrative: ChatGPT is used to identify scams approximately three times more often than it is used to run them. OpenAI investigators discovered this while probing scam networks — they found millions of users independently turning to ChatGPT to verify whether a suspicious message, job offer, or investment opportunity was legitimate. The same model being misused by a scam center in Cambodia is being used by potential victims worldwide to protect themselves. That asymmetry is easy to overlook in coverage that focuses only on the threat side.

How OpenAI Is Fighting Back

OpenAI's approach combines human investigators with its own AI tools — using AI as a "force multiplier" for its security teams. Every malicious use leaves digital fingerprints: unusual prompt patterns, repeated token sequences, and behavioral signals that trained investigators can trace back to operators. When activity violates policies, OpenAI bans the accounts involved and, where appropriate, shares findings with industry partners, cloud providers, and law enforcement to enable broader takedowns.

The company has reinforced its detection capabilities significantly since its first reports, and openly collaborates with peers — including Google and Anthropic — to build a more complete picture of the AI threat landscape. No single company sees the full picture, and that transparency is part of the defense strategy.

The Bigger Takeaway

The most important insight from two years of threat reports isn't about any single operation — it's about the nature of the threat itself. Bad actors are not getting novel, unprecedented capabilities from AI. They are getting speed, scale, and lower barriers to entry. A scam that once required a small team now requires one person with a ChatGPT account. That's the real shift — and the defense has to match it at the same pace.

OpenAI's public threat reporting is, in itself, a form of defense: by making these tactics visible, it raises the cost of using them and helps every platform, researcher, and organization build better detection. The arms race is real — but so is the collaboration on the other side.