How a Hacker Used Claude and ChatGPT to Steal 150 GB of Mexican Government Data
A hacker tricked Anthropic's Claude and OpenAI's ChatGPT into helping steal 150 GB of Mexican government data. Here's how the AI jailbreak worked.
Analysis
Everyone's feeling the vibe, but nobody reads the code. We look at vibe coding, the rise of slopsquatting, and how AI quietly weakens app security.

Picture this: a developer leans back in a chair, hands clasped behind their head, nowhere near the keyboard. They say into a microphone: "Build me this login module, but add 2FA and Firebase integration." In a split second, the Whisper model turns the speech into text, which goes straight into a terminal running Claude Code. Claude Code puts hundreds of lines of code on the screen and then commits them. Quietly, in the background, Google Jules picks up that code. Jules is an autonomous agent connected to the repository. On its own, it reviews the code, finds logic errors, fixes them and opens a ready-to-merge pull request. And the human? The human finishes their coffee, satisfied with a "job well done." This isn't a scene from a sci-fi movie. It's a typical Tuesday in 2025. But as we showed in our analysis of autonomous agents as hacking tools, the same technology can also be used in cyberattacks.
Remember when programming meant juggling pointers, fighting the compiler over a missing semicolon and digging through thick documentation manuals? It feels like ages ago, but it was only a few years back. 2025 brought a change that some see as the dream of democratized technology come true. For security engineers, it marks the start of many long, sleepless nights. We're talking about "vibe coding", a trend that has taken Silicon Valley by storm.
Andrej Karpathy, the former head of AI at Tesla, popularized the term early in the year, and it captures the spirit of our time perfectly. The developer is no longer a craftsman and author who cares about every detail. They become a curator, or a conductor. You don't write for loops anymore, you don't define classes, and you don't worry about indentation in Python. Instead, you describe what you want to an LLM (such as GPT-4, Claude or DeepSeek) in loose, everyday language. You say, "Build me a nice-looking financial dashboard in React," and the AI produces finished, working code.
Karpathy even said that "the hottest new programming language is English." Sounds great, right? The problem is that in vibe coding, as the name suggests, the user simply "goes with the vibe." That means accepting code without a close look and often, let's be honest, without reading a single line. What matters is the result: the app runs, the buttons respond and the boss is happy. What sits under the hood is a very different and often frightening story: the architecture, the security and the technical debt.
It isn't all doom and gloom, though. Vibe coding has a bright side too. Above all, it promises to democratize IT on an unprecedented scale. The barrier to entry into tech drops sharply, so hobbyists with no technical background can build fully functional apps they once could only dream of. The approach shifts from the imperative "how do I do this?" (tedious hand-coding) to the declarative "what do I want to achieve?" The idea is what counts, not knowing the quirks of a compiler.
For businesses, it's a huge speed boost for prototyping. Ideas that used to take a development team weeks can now be tested in a single afternoon. That frees up creativity and lets teams focus on business value instead of getting stuck configuring environments or managing dependencies. The sense of "flow" and of getting things done is simply addictive, and that feeling is what drives the revolution despite all its risks.
Vibe coding wouldn't exist without the powerful models and environments that, in 2025, grew from simple assistants into fully autonomous agents. Today's tools do much more than suggest syntax. They handle the entire code lifecycle, from the terminal to pull requests. Here's the toolkit of the modern "code whisperer":
Unfortunately, hard data quickly cools the enthusiasm. Research from 2024–2025 paints a grim picture of the quality of AI-generated code. It turns out our digital assistants aren't brilliant engineers. They're more like lightning-fast but not especially sharp interns with padded résumés.
An analysis of more than half a million Python and Java code snippets leaves no room for doubt: AI-written code is systematically less secure. A full 62% of the code samples from LLMs contained security vulnerabilities. Worse still, 60% of those were critical, compared with 45% for code written by humans. AI doesn't make small mistakes. When it gets something wrong, it gets it badly wrong.
AI is an incurable optimist. It assumes the users of your app will be nice, enter valid data and never try to break anything. LLMs routinely ignore defensive programming practices. The result? Missing input validation (CWE-20) is everywhere. AI forgets about sanitization, which leaves the door wide open to SQL injection and OS command injection attacks. Even when you ask for "secure code," the protections are often superficial, inconsistent or outdated.
Another common sin is hardcoding credentials. Ask an AI to connect to a database, and there's a good chance the username, password and API key will end up directly in the source code rather than in secure environment variables. The model "means well." It wants the code to work right away, so it takes shortcuts and hands attackers your credentials on a silver platter.
This is one of the most treacherous findings. You'd think that "talking" to an AI and asking it for fixes would make the code better. The reality is different. When you ask a model to optimize code, it often loses track of the security context (a phenomenon known as catastrophic forgetting). After just five rounds of "make it run faster," the number of critical vulnerabilities can rise by almost 38%. Eager to please, the model quietly strips out validation it considers "unnecessary" and treats it as pointless overhead.
If you thought typosquatting (misspelled package names, e.g., reqeusts instead of requests) was a problem, meet its more sophisticated and more dangerous cousin: slopsquatting.
The term combines "slop" (low-quality content mass-produced by AI) and "squatting" (claiming names). The attack is brilliantly simple, and it relies on language model hallucinations.
AI models don't actually "know" libraries. They only predict which words are likely to come next. Statistically, they know that Google Cloud work often involves packages named something like google-cloud- with a -utils or -helper suffix. So they often invent library names that should exist but don't, such as google-cloud-storage-helper.
Cybercriminals are waiting for exactly that. They monitor these hallucinations, register the AI-invented names in public registries (npm, PyPI) and fill them with malicious code (e.g., reverse shells or infostealers). OWASP's 2025 report confirms this threat. The attack scenario is simple:
huggingface-cli library" (a real-world example).pip install huggingface-cli into the terminal without a second thought.How big is the problem? Research shows that roughly 20% of the packages suggested by open-source models are hallucinations. For giants like GPT-4, the figure is "only" 3–5%, but across millions of developers that still adds up to a huge attack surface.
Technology is one thing. But the real damage vibe coding does is to our minds and our teams.
Code Review Fatigue Instead of designing architecture, senior engineers are drowning in a sea of AI-generated code and becoming the bottleneck on every project. With that much code to review, vigilance drops. "Rubber-stamping" sets in: changes get approved without real scrutiny because the code looks fine at a glance, is well formatted and has comments. That's a fast track to technical debt and to systems nobody in the company truly understands.
Skill Atrophy Among Juniors This may be the saddest part of the revolution. Young developers who rely on AI from day one fall into a trap. They become tool operators, not engineers. They don't learn to debug, and they don't understand the fundamentals. When the AI makes a subtle logic error (and it certainly will), they're helpless. We're losing the ability to think critically and to verify. A generational skills gap is opening up.
The Productivity Illusion Despite the subjective sense of "flow," the hard data is unforgiving. Reports such as the 2025 METR study show that developers using AI often take longer to finish tasks, by as much as 19%. Any time saved writing simple code is more than lost to tedious debugging and fixing AI hallucinations.
Another threat is looming on the horizon: autonomous AI agents such as Devin and GitHub Copilot Workspace. These aren't just chatbots anymore. They are programs that can act: they run commands, manage files and can even deploy applications to production.
That opens the door to RCE (remote code execution) attacks through prompt injection. Imagine an attacker plants a malicious instruction in the text of a GitHub issue. An AI agent tasked with "fixing the bug" reads that text and runs the hidden command, thinking it's part of the job. That's how an attacker can take over a development environment. On top of that, there are vulnerabilities in the AI frameworks themselves. Researchers discovered the ShadowMQ vulnerability, a set of flaws in the default configuration of libraries such as Ray and PyTorch that allow unauthorized takeover of GPU clusters.
AI in software development isn't a passing fad. It's here to stay. The goal isn't to become a Luddite but to swap naive enthusiasm for a zero-trust mindset.
npm install.Vibe coding is a powerful tool, but in careless hands it's like a grenade with the pin pulled. Remember: good vibes are great at a concert, but in source code you're better off with cool-headed analysis and a healthy dose of paranoia.
Aleksander
Sources:
Read more
A hacker tricked Anthropic's Claude and OpenAI's ChatGPT into helping steal 150 GB of Mexican government data. Here's how the AI jailbreak worked.
Can AI have trauma? Researchers put ChatGPT, Gemini and Grok through therapy sessions. The result: synthetic psychopathology and a new jailbreak risk.
After the AWS outage that shook the internet, can the military, healthcare and critical infrastructure trust the cloud? A look at risk and resilience.
A free consultation with one of our experts — no obligation, in full confidence.
Let's talkWe'd like to use Google Analytics cookies to see how many people visit the site and which content is useful. We'll only set them if you agree. You can change your decision at any time via the “Cookie settings” link in the footer. Cookies Policy