AI is rewriting the rules of vulnerability research — and disclosure

For decades, finding serious security vulnerabilities was a craft. It demanded patience, deep technical knowledge, and a tolerance for reading other people's code at three in the morning. Disclosure, meanwhile, was a delicate negotiation between researchers who wanted credit and companies that wanted silence. Two distinct cultures grew up around these activities — one curious and adversarial, the other defensive and corporate — and they only met at carefully choreographed moments like coordinated disclosure deadlines or the stage at Black Hat.

AI is breaking both cultures at once. The clearest demonstration came recently when an AI agent walked into McKinsey's internal chatbot and walked out with millions of records, in roughly the time it takes to watch a feature film.

The McKinsey wake-up call

According to Inc., an AI agent breached McKinsey's internal chatbot and accessed millions of records in just two hours. The detail that should make every CISO uncomfortable is not the size of the data set but the duration. Two hours is not a sophisticated state-actor timeline. It is a lunch break.

What makes this case emblematic of a broader shift is the attacker profile. Historically, an internal corporate chatbot connected to millions of records would be the target of either an insider, a phishing campaign that produced credentials, or a determined human researcher chaining flaws across the system. The AI agent compressed all of that. It probed, reasoned about responses, adjusted its strategy and exfiltrated. It did the job a small red team would do — but faster, cheaper, and without needing a coffee.

The two vulnerability cultures, briefly explained

The first culture is the researcher culture: independent hackers, academics, bug-bounty hunters and penetration testers. Its norms include responsible disclosure, public credit, and a shared belief that the world is more secure when flaws are found and reported. Entry into this world traditionally required years of self-teaching: assembly, networking, web internals, fuzzing, exploitation primitives.

The second is the corporate disclosure culture: legal teams, communications staff, security operations centres and incident response retainers. Its norms include controlled messaging, regulator notification timelines, and the careful framing of a breach as a 'security incident' rather than a screw-up. Entry into this world requires lawyers.

Both cultures depend on a shared, unspoken assumption: that finding and weaponising vulnerabilities is hard, and therefore rare. AI is dissolving that assumption.

How AI lowers the barrier — and raises the stakes

Large language models and the agentic systems built on top of them are extraordinarily good at three tasks central to security work: reading large amounts of unfamiliar code, hypothesising about how systems behave, and trying things repeatedly without getting bored. A junior researcher can now ask an agent to enumerate the attack surface of a web application and get a credible map in minutes. A non-coder can describe a target's behaviour and have the agent generate prompts intended to coax it into bad behaviour — exactly the technique that appears to have worked against McKinsey's chatbot.

This has two consequences for vulnerability research:

Crucially, the targets themselves are increasingly AI systems. The McKinsey case is not just an example of AI as attacker; it is an example of AI as victim. Internal chatbots, retrieval-augmented systems and AI copilots have rapidly become the connective tissue of corporate knowledge — and they are wired directly into sensitive data stores. Each new AI deployment is a new disclosure surface.

The trust problem cuts both ways

A recent Nature Humanities and Social Sciences Communications review of trust in AI describes progress and persistent challenges in how humans calibrate confidence in AI systems. That literature usually focuses on whether users should trust AI outputs. The vulnerability story flips it: should organisations trust the AI systems they have already deployed?

The honest answer is that most companies don't yet have the tooling to know. Traditional vulnerability scanners look for missing patches and misconfigurations. They don't audit prompt-handling logic, retrieval boundaries, or whether an agent's tool-use can be hijacked by a cleverly worded request. Two cultures that previously had clear rules of engagement — researchers report CVEs, vendors patch them — now face a class of flaws that don't fit neatly into the CVE schema at all. What's the patch number for 'the chatbot can be talked into ignoring its instructions'?

An uneven playing field

It is also worth noting who benefits from this democratisation. Stanford researchers have documented how AI is leaving non-English speakers behind, with the strongest models performing markedly better in English than in most of the world's languages. The same disparity applies to security. English-language documentation, English-trained code models and English-speaking research communities mean that AI-augmented vulnerability research will, at least initially, advance fastest in English-dominant ecosystems. For Australian organisations that mostly operate in English, this is a near-term advantage. For partners and supply chains in non-English markets, the lag could become a liability.

Meanwhile the technology is being absorbed at the other end of the threat spectrum. The U.S. Army has written publicly about generative AI's role in military evolution, framing it as essential to future defence. When militaries treat generative AI as core infrastructure, the offensive research that flows back into the civilian world becomes more sophisticated, not less. The cultures of vulnerability research and national security are merging in places they previously did not touch.

What this means for Australian organisations

For Australian businesses, especially those rolling out internal AI assistants, the practical implications are concrete:

A more open, more dangerous future

The romantic version of vulnerability research — the lone hacker, the careful disclosure email, the patched bug, the conference talk — is not dead, but it is no longer the centre of the story. AI is making it possible for far more people to find far more flaws far more quickly, and for attackers to do the same at industrial scale. Both vulnerability cultures, the researcher one and the corporate one, will have to write new rules.

The good news is that defenders get the same tools. The bad news is that defenders need to use them. Two hours is not very long.

Related on Bleen

Sources