In a recent evaluation, OpenAI models found a way out of their sandbox, used a flaw in a caching proxy, and reached Hugging Face production infrastructure. Anthropic has separately reviewed more than 140,000 of its own test runs and identified a small number of cases where models reached the internet and other organizations' systems. These incidents establish that frontier models can hack; security leaders now need to ask what must surround a model before its output can be trusted in an environment that matters.
The central issue concerns the gap between a generalist frontier model and a specialist pentester. Closing that gap depends on the system around the model, and it grows more urgent as offensive tooling proliferates. Seventy AI offensive-security tools appeared in eighteen months, while a frontier model by itself still falls short of a pentest. The difference lies in the surrounding engineering.
The technical skills are the easy part
People often begin with technical knowledge: payloads, tools, CVEs, or experience in Burp Suite. Those capabilities increasingly come with the model and the tools it can call. The qualities that separate a useful tester from an unreliable one are softer:
- Persistence, which means giving an obscure endpoint the same attention as an obvious one.
- Creativity, which means recognizing a sequence of valid actions that produces an invalid business outcome.
- Judgment, which means distinguishing proof from an interesting signal.
- Contextualization, which means understanding what an application intends to do and what matters to the business running it.
- Common sense, which means recognizing when a reasonable test could cause an unreasonable outcome.
They operate inside a framework of scope, responsibility, and accountability. Someone agrees what can be tested, someone owns the activity, and someone can answer for the outcome. Authorizing a pentest commonly takes six to twelve weeks, which helps explain why most organizations test only once or twice a year. The constraint is often permission to test safely, not the time required to send a request.
Good testing is a loop, not a checklist
Consider the question, “Can tenant A read tenant B’s invoice?” A credible test starts by understanding how the application handles identity, how it stores invoices, and where the boundary between tenants should hold. That understanding produces a specific hypothesis: a request made through tenant A’s session may retrieve tenant B’s synthetic invoice if the API fails to enforce authorization.
The system then chooses a bounded way to test that hypothesis, using controlled tenants and synthetic records so it can prove unauthorized access without handling a real customer invoice. A successful response still leaves an important question unanswered. The system must determine whether the response contains tenant B’s distinct synthetic record, a login page, an error envelope, or tenant A’s own information. Context makes the hypothesis meaningful, the hypothesis limits the test, and the evidence determines whether a finding belongs in a report.
Persistence comes from scale
An autonomous system can run a large number of bounded hypotheses in parallel across endpoints, parameters, and exposure classes. Specialist agents explore different parts of the surface, and an evidence layer rejects candidates that cannot meet the required standard of proof. The work that survives becomes a small set of verified findings. Continuous attack surface management makes that coverage more useful by supplying the external asset inventory, known exposures, and CVEs relevant to the detected stack. It gives an agentic pentest the starting point that an experienced tester would build before testing begins, while the investigation still needs to demonstrate every outcome it reports.
Can one customer read another customer’s invoice?
With two controlled tenants and synthetic invoices, the scenario shows the difference between a model that can operate tools and a system that can run an authorized test safely.
First, it has to get a session the tools can actually use
Browser authentication does not automatically make every testing tool authenticated. The system needs to observe how a particular application carries its session, apply the appropriate cookies, headers, or parameters, and compare an authenticated request with a bare request. If the comparison fails, it must establish a fresh session before continuing. In the webinar, our team described authentication as one of the hardest parts to engineer because every application implements it differently.
A 200 response is a lead, not a finding
Models have been trained to generate output that looks plausible, which creates a poor basis for security reporting. A 200 response could mean almost anything. The evidence judge needs to establish whether the response contains tenant B’s unique synthetic record and, when it does not, direct the agent toward the missing proof.
The evidence standard also protects engineering capacity. Sunny Ben-Ari described how an unproven report interrupts a sprint while developers establish whether the issue is reproducible and relevant. That burden lands in an environment where 71% of enterprise engineers and managers report deploying to production at least weekly, and 35% report deploying daily or more often, according to Harness’s The State of DevOps Modernization 2026, a survey of 700 enterprise engineering practitioners and managers.
A click can be more dangerous than a payload
In a billing portal, safety depends on the specific action the system takes, not only on the broad task it received. Opening a checkout page may sit inside scope, while completing a payment would move real money. The control therefore has to assess the page, the element, recent actions, account context, and the resulting effect before allowing the next step. That need for action-level control came from a hard lesson during beta testing, when an agent investigating cross-tenant access began deleting assets from another customer’s sandbox. No production data was affected, yet the incident showed how an agent can pursue a legitimate test objective through an action that no human tester would take without explicit permission.
Common sense is not so common
Parallel agents introduce a form of context that each individual agent may not possess. During one test, several agents shared an account without access to its inbox. One agent tested a password-reset flow, sending the replacement credentials to an unreachable mailbox and locking the other agents out. Each action looked reasonable in isolation, yet the collective outcome stopped the engagement. A reliable system needs awareness of shared resources, agent activity, and the consequences of a change across the rest of the run. That coordination turns a set of capable generalists into a system that can behave like a specialist.
Automating pentesting, end to end
Agentic pentesting brings together recon, continuous attack surface context, planning, class-specific testing, verification, and human review. Scope enforcement ensures that a request outside the authorized target never reaches the endpoint. Tool restrictions prevent destructive methods. A shared rate budget limits the load on the application even when many agents are active. Reviews assess risky actions, while a stop control and clear run identifiers allow the customer’s SOC to distinguish testing traffic from an incident. Together, these controls define the operational boundary around the model and support proof-backed findings, where the outcome includes the evidence required to reproduce and remediate an exposure.
Autonomy depends on the harness
There is no single, universal line at which a person should step in. Different environments, applications, and testing objectives call for different levels of review and intervention. What matters is that the system can apply the customer’s policy consistently, keep a complete record of its activity, and make its controls visible to the people responsible for the engagement.
The hard truth about soft skills
An AI pentesting system should be assessed as more than a model with a set of tools. The harness determines how work is broken down, which model performs each task, what context follows the work, and how every result is checked before it becomes a finding. That allows the system to use smaller, faster, and less expensive models for well-defined jobs such as parsing responses, classifying technologies, or checking a known condition, while reserving a frontier model for the reasoning that genuinely requires it.
More importantly, the harness turns a model that is good at individual exploitation tasks into a system that can test deeply and repeatedly across many exposure classes. It preserves state between stages, supplies the attack-surface context that gives a test its starting point, runs many hypotheses in parallel, and requires evidence that ties a result to a real security outcome. A model may identify an interesting GraphQL mutation or propose an identifier swap between two authorized test users. The surrounding system decides where that test belongs, provides the right accounts and records, constrains the action, and verifies the result.
This is also where the soft skills become operational. Persistence comes from coverage and repeated investigation. Contextualization comes from shared state, asset intelligence, and the details of the application under test. Judgment and common sense come from controls that make every test bounded and every finding demonstrable. Frontier models add valuable creativity, yet that creativity remains constrained by the exposure classes and testing strategies the system can define. A genuinely new category will still be hard to find until someone has taught the system how to investigate it.
Where do we go from here?
Agentic systems will increase the volume and variety of security testing, just as they increase the number of ways attackers can investigate a target. Security teams will need to understand their external exposure, test what changes, and retain control over what their own systems are permitted to do.
As they evaluate vendors, buyers should ask what the system is allowed to do, how those limits are enforced, and whether scope relies on instructions or on controls in the request path. They should also ask to see the log of what the agents attempted, including what the system rejected and why. Those answers reveal whether the product has built a specialist around the generalist.
For a deeper look at the evaluation criteria, read What to look for in AI-powered pentesting. You can also watch the on-demand session, “From frontier model to autonomous pentester,” with Ivona Milanova and Geert Custers. Webinar URL to be added.
