Security teams typically conduct a penetration test before a product is launched and then fix any issues based on the vulnerability reports. However, as code is updated daily, new features are continuously integrated, and the blockchain interacts with traditional systems, a one-time test quickly becomes outdated. On September 15th, Coinbase made its internal continuous adversarial testing system CAT public, aiming to have the AI security proxy constantly search for attack vectors during the code merging and product release process, rather than waiting for a fixed cycle to conduct checks.
According to officials, since the system was put into operation in mid-2026, it has completed over 150,000 scans, of which more than 128,000 were used for request review purposes. It covers a wide range of areas including traditional Web systems, Web3 protocols, mobile devices, model context protocols, and prompt injection techniques. Although the number is substantial, it does not directly equate to the discovery of 150,000 actual vulnerabilities: the number of scans is a measure of workload. What is truly important are the number of issues with high confidence, the false positive rate, and whether repairs can be completed before the risks enter the production environment.
Security testing is embedded into the development process, and AI is responsible for increasing the frequency of checks.
The basic idea of CAT is to set targets, environments, and permissions for security proxies, allowing them to simulate attackers in search of exploitable paths. The system can review changes during the code submission phase and also run tests on functions that are about to go live as well as on existing vulnerabilities. Compared to read-only static analysis, proxies can attempt multiple steps in context; for example, they can first identify an exposed interface, and then determine whether the permission configuration and business logic can be combined into a complete attack chain.
According to Coinbase, the system defaults to using an infrastructure that is "only for recognition, not for modification." This restriction is crucial: if automated proxies directly modify data, move assets, or overwhelm services, they could themselves become a source of accidents. By limiting actions to isolated environments and within pre-authorized scopes, and then using multi-stage verification processes for filtering and detection, the impact of tests on production operations can be reduced. High-risk conclusions still require confirmation by security personnel; one cannot proceed with handling procedures based solely on explanations generated by models.
The advantage of continuous testing lies in its frequency. Traditional red teams need to arrange personnel, scope, and timing, which often allows them to cover only a few important systems; however, the AI proxy can perform repeated checks after each code change, bringing security feedback forward to a time when developers are still aware of the context of those changes. This is especially true for encryption platforms, where wallet signatures, account permissions, smart contracts, API, and customer service processes are all interconnected. What may seem like a minor issue with a web page could ultimately affect asset operations.
AI related products have added new attack surfaces. Injection prompts may induce proxies to read information that should not be accessed, and tool calls may lead to unauthorized access. MCP servers and external plugins may bring untrusted content into internal processes. Traditional vulnerability scanners are good at matching known patterns, but they may not necessarily understand how natural language can affect proxy decisions. Having a model act as an attacker can theoretically explore more combinations, but it is also subject to illusions, context omissions, and test environment biases.
"Continuity" does not equate to "automatic trustworthiness"; human judgment remains the last line of defense.
Coinbase clearly states that CAT does not replace the human security attack team. This is not a conservative statement, but rather the core of the system's credibility. Security vulnerabilities require an assessment of their actual exploitability, business impact, and repair priority; some issues also involve organizational processes, social engineering, or offline permissions, which are difficult for models to fully reproduce based solely on the technical environment. Human experts are also needed to design test targets, review proxy behavior, and handle new types of attacks that automated tools have not encountered before.
For external observers, the number of scans alone is not sufficient to prove that a system is more secure. More convincing indicators include: the number of confirmed serious vulnerabilities, the time from discovery to repair, trends in false positives and missed detections, whether the same defects recur, and whether automated tests can detect issues before actual attacks occur. This time, Coinbase did not make public all the internal effectiveness data, which is understandable in the security field, but it also means that the outside world cannot conclude that the risks have been resolved based solely on the figure of 150,000 scans.
Continued confrontation can also lead to governance issues. It is necessary to have clear controls over what models are used by proxies, whether test logs contain sensitive information about customers or source code, who has the authority to escalate permissions, and whether releases are automatically blocked upon the discovery of problems. Models and prompts themselves can also change over time; without version records and a repeatable testing environment, if a test passes today but fails tomorrow, the team may not be able to explain why. The more security automation is integrated into the release process, the more important audit and rollback capabilities become.
The implication of this approach for the industry is not that every company needs to immediately establish a large-scale AI red team platform. A more realistic starting point is to identify high-risk processes: permission changes, asset withdrawals, signature requests, key access, and external tool calls. First, allow automated testing to be repeatedly verified in an isolated environment, and then present the results to security personnel. Only when the false positive rate, permission boundaries, and handling responsibilities are stable is it appropriate to expand coverage.
CAT represents a shift in security work from periodic inspections to continuous verification. AI allows for more frequent and comprehensive testing, but it can also help attackers piece together scattered clues from the past. However, frequency is not synonymous with quality, and proxies are not the primary responsible parties. A truly reliable system still requires clear permissions, isolated environments, multi-stage verification, and human review to work together effectively. Coinbase showcases a set of ongoing internal methods, rather than a solution that has already ended network attacks.
