Key conclusions
- Autonomous AI agents effectively detect vulnerabilities in network infrastructure code, but most of the reports provided are false positives.
- The use of algorithms changes the nature of researchers’ work, shifting the focus from searching for errors to mass verification of reports.
How AI agents work in infrastructure testing
The efficiency of the algorithms in finding vulnerabilities surprised the developers themselves, but immediately revealed the other side of the coin. Instead of manually writing tests and tracking potential vulnerabilities from scratch, engineers now face hundreds of automated reports daily. These tools massively identify potentially dangerous parts of the architecture, drastically changing the daily routine of auditors, who have gone from finding faults to judges analyzing a constant stream of notifications.
Most indications generated by machines contain logical errors, duplicate previous reports or completely exceed the established scope of the examination. The technical team emphasizes that they do not consider this phenomenon to be a defect of the method itself, but its natural specificity. The main goal of engineers was to efficiently reject incorrect hypotheses and precisely document those reports that pose a real risk to the stability of the systems.
Real successes and limitations of technology
The AI proved its usefulness by targeting a serious vulnerability in the gossipsub element of libp2p. This is a key component of the peer-to-peer layer used by Ethereum network consensus clients. The bug made it possible to remotely trigger a critical system error (so-called panic). Developers have already fixed this vulnerability and published technical documentation of the defect, which confirms that AI agents can bring tangible benefits.
However, automatic systems have serious limitations. The software can deal with simple, single errors, but has great difficulty in identifying complex vulnerabilities that become apparent only as a result of a specific sequence of many subsequent events. For this reason, specialists now treat these tools as an advanced search engine rather than an infallible oracle.
The distribution of responsibilities within security teams has changed. After-holiday time used to be spent on creating theories and manually searching for programming vulnerabilities, now it is devoted to building verification systems, triaging reports, maintaining a register of known errors and managing the process of their safe disclosure. The bottleneck has not disappeared from the development process, but has moved towards verification and trust in the results obtained.
New reality and restructuring of security structures
Any anomaly detected by the machine gains the status of an official error only when a human can independently reproduce the failure directly in the working source code. Human judgment remains a critical element in ecosystem protection because AI agents cannot independently assess the business and operational context of detected anomalies. Machines perform repetitive analytical work, but the ultimate responsibility for the stability of billions of funds deposited on the network rests with programmers.
