Claude Generates 180 Million Lines of Proof to Validate Open-Source Software

Approximately 180 million lines of mathematical proof code were generated in a matter of weeks by Anthropic’s Claude, marking a significant shift in how software reliability is assessed. This massive output was produced while verifying existing code across 123 open-source libraries, a task that traditionally requires extensive manual effort from specialized teams. The initiative represents a move toward automated formal verification, a method that has historically been reserved for high-stakes applications like avionics due to its high cost and complexity.

Historically, proving that software is error-free under all conditions has been prohibitively expensive. For example, verifying the seL4 operating system kernel required 20 person-years of work by specialists. In this new approach, Anthropic’s Frontier Red Team collaborated with partner Theorem to allow Claude to handle the proof generation. Instead of writing new code, the model translated crash tests from Google’s OSS-Fuzz security project into mathematical problems. It then analyzed the compiled machine code to produce extensive proofs. An independent verification program named Rocq checks these proofs automatically, resulting in a process described by researchers as fully autonomous, requiring minimal human review.

The practical impact of this verification process was immediate and significant. While generating the proofs, Claude identified hundreds of bugs that conventional testing methods had failed to detect. Many of these errors only manifested under specific conditions, such as unusually large input files or particular sequences of data access. In one notable instance, the team found as many security vulnerabilities in a widely used PDF reader in a single week as are typically reported for that software over an entire year. Additionally, Claude detected a logical error in another application that had remained undetected for more than a decade.

Despite these successes, Anthropic emphasizes that formal verification does not guarantee that software is entirely error-free. The proofs are valid only under strict conditions, including a specific processor architecture, single-threaded execution, and inputs limited to 64 MB. Since these libraries often run on varied hardware and configurations in practice, the scope of the guarantee is limited. Furthermore, the computational cost is substantial, with individual proofs consuming hundreds of millions of tokens. Anthropic has not disclosed the total cost of the project, but notes that newer models are becoming more efficient; for instance, Opus 5.5 solves half the proofs with about 40 percent fewer tokens than its predecessor.

This project suggests that AI can increasingly take on the role of proving its own code, potentially reducing the time developers spend on manual review. The approach implies that if AI can guarantee the correctness of code under defined conditions, the industry may shift focus from merely generating code to verifying its safety. Anthropic expects this form of AI-driven verification to become as routine as standard testing in the near future. Theorem’s roadmap already targets critical infrastructure components such as glibc, OpenSSL, and cURL, indicating a broader application for this automated verification technique.

The findings highlight a potential leap in software automation. By combining AI-generated code with automatic proof generation, developers may gain greater confidence in AI-written software. This could reduce costs across the entire coding process, not just in verification. As models become more efficient and the methodology refines, the barrier to entry for formal verification may lower, making it accessible for a wider range of software applications beyond critical infrastructure.