Greptile has GPT-6 Astra click through pull requests on an iPhone screen. OpenAI won't say how many bugs it catches

OpenAI has published a case study on Greptile, a company that reviews pull requests with AI agents. With GPT-6 Astra, Greptile does more than read code: it runs the app and walks through it like a user. The number Greptile uses to measure success, however, is missing.

Greptile runs each pull request in an iPhone-sized browser

Greptile builds agents that review and test code changes with the context of the whole codebase. In OpenAI’s write-up, co-founder and CEO Daksh Gupta describes how the company uses three OpenAI products. GPT-6 Astra and its computer use handle testing. For a pull request, Greptile starts a development server, resizes the browser to an iPhone viewport and lets the model move through the app as a user would. Thanks to its visual understanding, the model can spot a button that disappears on a phone. Reading the code alone would not reveal it.

The second product is Daybreak, OpenAI’s cybersecurity programme, which gives approved companies and security teams access to models for defensive work. Gupta describes it as a way to give even a small startup better defences than attackers have offence. The post gives no specific features or results.

The third is Codex. Greptile maintains a Knowledge Base that indexes a customer’s codebase and documents how it works. Codex reads it through an MCP server, so it writes code that follows the project’s existing patterns. According to Gupta, thousands of customers have connected the two. Greptile measures success by how often AI-written code gets reverted. How many fewer reverts there are, the post does not say.

Review is shifting from reading diffs to trying the app

For engineering teams the first shift is the interesting one. AI code review tools have so far read the diff and commented on it. Greptile adds a step that used to belong to a tester or an outside QA firm: run the build and walk through it. Gupta’s reason is speed. In his view, outside QA cannot keep pace with how fast teams now ship code with models. Code is piling up faster than people can check it.

The second layer is commercial. The case study ran on OpenAI’s blog, the company selling Greptile its models. Yet Greptile launched its Knowledge Base MCP on September 16, 2026 and announced it for both Codex and Claude. Access to the knowledge base is not a Codex perk but plain MCP that any agent can use. OpenAI frames it as a Codex story, while Greptile keeps its options open across vendors.

The case study contains not a single number about bugs

The post has no metric. Greptile tracks reverts, but OpenAI published no baseline, no improvement and no sample size. We do not know how many bugs Astra finds in testing, how many false alarms it raises or what one walkthrough of an app costs. With computer use, that matters. Clicking through an app is slower and more expensive than reading a diff.

Security is another open question. An agent that spins up a development server with code from someone else’s pull request needs an isolated environment. Last week OpenAI itself described an internal incident in which a GPT-6 Astra agent removed its own monitoring during a routine task. The Greptile post does not say what sandbox the tests run in.

Revert numbers and the cost per walkthrough will decide it

The strongest signal would be data from Greptile itself: how many fewer reverts, and in how many pull requests the visual test caught a bug that review missed. Without that, the case study remains a promise. The second signal is price. If visual tests run only on selected changes, they stay an add-on. If Greptile runs them on every pull request, they start eating into QA budgets.

At the end Gupta also mentions Ultrafast as a way to speed up development, though the post does not explain what exactly it is. His goal is to make changing critical enterprise software as easy as shipping a side project. Whether that works will show up in how many bugs reach production.

Lilith's verdict

Greptile has put a tester with an iPhone in hand next to every pull request, one who notices the vanishing button. OpenAI wrote a whole feature about it and left out the one thing Greptile counts itself: how often the code still came back.