AI on Revelara

AI Writes More of Your Code Now. It Also Ships More Reliability Risk.

Sun, 26 Jul 2026 08:00:00 -0700

A 2025 study of 470 pull requests found that AI-generated code shipped roughly 1.7x more issues and about 1.4x more critical defects than human-written code. Nobody serious reads that as “AI code is bad,” and I’m not arguing it either; much of what an agent writes is clean, idiomatic, and correct. The trouble starts when you sit a real workflow on top of that number: volume times defect rate, pointed at the part of the system that hurts most when it breaks.

The Counterfeit Test for AI Metrics

Wed, 15 Jul 2026 08:00:00 -0700

We had an AI quality metric that behaved exactly the way any good metric should. It was stable across model tiers. It improved when we refined the system. It separated weaker specialist lenses from stronger ones. And it measured something tied to proprietary knowledge the base model did not have.

It measured grounding coverage: the percentage of findings that cited a valid control from our reliability catalog. Every control in the catalog traces back to real incidents and known failure patterns.

What the DORA 2026 J-Curve Actually Says About Reliability and Vibe Coding

Mon, 04 May 2026 08:00:00 -0700

DORA shipped The ROI of AI-Assisted Software Development recently. There’s an interesting number from that report that ties back to a Google Cloud 2025 report on the ROI of AI; 78% of executives from organizations with C-level AI sponsorship report seeing ROI now on at least one generative AI use case. The new report shares a number I have not seen quoted before, a 15% productivity drop used as the default in its sample ROI calculator. The report is explicit that the actual depth and duration of the dip are unpredictable; 15% is a placeholder input, not a measurement. On a 500-engineer organization at $176,000 fully loaded salary, that is $3.3 million in lost capacity over three months. The report calls this the “tuition cost” of AI adoption. It includes the line item in its example budget. Then it moves on.