Working Notes on Revelara

Fewer moving goal posts for more reliable commits

Sun, 16 Aug 2026 09:00:00 -0700

The moment I understood what we had built was not during a benchmark or eval. It was during a commit that took three tries.

I asked an agent to make a change. It finished, staged the work, and tried to commit. The pre-commit hook ran a scan and blocked it on a reliability finding. The agent read the finding, understood it, and fixed it. It tried to commit again, and this time its own fix had introduced a second blocking finding. It read that one, fixed it too, and ran again. This time it passed, and committed the whole change set.

Code Review Is a Memory Problem

Sun, 02 Aug 2026 08:00:00 -0700

I have been building Revelara for a while, and there’s one aspect I’ve had a hard time explaining to people. The easy version is “this is a reliability gate: it catches risky changes before they ship.” It’s easy for people to understand the value, but it’s too small for what it really does. The bigger version kept coming out vague, something about knowledge, memory, and the importance of having it in the developers coding context. Every time I said it out loud it sounded like a vague promise in the worst case, and unconvincing and easily dismissed in the best case. So I usually stick with the things that are easy for me to explain and for people to grok.

Security Shifted Left. Reliability Can Too.

Tue, 28 Jul 2026 08:00:00 -0700

For about a decade, security had the same shape as a punishment. You wrote your code, you opened your pull request, you felt good about your week, and then a scanner you didn’t run and couldn’t see filed a report against you. The findings landed after the fact, in a different tool, owned by a different team, and by the time they reached you the context that produced the code was gone. You were being asked to remember why you wrote a line you’d already forgotten writing.

AI Writes More of Your Code Now. It Also Ships More Reliability Risk.

Sun, 26 Jul 2026 08:00:00 -0700

A 2025 study of 470 pull requests found that AI-generated code shipped roughly 1.7x more issues and about 1.4x more critical defects than human-written code. Nobody serious reads that as “AI code is bad,” and I’m not arguing it either; much of what an agent writes is clean, idiomatic, and correct. The trouble starts when you sit a real workflow on top of that number: volume times defect rate, pointed at the part of the system that hurts most when it breaks.

How to Scan a Codebase for Reliability Risks with rvl-cli

Sun, 19 Jul 2026 08:00:00 -0700

Clone a repo you’ve never scanned, install the Revelara CLI, run one scan, and read a ranked list of the reliability risks in that codebase. Budget about fifteen minutes end to end, and know that your hands-on part is a fraction of it. The scan does analysis, reading your code against a controls catalog with expert lenses, and real analysis takes longer than a grep. Kick it off, click through the permissions prompts, and load the risk register. This post is a walkthrough, using a public repo you can clone yourself.

The Counterfeit Test for AI Metrics

Wed, 15 Jul 2026 08:00:00 -0700

We had an AI quality metric that behaved exactly the way any good metric should. It was stable across model tiers. It improved when we refined the system. It separated weaker specialist lenses from stronger ones. And it measured something tied to proprietary knowledge the base model did not have.

It measured grounding coverage: the percentage of findings that cited a valid control from our reliability catalog. Every control in the catalog traces back to real incidents and known failure patterns.

GitHub's February 9 Incidents: What 'Mitigation' Leaves Out

Tue, 26 May 2026 08:00:00 -0700

GitHub’s availability report for February 2026 covers six incidents and there are two on February 9 worth reviewing. Two windows of degraded service an hour and fourteen minutes apart, and GitHub’s own investigation found they came from the same underlying cause. The first was declared mitigated before the second began. The mitigation GitHub applied was a correct response to what the team could see at the time. It also was not enough to prevent the second incident.

The Medium-Severity Paradox: Where Engineering Time Actually Goes

Sun, 10 May 2026 08:00:00 -0700

The average critical-severity incident in the Revelara corpus takes about 19 hours to resolve. The average medium-severity incident takes about 17 hours. On those two numbers alone, the natural conclusion is that critical incidents are roughly where engineering time goes, and that medium-severity work is the manageable middle. The conclusion is wrong by roughly an order of magnitude, and the data shows why.

The numbers, and how the analysis was run

This analysis pulls from the Revelara public postmortem corpus, which currently spans nearly 3,000 published incidents and grows daily. Two severity-targeted comparisons were run. The “critical” set returned 272 incidents matching body language like SEV1, P0, total outage, and major. The “medium” set returned 284 incidents matching SEV3, P2, degraded, and partial.

What the DORA 2026 J-Curve Actually Says About Reliability and Vibe Coding

Mon, 04 May 2026 08:00:00 -0700

DORA shipped The ROI of AI-Assisted Software Development recently. There’s an interesting number from that report that ties back to a Google Cloud 2025 report on the ROI of AI; 78% of executives from organizations with C-level AI sponsorship report seeing ROI now on at least one generative AI use case. The new report shares a number I have not seen quoted before, a 15% productivity drop used as the default in its sample ROI calculator. The report is explicit that the actual depth and duration of the dip are unpredictable; 15% is a placeholder input, not a measurement. On a 500-engineer organization at $176,000 fully loaded salary, that is $3.3 million in lost capacity over three months. The report calls this the “tuition cost” of AI adoption. It includes the line item in its example budget. Then it moves on.

The Observability System I Almost Shipped

Tue, 21 Apr 2026 08:00:00 -0700

I finished a PRD on Sunday for the LLM observability and evaluation stack we’re adding to Revelara. Five layers, twenty-one user stories, async eval scoring, drift detection, the whole thing. I thought it looked great. I was happy with it. Before I started on the implementation, I ran it through our new STPA review tool (mostly because I didn’t trust how happy I was with it, so dogfooding seemed like the best test).