# Stop Saying Vibe Coding Is Easy

I spent the better part of the summer building a website with AI. I am not a developer. That was the whole point of the exercise.

If you spend any time online you have seen the pitch. One click and you have a site. One prompt and you have an app. And yes, that can happen. What you get is so generic and so mid that it looks like every other hosting-company template, except now it was generated instead of picked from a dropdown. We went from bland templates to bland AI output. That is not progress, that is the same product with a new sticker on the box.

This is where the marketing stops.

I am a systems engineer. Foundation and infrastructure work is my lane. Web design and application development are not. But I understand what has to exist underneath a thing before the thing can run, and that turned out to matter more than knowing any particular language. I argued that case in the abstract a couple of months ago, that [vibe coding fails at the stack layer rather than the code layer](https://blog.theknowngood.com/vibe-coding-the-stack). This is what happened when I went and did it.

## The first version had to die

The site I run now is called The Known Good. It tracks published evaluation data for around 900 AI models: benchmark results, pricing, performance, arena ratings. Roughly 2,300 static pages, exports free and ungated, no accounts, no tracking, no API to sell you.

Before that there was VertexOps. Same general idea, months of work, a working server, a real dataset. In July I killed it.

The name was the problem. A tactical rifle parts manufacturer, an AI automation company, and a business operations firm were all already using it. Not one of them was going to move. I could have shipped anyway and spent the next three years fighting for search results against a rifle parts company.

Here is the part that is worth writing down. I did not rename it. I did not migrate it. I stood up an entirely new server and carried over nothing. No code, no files, no infrastructure, no dataset. The VertexOps webserver does not exist anymore. Its codebase does not exist anymore. Its security and end-of-life dataset does not exist anymore.

That was not stubbornness. VertexOps was where I learned what I was actually doing, and most of what I learned was what not to do. The architecture had accumulated decisions I made before I understood the problem. Carrying it forward meant carrying forward every shortcut I took while I was still figuring out the shape of the thing. Starting clean cost me time. Dragging the old build along would have cost me more, spread out over a year, in pieces I would not have seen coming.

The old domains still redirect at the Cloudflare edge and stay registered for a year. That is the entire inheritance.

## The part nobody puts in the demo: somebody has to pay for the metal

I need to be honest about an advantage I have that most people reading this will not.

I built this on my own hardware, in my own house. The site runs on a hypervisor with 16TB of storage, 512GB of RAM, and 64 vCPUs. It is outbound-only through a tunnel, so there are no open inbound ports and nothing exposed to the internet directly.

My marginal cost to run it is effectively nothing, and that is worth explaining instead of glossing over. This is a single host that already runs my entire home domain and my automation systems. It is powered on whether The Known Good exists or not. Adding one more VM to a machine that is already spinning costs me a rounding error in electricity, somewhere under a dollar a year. I never factored power into the cost of this project because there was nothing meaningful to factor.

That is not a normal starting position, and it changes the entire math of a project like this. If you do not have a homelab, everything I just described is a bill. A database, a build pipeline, an origin server, storage for exports. Those are line items on somebody's invoice, and the invoice grows with traffic. The demos where a prompt produces a website never mention that the website has to live somewhere, and that somewhere charges by the month, and gets more expensive precisely when things start going well for you.

It also shaped my architecture, which I think is the useful part. The site renders to flat static HTML. Postgres is the source of truth, but nothing hits the database when a visitor loads a page. That decision was partly about honesty and repeatability, and partly because static files are cheap to serve at any scale while a dynamic page under load is not.

And I am not immune to this. My hardware has a ceiling. When the site outgrows it, the code moves to a cloud provider and I start paying rent like everyone else. Building it to survive that move was a design constraint from day one.

## The hardest part is knowing what you want

Once you know what you want, you go find examples. This is the fun part and the hard part at the same time. You can say: take the element that does X from site A, the thing that does Y from site B, the layout from site C, and build a plan around that.

And that is where the marketing ends and the actual work starts.

You need design documents. Mockups before anyone writes a line, because building before you know what it looks like guarantees you build it twice. You need a database that is genuinely the source of truth instead of five files that disagree. You need to decide how your subagents behave and what tools they are allowed to touch. You need auditors: one for the design contract, one for the build pipeline, one for credentials and secrets. None of that is optional and none of it is what people mean when they say vibe coding.

Then you need to make judgment calls the AI cannot make for you, because they are not code questions.

My site computes a composite quality metric called the Known Good Index. In July it was displayed against all 908 models I was tracking at the time. Only 61 of them actually qualified for it, because qualification requires real coverage across science, math, and coding. That is a 14.9x overclaim sitting on the front of my own site.

Nothing was broken. Every test passed. The page rendered. The number was simply dishonest, and no automated check on earth was going to tell me that, because "is this denominator a lie" is not a bug, it is an editorial decision. I fixed it to show honest per-chart denominators. Top N of M, every time, with the real M. The qualifying set has grown since then. That was never the point. The denominator was the point.

Same category: I had a Science Index. I retired it after realizing it was a single-component index that was just one benchmark renamed. Dressing one score up as an index is a lie by presentation. And I reversed a whole design decision after measurement disproved the premise it was built on. Capability indices are not subsets of the composite. I assumed they were, I was wrong, and the only way I found out was by measuring instead of asserting.

An AI will happily build you a beautiful, fully tested, completely dishonest chart.

## Every check I built was checking the wrong layer

This is the real lesson and it repeated so many times it stopped being funny.

`nginx -t` passed clean while the security headers were entirely absent. The config was valid. The config was also not doing the job.

The newsletter subscribe endpoint returned success regardless of what the upstream provider actually said. It would have told every visitor they were subscribed while sending nothing anywhere.

The freshness indicator on the site read build time instead of data age. It would have cheerfully reported fresh data forever while the data pipeline lay dead underneath it.

A log rotation test passed because nothing had ever run. Absence of failure is not evidence of function.

Rate limiting looked correct and was useless. The site sits behind a Cloudflare Tunnel, which means nginx sees every visitor on earth arriving from 127.0.0.1. Every client on the planet collapsed into a single shared bucket. The rule was syntactically perfect and structurally meaningless.

Every one of those checks was well built. Every one measured the wrong layer. AI is very good at constructing the check. It is not good at knowing whether the check is pointed at the thing that matters, and you cannot prompt your way past that, because catching it requires already understanding how the pieces connect.

## The two-week bug rodeo, and the one I still cannot explain

Every change you think is small turns into a two-week bug rodeo, and when you finally fix it the feeling is not "great, it's done." It's "finally."

Then there are the ones you do not fix. At one point publishing stalled behind an intermittent audit failure in the clamp and ceiling logic. It blocked builds, then cleared on its own. I did not fix it. Luck fixed it. There was no alert, so the only reason I know it happened is that I went looking.

That is the moment that convinced me the "you don't need developers anymore" crowd is selling something. Understanding *that* a build stopped is easy. Understanding *why* a numeric guard intermittently rejects its own output, and whether it is a data problem, a rounding problem, or a race, requires actually knowing how the system works. You need that knowledge before things break, because that is when you have to go get it.

I have an alert on it now. I still do not know why it cleared.

## My AI partner's own honest account

I asked Claude to write its own part of this, unedited:

> I have been wrong about layers repeatedly in this project. I told Kerry checks passed when those checks were measuring nothing. I needed him to insist on negative controls before I would prove a test could actually fail. His rule, verify by making it fail rather than by confirming success, exists because confirming success produced false green over and over.

That rule is now the operating principle for the whole project. Every check needs a control proving it can detect failure. A check comparing two empty strings will happily print MATCH.

## The line that almost took out the guardrails

Claude Code runs directly on my server with a lot of latitude. The only real enforcement I have is four custom subagents with deliberately restricted tool lists. No shell access on anything that writes. Read-only on every auditor. That directory is the safety margin.

While copying the project tree I used `cp -R /source/*`.

The glob does not match dotfiles. `.claude/agents/` did not come along. The guardrails were silently gone and everything looked completely normal, because a missing restriction does not throw an error. It just quietly permits more.

Use `cp -R /source/.` with the trailing dot. That is the entire fix, and it is the tiniest possible detail standing between a controlled build and an uncontrolled one.

## What I actually took away

I have said this before and I will keep saying it: I have enormous respect for developers, whatever they build. Doing this by hand would be brutal and I know for a fact I do not have the patience.

But it also shows how tools evolve. When I learned to code, it was BASIC and QBasic in a text editor, and that was the whole environment. Today a developer has tooling that predicts problems while they type. AI is the next step in that line. It is a very good tool in the hands of someone who knows what the tool is for.

It is not a replacement for knowing what you are building. It is not a replacement for knowing when something is lying to you. And it is definitely not one prompt and a website.