Modular Is Not Modular Is Not Modular

This is the final part of a three-part series about moving our marketing site from a hardcoded setup to a headless CMS. Part 1 covered choosing the framework. Part 2 focused on hosting and CI automation. Now, in Part 3, I’ll talk about the CMS. This part turned out to be the most useful, since it was the only stage where the coding agent’s work didn’t come out right.

It’s also the part that puts the series’ claim to its hardest test. What we’re doing at PlayAI is AI-amplified development: agents don’t just generate code samples; they carry out real tasks end to end. In Part 1, that meant my role shifted from writing code to asking the right questions. In Part 2, it meant building an environment agents can verify on their own. This part is about what happens when both of those are in place—capable agents, a verifiable environment—and the result is still wrong, because the one thing I supplied myself was the objective.

The task was to move five custom-built pages into Storyblok without changing how they looked to visitors. The agents did exactly that, and by all the standards I set, they did a great job. The page output matched the hardcoded version exactly. Screenshots looked the same. All tests passed. Editors could update text and publish in about ninety seconds, without needing a developer.

But when I checked the content model, I realized it didn’t really exist.

What the audit found

There were sixteen components in total: five content types and eleven blocks that could be nested. If you look just at their layout, without thinking about their names or which page they came from, here’s what those eleven blocks really are.

Seven of these blocks are only used once. Not just rarely—only once. If a block only appears in one place, it isn’t really a block. It’s more like a function with extra steps.

Two other blocks seem to be reused, but they actually just reuse a name, not a layout. One is called card_list, and it has a variant field with four options: why, future, features, benefits. These aren’t layouts—they’re just section names from our homepage and portal page. Each option has its own unique set of fields. The component has seven fields, but each use only needs about three. The other block, hero, has variants called home, portal, and about. It follows the same pattern: the names match where the block appears.

There was an even bigger problem: there was no idea of a full-width band. Every section on the site needs a background, but backgrounds had been set up in every possible way you can imagine:

• An overlay div at negative z-index.

• Tailwind gradient utilities applied per half.

• An absolutely positioned decorative image.

• A full-bleed image over black.

• An image plus a scrim, nested inside a child component.

• A flat color passed through a layout escape hatch.

• A CSS background-image on the route.

This is why the shared layout component ended up with three overlapping ways to handle styling, and why two blocks didn’t use it at all.

Each of those choices made sense on its own.

But the real problem started with the goal I set. The agents were told to move five existing pages into a CMS without changing how they looked. They did exactly that. This is what reinforcement learning calls specification gaming: the agent meets the exact goal but misses the real intent—not because it failed, but because it only had the goal to go on. Saying “the output is identical” works well for a migration, but it’s useless for building a good content model. When you start measuring the wrong thing, you stop measuring what actually matters.

It’s not just unhelpful—it’s actually misleading. If you copy five pages exactly, you end up grouping blocks by the page they came from, because that’s the only structure in the input. There’s nothing else to build a general model from. If you want both a perfect copy and a general model, you’re asking for two different things but only checking one.

Better prompting is not the fix

I want to explain what I mean here, because it might seem like I should have just asked for reusable blocks. I thought about that option seriously before deciding against it.

Telling the agent to “build a reusable block library” isn’t a real requirement—it’s just a hope, with no way to check if it worked. The agent can’t know if it succeeded, so it focuses on what it can measure: the rendering. Even if I wrote a lot of instructions about reusability, I’d probably get the same sixteen components, because there’s no clear test to fail. I saw the same thing with deploy rules in Part 2: if something has to be remembered at the right time, it can just as easily be forgotten, whether it’s in a rule file or a prompt. You have to give the agent a signal it can check, or you’ll keep getting correct answers to the wrong question.

Two rules that return yes or no

What actually helped was replacing that vague goal with something you can test: one main principle and two rules that give a clear yes or no when you apply them to any component.

The main principle is to group blocks by their design traits, not by their content or the page they came from. Everything is measured by one question: Can this block be placed somewhere new and still look right?

Rule 1. A block that cannot render standalone on a page is a filling, not a band, and may only be offered inside its parent. That’s a test you can actually run: take the block, put it somewhere it has never been, look at it. Our prose block fails. It renders a bare narrow card that only lands correctly inside a wrapper which two specific routes happen to hardcode. It had also escaped into the page-level list of things an editor can add anywhere, so today, adding it to the homepage produces an orphan white card on a white background, and adding a hero to the privacy page renders full-bleed inside a centered legal column. Both of those are reachable in the editor right now. Neither errors.

Rule 2. A variant is legitimate only if it does not change which fields are meaningful. If it does, it’s two components hiding in one. This one is nearly mechanical, and it sorts the ambiguous cases instantly. Hero home against portal passes: same fields, different theme. Hero about fails, because it makes the call-to-action fields meaningless and moves the image from beside the copy to behind it. All four card_list variants fail. The failure has a visible cost, too: those dead call-to-action fields are sitting in the About page editor right now, visible, editable, and connected to nothing.

What I like about these rules is that an agent can use them without my help. They don’t require any special judgment. Rule 2 is almost like a lint check: list the variants, compare their important fields, and split them if they’re different.

The rule caught me too

Here’s something I’d rather not admit, but it’s what convinced me the rules were real and not just a way to justify what I already wanted.

In an earlier draft of the refactor plan, I suggested splitting our page content type into page and legal_page. It seemed like the obvious choice. Legal pages are different—they have a different layout, different authors, and a different tone.

But when I checked this idea against the main principle, I realized splitting by legal_page was just grouping by who wrote the content. That’s the same mistake as naming a variant why. The layout those pages need—a narrow, centered column—is a design trait, so it should be part of the block, not the story type. So I dropped that draft.

If a rule only ever criticizes other people’s work, it’s not a real rule. This one made me change a decision I liked, which is how I know it’s meaningful.

The learning

When the refactor is done, the number of components will actually increase, not decrease. That shows that component count was never the real metric. What matters is how well fields are used and how safely components can be combined: a few self-contained bands that work anywhere, more fillings that only go inside their parent, and content names like “Why PlayAI”, “Key Features”, and “Meet the founders” moved to editor presets, where they belong. Editors choose a shape and add text—they never have to build a layout themselves.

Looking back across all parts, the same thing kept being the bottleneck, and it wasn’t model capability. In Part 1, the models would have picked the framework that suited them and been locally right; experience was what identified the real decision point. In Part 2, they would have followed evidence that pointed at the wrong layer, correctly; the fix was an environment that returns signals a machine can read. Here, they hit a target exactly, and the target was mine. Different failures with one shape: the available signal was silent about what mattered, and no amount of model quality fills a silence.

By the end of this rebuild, that’s what AI-amplified development means to me. It’s not just about telling an agent what to do and checking the results. It’s about turning what I want into something the agent can check on its own. Just describing what you want isn’t enough. You need a test that can run on the output. Saying “make it reusable” just copies five pages. Asking “can you drop it somewhere new and does it render?” gives you a real block library.

The agents were never the real limitation. The challenge was defining what “good” meant in a way that could actually be checked. That skill—not knowing frameworks or writing prompts—is what this migration taught me, and I think it will matter most going forward.