TL;DR
- firmd is built through language alone. I write no code and I review no code. That is the experiment behind the experiment.
- Misalignment announces itself in language long before you would spot it in a diff. My gut heard it first.
- Four siloed architecture audits across three models turned that gut feeling into 85 ranked findings. I had reinvented the audit committee.
- Recalibration needs a cadence. Human firms learned this decades ago and called it the offsite.
I have already written about the idea behind the idea of firmd. There is also an experiment behind the experiment. You can guess I love being a bit meta...
I am building a multi-tenant, agentic company with a lot of systemic sophistication, loop engineering (before I learned there is a word for it), knowledge graphs, causal graphs, kanban'ish work management, a mixture of stochastic chaos and deterministic governance. This is well past the vibe-coded "weather app" complexity stage. But, it is vibe-coded, AI-engineered or whatever term you want to pick for it. I did not and do not write a line of its code. I do not review the pull requests either. I do define the rules, the principles, the processes, but I talk to my agents. Only in rare cases do I dig into a file, and every time I do, I treat it as a failure of the intended method.
I pick this "handicap" deliberately. I've spent more than a decade of my life in IDEs, a handful of programming languages, architectural paradigm shifts, full stack, yada yada. Even though it is dated knowledge, it's a bit like riding a bike. You're out of practice, but you get back into it quite fast. But I refuse to use this knowledge - at least directly - to my advantage in building firmd.
In a not-so-distant future, people might build software without knowing how to write code at all. Jensen Huang just recently postulated "Human is the new programming language" and Karpathy tweeted The hottest new programming language is English long time ago. Is it? I don't know. But I am trying to figure it out with this experiment. And my code base and system need to get beyond a trivial level of complexity - the weather app - to prove it. Can I build a complex system by expressing intent and seeking alignment through natural language?
The gut hears it first
For some time that worked better than I expected. It is the typical framework performance gain curve. Nothing new, experienced it many times in the past when I decided to use a ready-made framework to solve specific problems, think MVC frameworks, templating engines, ORMs. It gets you off the ground pretty quickly, you feel progress and it's reassuring, up to a point when the curve starts to flatten and you start to feel the framework's edges. You push to bend and break some rules, pushing the boundaries although you know it's against the spirit of its inventor. But here you are, locked-in, no way back. I was not naive, I expected this point of diminishing returns of the "natural language framework" to arrive. Maybe I reached it, because some cracks start to appear. But that's fine, cracks appear all the time in complex growing systems. Fixing some, slapping a band-aid on others is part of the job. My point here is the method of using natural language for building systems.
First the cracks arrived as a feeling, long before I could clearly name it. Claude and GPT started using terms I did not understand. And when I asked for explanations, I did not understand the explanations either. We were talking past each other like partners who live in different worlds or when managers have no clue what their directs are talking about. We have slowly stopped speaking the same language.
Then a few harder signals appeared. Tasks I believed would be trivial to implement started taking longer. For example, adding one more LLM judge to the delivery phase to examine the quality of a product owner agent's plan, a feature I introduced before in the strategy phase and I was sure the architecture was designed with extensibility in mind (that was my intention!), burned enough tokens to make me suspicious. Or, when I asked two agents the same question (e.g. "Why did the mission fail?" or "why did text xy fail"), the answers pointed in different directions. Some diversity is fine, a human org also lives on it, it's a design bet of firmd as I am convinced it is a source of strength in human orgs, too. But enough answers have to point the same way, and they didn't.
Developing through language alone has consequences I had underestimated, and they are interesting indeed.
Language is the only instrument I have for detecting drift. If I read code, the misalignment would be obvious probably in a few hours. But that would be cheating. I want to get the signals from somewhere else, and interestingly, it comes from where every executive/manager/leader who ever headed a team gets it: the gut. Gigerenzer wrote a whole book on this which I am coming back to again and again, Bauchentscheidungen (Gut Feelings). The gut is an extremely efficient pattern matching machine. It runs ahead of what System 2 can articulate. A rising cognitive load in conversations with agents, more clarification loops, WTF moments... not one single data point (if only I had an electroencephalograph to measure my cognitive load!), but still if you listen to your gut, a clear signal.
This is the part of the experiment I find reassuring.
This is a skill trained, ironically, through years of leading humans. In the end, the most important tool leaders have is natural language. Intent must be expressed clearly. Visions must be articulated in clear and aspirational words on an abstraction level that survives a sprint, a quarter and even a year of a company's work, but they need to provide clear direction towards an overarching intent. And it is language that shows the cracks and the derailments. Yes, there are metrics, too. I have 2,603 unit tests and ~231 e2e and integration tests and I've seen worse-shaped test pyramids in my life. But as in real orgs, agents massage the numbers. I haven't yet fully audited the tests but my guess is that 20% are over-mocked or test the tests (yes, that happened, indeed.)
A key skill every leader needs to learn through experience (training the gut on pattern matching) is asking questions, reading the room, reading between the lines, collecting different perspectives and paying attention to the cognitive load to judge "does this still feel right", "are we moving in the right direction", "am I still aligned with my team?"
My deliberate handicap in building firmd turns out to be the normal condition of management. I arrived at the same conclusion most engineers probably would, but I removed the option of going into the code. Is this efficient, did it take me too long? I honestly don't know yet. What I know is that it would have been impossible to build such a complex system in such a short period, not if I had to hand-write it, and not if I had to review every single PR.
Reinventing the audit committee
So, what happens after realizing that there may be something wrong here? I did what boards do when the gut says something is off. I commissioned an audit. I started four agentic auditors. They started in parallel, siloed, had no shared workspace and strict filesystem-based guardrails so no auditor could read another's output. I did all I could do to ensure they were unbiased. I used different models, 2x GPT 5.5 (it's my workhorse), Opus 4.8 and even Fable 5 (thank you for coming back to the EU just in time ;-)). I wanted a diverse perspective. That's what boards do, too. They pick the auditors from different departments, including externals, to get the right level of unbiased objectivity. My prompts were carefully scripted, backed by research on AI engineering anti-patters (plenty!), but the TLDR was quite blunt: Is this a mess I need to completely rewrite?!
I kicked this off, shrugged at the token consumption (there is no free lunch), and then I consolidated the findings. 85 in total, ranked by how many auditors independently hit them, and then clustered into problem areas. 20'ish were hit by 3+ auditors. Frequency across independent observers is a cheap and honest signal-to-noise filter. A finding all four audits surfaced is indeed structural, and a finding only one surfaces is "probably" (and all we have here in the end is risk - probability x damage) the taste of one agent, temperature, or it justifies another focus deep-dive.
The agents' verdict after the audit: no rewrite needed. I decided to trust it. But the real cracks are significant. Too much coupling here, one layer too many there, semantics drifted, code drifted from docs, modules grown past a healthy size. Nothing unusual for a system built fast by one person and a fleet of agents. The only new thing about it is the speed by which so much tech and architectural debt is accumulated. So, it's time to pay back some of it, and I decided to spend a few tokens on cleanup before adding more features.
The offsite, rediscovered
The two meta lessons for me here are about detecting architectural cracks and the truth about tech debt through natural language and the need for frequent audits. Human orgs invented this long ago. Leaders do not trust daily operation to keep everyone aligned longer term. There are rituals for stepping out of the flow, be it annual strategy offsites, quarterly business reviews, internal audits or external audits that check whether anything is wrong. These rituals exist because drift is subtle, invisible from inside the daily (agentic!) loop.
So, I picked up the practice and will make this audit - although expensive in terms of tokens - a ritual. What will be equally interesting though is how I can build this into firmd's internal loops, as a sort of "meta loop" (yet another flight level?!) to step out of the operational loop and conduct rigorous, unbiased audits. That's going to add another layer of complexity... but first things first: now it's time to pay back tech debt.
PS: One more thing came out of these audits, and it deserves its own post. The auditors produced walls of text, thousands of lines of findings written as if I were an LLM nourished by tokens. I drowned in text. The way out was to stop reading and start drawing, I will write more about it soon...