We developed this product for over 1.5 years, closed clients left right and center, and now we're throwing everything away.
In case you don't know me (or Autonoma), we're no strangers to pivots. Funnily enough, we pivoted like 4 times already (enterprise search, documentation generation, coding agent, QA testing platform). The reasons are beyond the scope of this article. In all cases, we knew bugs were painful, we just didn't know what was the best way of solving the problem.
On our last pivot, we actually started getting customers and raised a round from one of the biggest names in the industry. We hired a team of 14 (as I'm writing this article) and started closing clients left right and center. So why would we take such a drastic decision over something that's working for many customers? Well, many reasons. That's why I'm writing this article.
The No-Tests Era (and Why I Regret It)
In my past, I've been guilty of overengineering (as many of you probably also did, and if you say you didn't you're lying). I was Uncle Bob-Pilled but I've been burnt so hard with this that in this startup, I went 180 deg in the opposite direction: TypeScript monorepo, no strict, no tests. Just ship and that's all that matters.
That worked great for our early customers when there were only two of us writing code and each of us owned a huge portion of it and knew everything. After we started hiring, it became a disaster. Bugs were appearing everywhere out of the blue. The codebase was a huge mess of nulls, undefined behaviour, bad error handling. It was so bad that we actually lost a client over this.
For the longest time, I would NOT allow people to write tests because I thought that culturally, we need to have a culture of shipping fast and we should be dogfooding our own product and that should be enough. At some point, I realized that I was affecting the quality of our product and productivity and I changed my mind. In some ways, it was too late. That's why now that we're rewriting everything, we started with tests from the ground up and the most strict TypeScript mode.
Why a Rewrite and Not a Refactor
Initially, when we set out to build this product, we wanted a fully agentic solution. At the time, the tech wasn't there. It was GPT-4 era (not 4o, just 4). The models were so bad that we needed to build huge guardrails and give them very accurate information in order for them to kind of work. We built huge Playwright and Appium wrappers that would do very complex inspections over the code and extract a bunch of information. I'm proud to say that none of the open source solutions I've seen come close to the level of sophistication and replicability we built. We had like 7 clicking strategies that would self-heal on the fly if something changed, for example. That made running tests super fast and reliable.
Now, models have actually advanced so much that the sophisticated inspection is not necessary anymore. And we'd be bringing a huge codebase, with crippling tech debt and vestiges of a worse past of unstrict TypeScript, with not much to gain. We discussed it with the team and decided the best way moving forward was to rewrite the whole thing (bringing the agentic stuff over).
... continue reading