GoKawiilShould you even use an LLM? Cameron Palmer 7 min read · 2 days ago 2 days ago -- 2 Listen Share
The Agent That Did Too Much
I had a problem: as an AI implementation consultant, I had no automated pipeline to discover and qualify leads and insert them into my self-hosted Twenty CRM instance. I thought I could solve this whole problem using LLMs.
The first version of my prospect discovery system gave one LLM agent the full scouting pipeline: web search, deduplication, validation, and database insertion. About 10–20% of the prospects found were irrelevant, duplicated, or improperly inserted into the CRM. To make matters worse, the system was extremely difficult to debug, as each instance of the agent took a different approach and ran into its own unique process and tooling speed bumps. I ended up having to search through the entire list of 800+ prospects in the CRM manually myself, checking for duplicates and validating if the prospect was relevant. Clearly this solution wasn’t going to work as-is.
“Where Can We Use AI?” Is Backwards
Executives seem obsessed with implementing AI in any form possible. I often hear that AI adoption progress is measured using arbitrary metrics such as token usage or lines of code written, which don’t measure if AI is being used in a helpful or a harmful way. Large language models (LLMs) are fundamentally just a tool. If your CEO thinks AI is a hammer, and she wants to use that hammer for everything, then everything ends up looking like a nail. This is the wrong approach to AI solution architecture.
The goal should not be to say “we’re an AI-first company” or “our product uses AI”. The goal should be to solve the most pressing and relevant problems using the appropriate tools. LLMs are one of those tools. The first question asked should be “what problem are we solving?”, not “where can we apply AI?” When the problem to be solved is decided, only then can the solution be specified. What capabilities does the solution require? Where does the current solution, if any, fail? And is integrating an LLM into that solution actually necessary?
LLMs Trade Determinism for Flexibility
Every tool in the software development tool belt has strengths and weaknesses, and LLMs are no different. The question is not whether LLMs are useful, but whether the capabilities justify the trade offs they bring with them. LLMs provide flexibility by sacrificing determinism. They are useful when the work involved requires interpretation of natural language, synthesis across abundant or varied information, and step-by-step reasoning where the rules of a process can’t be specified before it begins.
These gains in flexibility result in losses in repeatability and determinism. LLMs produce variable output — that is, the same prompt given to the same model twice will produce tangibly different results. This makes testing and failure analysis much more difficult than for traditional code because success criteria are often subjective. Additionally, a process executed by an LLM will typically take much longer and cost more than the same workflow executed using plain code. These limitations mean that we need to evaluate the need for AI in a solution thoroughly before we blindly assume it will be helpful.
... continue reading