Simple prompt or agent workflow? How not to overthink AI

Photobank2/iStock/Getty Images Plus Follow ZDNET: Add us as a preferred source on Google. ZDNET's key takeaways Gen AI success is about knowing which approach to use. Start with the simplest tool that solves your problem. Sometimes you'll need a prompt, and other times an agent workflow. "You're probably making AI harder than it needs to be." This advice from Corey Noles and Grant Harvey's latest episode of The Neuron podcast urges greater simplicity in what has become a complicated and confusing affair in recent years -- making generative AI fit for the organization. Also: 3 smart ways business leaders can build successful AI strategies - before it's too late With 95% of generative AI projects failing, as found in a recent MIT study, it's time to put these initiatives into perspective. Many people inside and outside the technology profession view implementing gen AI as a major task that requires rigorous project management discipline. Experts even refer to that phenomenon with ZDNET, telling people not to rush into AI, but to think carefully about what they are doing first. Also: Gartner says add AI agents ASAP - or else. Oh, and they're also overhyped While some AI initiatives, at a base level, don't require such overthinking, others, on the other hand, do require more thoughtful planning. So, where does a budding AI professional draw the line? Overall, people "are getting it all wrong" with implementing AI, Noles and Harvey asserted. Success is a matter of knowing when the simplest approaches apply -- using a screwdriver instead of applying a sledgehammer for a challenge that may require AI. It's also about knowing when more sophisticated approaches are needed to prevent AI projects from going off the rails. The prodcasters suggested four ways to solve a problem with gen AI: Simply "chatting." Writing a structured prompt. Building a project. Deploying an agent. Also: Gen AI disillusionment looms, according to Gartner's 2025 Hype Cycle report "Start with the simplest tool that solves your problem," Noles advised. "Don't build an agent when a basic chat will do. We find people get super excited about all these awesome tools. But in practice, you really don't need to jump straight to building complex agents when a simple prompt works." But also "don't spend two hours pulling your hair crafting the perfect prompt and trying to one-shot a complicated task when an agent could handle the whole workflow for you," he added. "There are definitely times you really do need to craft an epic prompt or build an agent workflow. The trick is knowing when to use what." Also: 5 ways automation can speed up your daily workflow - and implementation is easy Noles and Harvey provide some guidelines to help professionals decide how much effort an AI-related project needs: When to simply chat : "If you want to dive deep down some rabbit hole about the history of bunny rabbits and whether they really eat carrots, you can absolutely do that in a normal chat window," said Noles. "You can do an awful lot more there than what most people would think. There is a whole lot of stuff you can do in a chat window just asking it a question without having to do some fancy prompt, without having to get into all of that." : "If you want to dive deep down some rabbit hole about the history of bunny rabbits and whether they really eat carrots, you can absolutely do that in a normal chat window," said Noles. "You can do an awful lot more there than what most people would think. There is a whole lot of stuff you can do in a chat window just asking it a question without having to do some fancy prompt, without having to get into all of that." When to do a structured prompt : A structured prompt is best for a multi-part or a formatted output, said Harvey: "A multi-part task would be you need the AI to go out and do multiple things at once: reference multiple documents, or anything more complicated than a one-off task." Structured prompts are useful for tailored formatting, hyperlinks, and presentation. Structured prompts are also the foundation for the next two higher-level categories discussed, projects and agents. : A structured prompt is best for a multi-part or a formatted output, said Harvey: "A multi-part task would be you need the AI to go out and do multiple things at once: reference multiple documents, or anything more complicated than a one-off task." Structured prompts are useful for tailored formatting, hyperlinks, and presentation. Structured prompts are also the foundation for the next two higher-level categories discussed, projects and agents. When to employ projects : Projects work best for repeated-use tasks. "Choose a project when what you need is a predictable outcome every time," said Noles. This approach works "where control and transparency matter. The steps are right there, your instructions are right there. It can be updated quickly on the fly. If something isn't working, you can go in and make tweaks really easily without having to log into some other software and pull up a node where you have to go in and make changes." For example, "if you know that we change how we do reports every 30 days, a project is probably right for you because you can go in, pull one out, drop another in -- 20 seconds of your day." : Projects work best for repeated-use tasks. "Choose a project when what you need is a predictable outcome every time," said Noles. This approach works "where control and transparency matter. The steps are right there, your instructions are right there. It can be updated quickly on the fly. If something isn't working, you can go in and make tweaks really easily without having to log into some other software and pull up a node where you have to go in and make changes." For example, "if you know that we change how we do reports every 30 days, a project is probably right for you because you can go in, pull one out, drop another in -- 20 seconds of your day." When to use agents: Agents are required when autonomy is part of the system. They come in to handle complicated, asynchronous tasks that require tools and time. "Use an agent when the task requires autonomy, requires dynamic decision and tool use," said Harvey. "You need the AI to just go and figure it out itself. I can't be babysitting it the whole time; I need it to just do this for me. I need it to be able to make multiple decisions on the fly on its own. And I need you to be able to use tools like MCP connectors so that you can go out and do it." Gen AI has the potential to advance productivity and innovation if employed smartly. Understanding the four stages of deployment and the effort required can prevent gen AI from becoming a counterforce to the productivity and innovation the technology is supposed to deliver.