Most people's mental model of AI is a chatbot — you ask a question, you get an answer. Agentic AI is fundamentally different. An AI agent doesn't just respond; it reasons, plans and takes sequences of actions to accomplish a goal. It can use tools, call APIs, read and write files, and loop until the task is done.
How Agentic Systems Work
The key building block of an agentic system is the tool-use pattern. Instead of just generating text, the model can invoke external functions — search the web, query a database, send an email, submit a form. The model decides which tools to call, in what order, based on the goal it's been given.
Real-World Applications
This opens up a new class of automation that wasn't possible with traditional RPA or simple chatbots. Agentic systems can handle tasks that require reasoning and judgment — not just rule-following. Think of a claims processing agent that can read a document, cross-reference a policy, check for fraud signals and draft a decision letter — all in one workflow.
Single-Agent vs Multi-Agent Systems
A single agent with a well-defined toolset handles most business automation tasks fine — read a document, look something up, take an action, report back. Multi-agent systems, where several specialised agents collaborate (one plans, one researches, one writes, one reviews), become useful when a task genuinely benefits from separation of concerns, but they add real coordination overhead: agents can talk past each other, duplicate work or get stuck waiting on each other.
Our rule of thumb is to start with a single agent and only split into multiple agents when there's a specific, demonstrated reason — for example, needing a dedicated review agent with a different, more conservative prompt to catch mistakes the primary agent makes. Multi-agent architecture for its own sake tends to add fragility without adding capability.
Evaluating Agent Performance
Evaluating agents is harder than evaluating a single-turn chatbot response, because success depends on the whole trajectory of actions, not just the final output. Two agents can reach the same correct answer by very different paths — one efficient and auditable, one wasteful and lucky — and a naive 'did it get the right answer' eval won't tell them apart.
In production, we track task completion rate, average number of tool calls per task (a proxy for efficiency), and — critically — a human-reviewed sample of full trajectories, not just final outputs. Sampling trajectories catches agents that are getting the right answer for the wrong reason, which matters a great deal once you scale beyond the cases you've personally reviewed.
Where Agentic AI Falls Short Today
It's worth being honest about the limits. Agentic systems still struggle with tasks that require long-horizon planning across many steps with sparse feedback, with recovering gracefully from a tool returning unexpected or malformed data, and with knowing when to stop and ask for clarification rather than guessing. These aren't reasons to avoid agentic AI — they're reasons to scope early deployments narrowly and keep a human checkpoint on anything consequential.
The Challenge: Reliability and Control
The challenge with agentic systems is reliability and control. Agents can take wrong turns, call tools in unexpected ways, or loop indefinitely. Production agentic systems need careful design: clear tool specifications, guardrails that limit what actions are permitted, and human-in-the-loop checkpoints for high-stakes decisions.
LangChain, LangGraph and the Azure AI Agent Service are the tools we use most in production. If you're exploring agentic AI for your organisation, we'd be happy to walk you through what's practical today versus what's still maturing.
Want to explore this topic for your organisation?
Talk to our team →