From Chatbot to Agent: What Tool Calling and MCP Actually Change

A chatbot writes an answer. An agent does the thing. The step between them has a name, a protocol, and a very specific set of failure modes.

Six months ago, most enterprise LLM projects looked the same: a chat window, a prompt, a wall of retrieved text, and a hope that the answer holds up. Today the same teams are asking a different question. They no longer want the model to tell them what the customer status is. They want it to look it up, update it, and file the ticket. That is not a bigger chatbot. That is an agent.

The step between the two is tool calling. And the reason tool calling has suddenly become boring plumbing instead of a research project is a small, quiet protocol called MCP — the Model Context Protocol. This post is about what actually changes when you cross that line.

Core insight

An LLM without tools is a well-read colleague who can only write memos. An LLM with tools can do the work. The interesting engineering is not the model — it is the tools you let it call, and what happens when it calls them wrong.

What tool calling actually is

Underneath the marketing, tool calling is a very small idea.

You give the model a JSON description of the functions available to it: their names, their arguments, and what they do. You ask a question. Instead of writing prose, the model can now answer with a structured message: "call get_customer_orders(customer_id="42")." Your code runs the function, feeds the result back into the conversation, and the model continues from there.

That is the whole mechanism. Four sentences. Everything else — routing, retries, fallbacks, memory, multi-step reasoning — is engineering that sits on top of those four sentences.

What changed is the model's reliability at doing it. Two years ago, tool calls hallucinated the argument names half the time. Today, the top-tier models (Claude, GPT-4o, Gemini) treat structured tool calls as first-class output. You can trust the schema. You cannot yet trust the intent — but that is a different problem.

From tool calling to agent

An agent is a tool-calling LLM in a loop. That is the practical definition.

Loop iteration one: ask the model. It calls a tool. You run it. You put the result back. Loop iteration two: same thing. Loop iteration N: the model answers in prose. Done.

Everything else people mean by "agent" — memory, planning, self-correction, sub-agents, review passes — is orchestration around that loop. The loop is the atom.

Where teams get in trouble is not with the loop itself, but with what they let the loop touch. A read-only agent that queries a data warehouse and writes a summary is a very different risk profile from an agent that can send a wire transfer. Both are agents. Only one gets an audit committee involved.

Why MCP appeared, and why it matters

Once you have a tool-calling loop, the next question is: how do you expose your tools to it?

For eighteen months the answer was: however you like. Every framework had its own way. LangChain wrapped functions. LlamaIndex wrapped functions differently. OpenAI's Assistants API wrapped them a third way. Each vendor's own tool-server format. Nothing composed. If your CRM team built a create_ticket tool for their agent, the pricing team had to rebuild it from scratch for theirs. And the moment you swapped the underlying model, everything broke.

MCP — the Model Context Protocol, published by Anthropic in late 2024 and now supported by Anthropic, OpenAI, and most orchestration frameworks — is what the industry converged on. It is deliberately small: a client (the model host) talks to a server (your tool provider) over JSON-RPC. The server advertises tools, resources, and prompts. That's essentially it.

The reason it matters is not technical elegance. It matters because it decouples who owns the tool from who owns the agent.

Your SAP team can publish an MCP server for the two hundred functions they are willing to expose. Your CRM team publishes theirs. Your data platform publishes theirs. Every agent in the company — whether it's built by the marketing team on Claude, by finance on GPT-4o, or by ops on a self-hosted model — talks to those servers the same way. No re-wrapping. No custom glue per model. The tool catalogue becomes an asset. The model becomes swappable.

For an enterprise, that decoupling is worth more than any single model capability.

The next hard problem: finding the right tool

Standardising the protocol solves one problem — and quietly creates the next. Once every team in your organisation publishes an MCP server, an agent may have hundreds of tools available. Which one does it call for "cancel this order"? Which one for "who owns this incident"? The catalogue no longer fits into a single prompt, and simply throwing all tool descriptions into context degrades both cost and reasoning quality.

This is the same problem we saw four years ago with vector databases and RAG, just wearing a new hat. Back then: too many documents, which paragraph do you feed the model? The answer became semantic search over embeddings, with re-ranking, with negative examples, with all the messy quality work it takes to make retrieval actually reliable. Now: too many tools, which one should the agent even consider? The likely answer is the same shape — semantic search over tool descriptions, ranked, filtered, and evaluated against a golden set of representative queries.

The research on this is not done. Prof. Wolfgang Nejdl's group at L3S / Leibniz University Hannover — where I also lecture — is one of several teams actively working on tool retrieval and agent-side reasoning about tool catalogues. In practice today, teams either cap each agent to ten or fifteen tools (which limits the design), route through a meta-agent that picks a sub-agent (which pushes the same problem one level up), or start layering MCP-native tool retrieval on top of the catalogue. Expect this to be a solved-enough problem within twelve months. Expect it to eat a lot of engineering time in the meantime.

What actually happens in production

The failure modes are surprisingly consistent across the deployments we have run. Four keep showing up.

1. Tools without teeth. Someone exposes a get_orders tool that returns two thousand rows. The model's context window fills, quality drops, latency spikes. Good tools are opinionated: they paginate, they filter, they summarise. A tool that returns "everything" is not a tool — it is a data pipe, and the model will drown in it.
2. Descriptions that lie. The model chooses which tool to call based on the tool's description text, not its name. A description that says "get customer information" gets picked for questions the tool cannot actually answer. If a tool description overpromises, the loop will call it, get useless results, and then either hallucinate around the gap or give up. The fix is unglamorous: write descriptions that are precise about scope and explicit about what the tool does not do.
3. Write actions with no gates. An agent that can write into your CRM without a confirmation step is one bad prompt away from a support incident. Every writing tool we build now has either a human-in-the-loop confirmation, a dry-run mode with a diff, or both. The friction is deliberate.
4. Silent failure. A tool that returns {"result": null} on error looks to the model like a valid empty result. It will proceed. Tools that fail should throw errors the model can see and react to — "error": "Customer 42 does not exist" — not silence.

None of these are exotic. All four fail the same way: the loop keeps running, the answer looks plausible, and the mistake is only visible if someone actually checks.

Where to start

If you are moving your first application from chatbot to agent, three moves get you most of the way.

First, catalogue your read-only tools before your write tools. Read-only agents are five times faster to ship and one hundred times safer to iterate on. Your first agent should be able to look things up across three or four systems. That alone eliminates a shocking amount of manual work.

Second, put your tools behind MCP servers from day one. Even if you only have one team building agents today, you will have three next quarter. Structure for that.

Third, build a golden set of prompts before you build the agent. Twenty to fifty representative questions with expected outcomes. Run it against every change. Without it, every model swap and every prompt tweak is a leap of faith.

That last point deserves its own post, and it has one — What is a Golden Set, and Why Every Genie Space Needs One.

When the framework does not fit

Not every automation deserves an agent. If the flow is deterministic — five steps, always the same order, always the same inputs — a plain function is faster to build, cheaper to run, and easier to reason about. Tool calling shines when the sequence of steps is unpredictable but the steps themselves are well-defined. If both are stable, you have a workflow, not an agent, and a workflow engine will serve you better.

Similarly, if the underlying data quality is bad, no amount of tool wiring rescues the answer. An agent that queries a warehouse full of unclear column semantics will produce wrong numbers with confidence. Fix the data layer first. The agent is downstream of it.

Hands-on

We took the concepts here and wired them into a working demo you can try. Ask our public chat widget on ultramainds.com about any blog post — it uses the same tool-calling pattern described above (via Claude), with MCP-style tool descriptions and a read-only knowledge tool. You can see the loop in action, latency and all. No signup. It uses about half a cent of tokens per exchange.

Everything from the tool descriptions to the retry policy is intentionally boring. That is the point.

Your first MCP server in fifteen lines of Python

If you want to try MCP on your own machine, the Python SDK makes it about as boring as it should be. Install it, expose a function, run it, connect Claude Desktop to it. That is the whole tutorial.

# pip install mcp
from mcp.server.fastmcp import FastMCP

mcp = FastMCP("customer-tools")

@mcp.tool()
def get_customer_orders(customer_id: str) -> list[dict]:
    """Return the last twenty orders for a given customer.
    Use this when the user asks about a customer's purchase history."""
    # your real lookup goes here
    return [{"id": "42-001", "total_eur": 1249.00, "status": "shipped"}]

if __name__ == "__main__":
    mcp.run()

Two things worth noting even in a fifteen-line example. First, the docstring is not documentation — it is the description the model uses to decide when to call this tool. Bad docstring, wrong tool call. Second, the type hints are how MCP builds the JSON schema the model sees. customer_id: str becomes a required string argument. list[dict] becomes the promised return shape. Everything else — logging, retries, auth — you add later.

Register the server in Claude Desktop's claude_desktop_config.json, restart, and ask "what did customer 42 buy?" — the loop runs. If the docstring is precise, the model always picks this tool. If it isn't, it doesn't. That single feedback loop is worth more than a stack of theoretical MCP posts.

What comes next

This is the first of a four-part series on how agentic AI is actually changing enterprise software delivery — not through radical rewrites, but through steady replacement of narrow, repetitive tasks with model-driven loops.

Next week: What is a Golden Set, and Why Every Genie Space Needs One — how a mid-sized trading group nearly reported a quarter's revenue that was 34.9% too high, and how a golden set with sixteen questions caught it.

Free live webinar · 60 min · See agentic AI in action — live.

Three real customer stories, two live demos, and a five-step framework you can apply on Monday.

Germany's first GenAI Boutique, based in Magdeburg. Previously responsible for data and AI strategy at Volkswagen Commercial Vehicles. Author of Artificial Intelligence in the Automotive Industry.