Sandeep Shivam is an Affiliate Director at Tavant, constructing AI-powered lending merchandise that enhance effectivity and buyer expertise.
Throughout a potential shopper engagement at a monetary establishment, I realized that that they had spent 4 months constructing an AI agent for a consumer-facing workflow. The group was skilled, the finances was actual and the mannequin choice appeared apparent: Choose the strongest giant language mannequin (LLM) accessible and construct the agent round it.
The pilot seemed promising. The manufacturing rollout didn’t. Latency on client journeys crept previous acceptable thresholds. Inference prices climbed sooner than the worth created. Accuracy was uneven throughout the workflow. After 4 months, the initiative was paused and redesigned. Not as a result of the expertise was mistaken, however as a result of the structure was that’s when my group was engaged to revamp it.
That story is changing into widespread, with the extensively cited report from MIT’s NANDA initiative final 12 months discovering that 95% of enterprise generative AI pilots are failing to ship measurable monetary impression.
From my expertise, whereas the mannequin is commonly the primary suspect for pilots stalling, the structure is the extra probably offender. Typically, a manufacturing AI agent shouldn’t be one mannequin doing the whole lot. It must be a coordinated system of specialised fashions, every doing what it’s best suited to carry out.
The Kitchen Brigade Analogy
Take into consideration how a critical restaurant kitchen works. There isn’t a single chef getting ready each dish. There’s a brigade. The saucier handles sauces. The grill station handles proteins. The pastry chef handles desserts. The expediter coordinates the cross.
Every function exists as a result of no particular person, regardless of how proficient, can ship velocity, precision and consistency throughout each delicacies approach without delay.
A manufacturing AI agent must be designed the identical manner. Assigning each activity to at least one giant mannequin is the equal of asking your head chef to additionally plate desserts and pour wine. It may be performed, but it surely produces a slower, costlier and fewer dependable kitchen.
Why One Mannequin Breaks Down in Manufacturing
Take into account a typical monetary providers workflow. A buyer uploads a doc. The agent should learn it, extract knowledge, classify the intent, validate it towards coverage, question system data, plan the following motion, execute an API name, confirm the end result and reply clearly.
That could be a chain of very totally different duties. Studying a doc wants visible understanding. Intent classification wants velocity. Coverage lookup wants retrieval and rating. Workflow planning wants reasoning. Instrument execution wants a structured output. Exception dealing with generally wants deeper verification.
A single giant mannequin can technically do all of those. But it surely does them at the price of the most costly useful resource you’ve gotten, and on the velocity of the slowest path by means of the mannequin. In consumer-facing monetary workflows, the place seconds matter and unit economics are scrutinized, that mixture shouldn’t be survivable.
You do not scale AI by selecting an even bigger mannequin. You scale it by selecting the correct one for every job.
Specialised Fashions, Specialised Roles
The subsequent section of agentic AI will probably be constructed on composition, not simply mannequin measurement.
This doesn’t imply fashions function independently. A central orchestrator is required to route every activity, maintain shared context and implement coverage at each step.
Oversight then runs by means of one management airplane: each name logged, each determination traceable, fallbacks triggered when a mannequin is unavailable or returns low confidence. The shopper sees one agent. Beneath, a coordinated group is at work.
The structure I now suggest has clear roles for every mannequin class:
• Small language fashions, or SLM, typically underneath ten billion parameters, are suited to high-volume work reminiscent of intent classification, routing and preprocessing. When the inputs are well-defined and structured—capturing software particulars, classifying a service request or normalizing consumer enter—an SLM handles them rapidly and cheaply. Routing this by means of a big mannequin can add latency and value for no actual profit.
• Imaginative and prescient-language fashions belong within the notion layer, the place paperwork, statements and types should be interpreted. In any workflow constructed on paperwork— statements, identification paperwork, contracts, invoices, claims types—a vision-language mannequin can extract structured knowledge immediately, way more reliably than a text-only mannequin working off uncooked OCR output.
• Common-purpose giant language fashions (LLMs) retain their place on the coronary heart of the agent, doing the reasoning and orchestration they have been designed for. When a buyer asks an open-ended query, or when the agent should coordinate steps throughout a number of downstream methods, a general-purpose LLM is commonly the proper instrument. It must be invoked intentionally, not because the default for each keystroke.
• Reasoning-optimized fashions earn their value on the smaller set of selections the place being mistaken has actual penalties like anomaly evaluate, exception dealing with and compliance-sensitive paths. Locations the place cautious thought is price greater than a sooner, shallower reply.
• Motion-oriented fashions can deal with dependable instrument use, producing structured calls and workflow actions somewhat than free-form textual content that hopes to land on a sound API request. Submitting transactions, triggering downstream providers or updating data by means of enterprise methods are instances the place a mannequin tuned for structured instrument calls is extra reliable than a common LLM making an attempt to compose the identical name.
What Leaders Ought to Do Subsequent
To get began with this structure, map your most vital agent workflow end-to-end and ask which mannequin handles every step. If one mannequin is doing the whole lot, you’ve got discovered your optimization alternative.
Be looking out for the widespread offenders of mannequin mismatch: giant fashions on easy classification, textual content fashions studying paperwork and unstructured technology driving API calls.
As soon as you’ve got discovered your alternatives, outline clear roles per mannequin, consider on workflow outcomes somewhat than benchmark scores and bake governance into the structure from day one: routing, logging, fallbacks, audit trails.
The early period of agentic AI was formed by the idea {that a} extra highly effective mannequin may resolve each drawback. That perception is being reconsidered the place economics and reliability matter greater than benchmark wins.
The subsequent period belongs to composition with agentic architectures constructed the best way nice kitchens are run: clear roles, specialised capabilities, sturdy coordination, measurable accountability.
Forbes Technology Council is an invitation-only group for world-class CIOs, CTOs and expertise executives. Do I qualify?
