My theory is that the future of AI specialization, specifically for individuals and companies not actively building Large Language Models (LLMs), will be determined by the ability to understand which different models, in their strengths and weaknesses, can be combined with agentic infrastructure to optimize tasks on the behalf of users.
Take this example: model switching is already being done at a conscious level by builders utilizing Claude Code or Codex when performing developmental tasks. When Claude usage starts to fill up fast through repeated Opus use, a developer understands quickly, that just maybe, it won’t be a bad idea to switch to Sonnet for a particular task instead. Context gets eaten at a fast rate. Even frontier labs are encouraging the use of different models for different types of work.
This article is inspired by research I’m doing in the AI space and to serve as a way for me to summarize and walk through, without the help of AI, the concepts that I’m reading in papers like the one from Google.
The reasoning behind writing this, which may also just be more of a personal note for me than for anyone else, is to keep my brain active in deep and slow learning. AI summaries make it easy to condense mass amounts of information into easily digestible bites. Taking the time to walk through research and attempt to communicate the concepts is a method I found helpful to keep the brain malleable, build a deeper educational understanding, and improve communicative skills.
The following will break down the core pillars of what I identified is most important in understanding the composition of an AI agent.
- The Model
- The Orchestration
- The Data Layer
There also exists the concept of the agent runtime, or the system and infrastructure needed to deploy an agent. I personally think that the system around deploying agents is independent of what an AI agent actually is, and will be covered in my next notes.
The Model:
The model is the brain that drives reasoning, it’s Opus, ChatGPT, Kimi, systems trained on billions of parameters that allow them to predict the next pattern in a sequence. AI, in the simplest terms, is a very specialized prediction engine. When looking at the agent through a human comparable metaphor, humans (or agents), can have a different output or “thought” based on the different “brains”, which is all dependent on the individual. Like how we can gain knowledge through applied learning, we also have the ability to “tune” the brain of the model to have agents specialize in certain domains. Just as an employee is evaluated on their knowledge in whatever domain they are applying for, model selection per agent is determined by a cost/latency/and quality needs. Once a model is selected then the tuning can begin.
This concept is pretty simple. Tuning is just training the model on your own dataset. I liken it to giving an intern all the company documentation, so they can get up to speed on their tasks.
Specialization will be driven by individuals who understand the tradeoffs per individual models, and what needs to be done to tune the model for that specific task.
The Orchestration
Since I’ve started on the analogy of agents comparable to humans, the way that I understand the orchestration is to think of it like nerves connected to all the muscles that move the arms, legs, etc.
In this way, AI agents are composed of tools that it uses (the limbs), an AI infrastructure that allows them to orchestrate tasks with these tools (the synapses connecting everything to the brain), and a storage “memory” to make orchestration more efficient (the hippocampus, and also the data layer that we will talk about in the next section).
Orchestration of agentic workflow might be the most singular important piece of using AI for business. If you can use orchestration to create an autonomous system that utilizes the ReAct (Reason + Action) framework, this energizes the reasoning and acting capabilities of large language models. Orchestration is the data infrastructure built out to connect the model from “what do I do now” to “how do I do it now”. The reason assesses the current state and the goal and determines if an action needs to be taken with a tool. Then it determines the tool that can carry out the accomplished goal. For example: I want to create a powerpoint → I ask agent → it sees if there is a PPT tool it can invoke → it then invokes the tool to create a slide deck.
Tools are just API calls, functions written in house, or even other agents. They are independent of the agent in terms of reasoning, and only help the agent to accomplish its goal.
Data Layer
Humans subconsciously know how to walk without the need to dig deep into our memory stores, but we do need to think hard when recalling old abstract concepts we learned years ago. The data layer of the agent is the memory, subconscious and conscious, to help perform tasks efficiently.
Data architecture for agentic systems can be broken up into three needs, persistent storage for long term retrieval, short term working memory to reduce latency (through caching), and transactional memory for auditing and recording actions done by the AI agent.
Persistent storage for long term memory is a key component that allows the agent to stop and start back up again whenever the user decides to bring it up. It acts and picks up on the previous work that was done. Long term persistence can be broken down into three additional core components, a knowledge base for grounding (connecting models to real time data to reduce hallucinations), user interaction data to personalize the experience, and a data lake for future training.
Short term and transactional memory are used more to speed up agents and keep track of what it does as it works. Just like us humans, we may not know the year that the first un-tethered space walk occurred. But, after googling (using the tool) we find out that it occurred in February 7, 1984, and if someone were to ask you literally thirty seconds again after you googled it, unless your brain was absolutely fried, you should be able to respond correctly without needing to google it again. This is the layer required to track and maintain ongoing conversations to reduce latency, so that agents don’t need to call tools again if the answer has already been cached.
Conclusion:
Understanding the core concepts of agents and how they can be used serves to position yourself for greater opportunities as the market continues to grow. My prediction is that the future lies in building the infrastructure that enable users to draw out the most productivity, and benefit, from AI.
