Tool calling, multi-agent architectures, reasoning patterns, and the inference requirements behind production agents.
Streaming in agentic pipelines is not the same as streaming chat tokens. Partial tool calls, pipelined steps, and early cancellation change what the user experiences.
Why running multiple tool calls in parallel changes the latency math of an agent, and how inference speed determines whether the parallelism is worth doing.
How agents reconstruct memory between turns, and the latency trade-offs between long context, RAG, summarization, and KV cache reuse.
A practical breakdown of the latency budget inside a code agent, step by step, and why every link in the chain needs to land under a second to keep the loop usable.
Popular agent reasoning patterns are described as prompt techniques, but they are inference cost multipliers. Here is how ReAct, Reflexion, and Chain-of-Thought actually shape the bill and the latency.
Orchestrator and worker patterns make multi-agent systems easy to design and expensive to run. Here is where the inference cost actually goes, and what it means for the infrastructure underneath.
Function calling looks simple on paper, but the latency budget of a tool-using LLM is dominated by short structured generations that most serving stacks are not optimized for. This is what actually makes tool calls feel slow.
Agents make many sequential LLM calls per task, and each one pays the full latency of decoding. This post walks through how that compounds and why fast inference changes which agents are even viable.