RAG vs agentic RAG: one fixed retrieval or a retrieval loop

In the 2020 paper by Lewis and colleagues that set out the approach, a neural retriever fetches passages from a dense vector index and a generator writes the answer from them. AWS's current description follows the same shape: embed your documents, match the query, add the best matches to the prompt, generate.

The pipeline has no judgement of its own. If the first search returns the wrong passages, the model answers from the wrong passages. For a single-topic question against one well-kept collection of documents, that is rarely a problem.

A 2025 survey by Singh and colleagues says traditional RAG is limited by 'static workflows' and lacks the adaptability for multi-step reasoning and complex task management. In the survey's framing, agentic RAG embeds autonomous agents in the retrieval pipeline, using reflection, planning, tool use and multi-agent collaboration.

In practice that means a loop. The agent rewrites or splits the question, chooses which source to search, grades what came back and decides whether to search again or answer. Retrieval stops being a fixed step and becomes a tool the agent calls.

Microsoft's documentation for agentic retrieval in Azure AI Search, read on 7 October 2026, describes a multi-query pipeline for complex questions. An LLM can plan the query by breaking it into focused subqueries, using chat history for context; that planning step is marked as preview. The subqueries run in parallel, each set of results is reranked, and the merged results come back, optionally with source references and an activity log.

The same page states the trade-off plainly. Agentic retrieval adds latency compared with a single-query pipeline, and billing moves from a cost per query to a cost per token that depends on the reasoning effort you configure.

Ask which suppliers had a late delivery last quarter and an unresolved audit finding. Standard RAG embeds the whole question, searches one index and returns passages that mention late deliveries or audits, rarely both for the same supplier.

An agentic RAG loop splits the question. It queries the delivery data for late suppliers, searches the audit reports for open findings limited to those suppliers, checks that the two sets overlap and answers with citations to both. When the audit search returns off-topic passages, the grading step catches it and the agent rewrites the query. Our agentic RAG guide walks through a similar trace step by step.

Anthropic’s guidance supports the conservative default: for many applications, a single model call with retrieval and good examples is enough, and multi-step agentic systems belong where simpler approaches fall short.

The failure modes differ too. Standard RAG fails quietly with a confident answer drawn from the wrong passages. Agentic RAG can fail by looping, searching again and again without converging, so it needs a cap on attempts and a budget per question.

• Stay with standard RAG for single-hop questions against one source, such as a product FAQ or a policy handbook.
• Move to agentic RAG when facts from one search are needed to build the next, or when questions span sources with different shapes.
• Measure first: if your failing questions fail because nothing relevant exists in your documents, a loop will not help.

Cortex includes on-device knowledge bases with retrieval, so a person can ask questions of their own files without sending them to a shared index. That is standard RAG on the desktop, with approvals on tool calls.

For an agentic loop, Studio lets you branch and loop on a canvas, call models through Connect and call your search systems as tools, so the routing, grading and answering steps can each use a different model. Neither Studio nor Connect replaces your vector store. Test the loop against a set of real questions with known answers before you rely on it.

• Agentic RAG guide: Architecture, a full trace and costs
• Swfte Cortex
• Swfte Connect: Route each step to a different model

Facts about other vendors and about the terms on this page were read on the pages below on 7 October 2026. Vendor plans, names and menus change, so check the vendor’s page before you rely on a detail.