Anyone building an AI knowledge agent today hears the same advice: load your documents into a vector index, set up Retrieval-Augmented Generation (RAG), done. What looks elegant in architecture diagrams has one property that rarely appears in proposals: the index costs money every month — even when nobody asks a question.

This article works through what a custom RAG index actually costs, using real Azure list prices verified in August 2026 — and shows when you don't need one at all. Because for SharePoint and Microsoft 365 content, a Microsoft-operated index already exists and is included in the license price.

A note on currency: All prices are list prices (West Europe region, as of August 2026) from the official Microsoft pricing pages. Microsoft now positions Azure AI Search under the Foundry IQ umbrella — the "context engineering platform" of Microsoft's intelligence layer. The billing logic is unchanged. CHF amounts are rounded conversions (assuming 1 USD ≈ 0.80 CHF); your Azure subscription bills in USD.

The meter starts on day 1 — per query and per month

A RAG system has two kinds of cost: variable cost per query (embedding, ranking, LLM answer) and fixed cost per month (the search service itself). The fixed costs are the most underestimated — they run 24/7, regardless of usage.

The animation shows where a cost meter is attached along the path of a single query:

Current list prices for the search service (per search unit per month):

A search unit (SU) is partitions (storage) × replicas (availability). And here is the first hidden trap: high availability requires at least 2 replicas for read HA, 3 for read-write HA. An S1 at $245 quickly becomes $490–$735 per month — for the same data.

Add the variable items:

• Semantic ranker: first 1,000 requests per month free, then $1 per 1,000 requests
• Agentic retrieval (the token-based query planning introduced in 2026): first 50M tokens per month free, then $0.022 per 1M tokens (minimal/low reasoning effort) or $0.10 per 1M tokens (medium)
• Embeddings (Azure OpenAI): text-embedding-3-small $0.022 per 1M tokens, text-embedding-3-large $0.143 per 1M tokens
• LLM answers (Azure OpenAI, GPT-5): $1.25 per 1M input tokens, $10 per 1M output tokens

Each item looks harmless in isolation. The sum — plus the staff time to run the pipeline — is not.

Worked example 1: SME with 50 employees, 10,000 documents

A typical Swiss SME: internal knowledge agent, 10,000 documents (contracts, manuals, project files), around 2,000 queries per month.

Three observations:

1. 1Embeddings are not the cost driver. They cost pennies. The myth that "vectorization is expensive" is wrong for text documents.
2. 293% of the cost is fixed ($319 of $339) — it accrues even if nobody asks a question in August.
3. 3High availability doubles the fixed block: with 2 replicas the total rises to ~$584 ≈ CHF 470 per month — over CHF 5,600 a year, before a single line of application code.

Not in the table: the build (typically 20–60 person-days for chunking strategy, permission mapping, evaluation) and ongoing pipeline maintenance (~0.5–1 person-day per month).

Worked example 2: company with 500 employees, 200,000 documents

At 200,000 documents including vectors, you realistically land at 150–250 GB of index volume — with growth headroom that means S1 with 2 partitions. Add 2 replicas for availability and you are at 4 search units.

Then come the classic hidden items that almost never appear in a proposal:

• Re-indexing on schema changes: change your chunking strategy or embedding model and the entire corpus is reprocessed — the $1,500 processing bill comes back.
• Forgotten dev/test indexes: every extra S1 service costs $245 per month — including the one the project team never switched off after the pilot.
• Monitoring and evaluation: if you measure answer quality (and you should), you pay for the evaluation queries too.
• Staff cost: 1–2 person-days per month of pipeline operations is conservative at this size — at CHF 1,200/day that is another CHF 1,200–2,400 monthly, often more than the entire Azure bill.

Annualized: CHF 14,000–17,000 of infrastructure plus CHF 15,000–30,000 of operations — for the index alone, before any business value is created.

The trend is accelerating: Microsoft is shifting to metered billing

On August 7, 2026, TechInformed reported that Microsoft is moving parts of its AI portfolio beyond pure per-seat pricing, pairing user licenses with usage-based charges. In the Dynamics 365 ecosystem, credit consumption rose fourfold within a quarter. For Azure AI Search, serverless billing has been announced for late 2026.

What this means for RAG projects: the cost structure is becoming more volatile. If you have no per-query cost transparency today, you certainly won't have it under consumption billing. A monthly cost dashboard per knowledge agent should now be standard equipment for every RAG project.

The key insight: SharePoint does not need a custom RAG index

Here lies the biggest lever — and the most frequently overlooked one: for content that already lives in Microsoft 365, you do not need to build your own index. Microsoft already runs it.

Concretely:

• Copilot Studio connects SharePoint, OneDrive and Dataverse as native knowledge sources — no Azure AI Search, no embedding pipeline, no re-indexing. Permissions are respected automatically: the agent only surfaces what the asking user is allowed to see. Billing runs on Copilot Credits: $200 per month for a capacity pack of 25,000 credits, or pay-as-you-go with no fixed cost.
• Microsoft 365 Copilot uses the Semantic Index — Microsoft already indexes tenant content as part of the service. There is no separate indexing infrastructure to pay for; you pay the user license (Microsoft 365 Copilot as a business add-on: list price $21 per user per month, annual commitment).
• Microsoft Graph connectors extend that same native index to external sources (Confluence, ServiceNow, file shares) — again without your own Azure AI Search service.

The direct comparison for the most common case — an internal knowledge agent over SharePoint content, 50 users:

The license option looks more expensive at first glance — but it includes Copilot in Word, Excel, Outlook and Teams for all 50 people. The custom RAG index delivers exactly one function and carries the entire operating burden with it.

The decision rule is simple:

• Data lives in SharePoint, OneDrive, Teams, Exchange or Dataverse → native grounding (Copilot Studio / M365 Copilot). No custom index.
• Data lives in ERP systems, line-of-business apps, external databases or public web sources that Graph connectors don't cover → then, and only then, a custom RAG index with Azure AI Search is justified.
• Mixed case: Copilot Studio can combine both — native SharePoint sources plus a (small!) Azure AI Search index only for the non-M365 data. That keeps the expensive index down to the minimum.

Seven levers to cut RAG cost

When a custom index is justified, the bill can almost always be reduced significantly:

1. Size the tier to actual need. Jumping from S1 to S2 quadruples the fixed cost ($245 → $981). S2 "just in case" is the single most expensive mistake; S1 partitions can be added later.

2. Tie replicas to your real availability requirement. An internal knowledge agent rarely needs 99.9% write availability. 1 replica instead of 3 saves $490 per month on S1.

3. Index incrementally instead of full rebuilds. Indexers with change detection process only changed documents — avoiding recurring processing costs (in the example above: $1,500 per full run).

4. Pick the embedding model per task. text-embedding-3-small costs a sixth of large ($0.022 vs. $0.143 per 1M tokens) and is demonstrably sufficient for many internal search scenarios. Measure first, upgrade later.

5. Cache answers. In internal knowledge agents, 20–40% of queries are typically recurring questions. An answer cache saves LLM and ranker cost at the most expensive point.

6. Clean up dev/staging indexes. Every forgotten Basic service costs $74, every S1 $245 per month. A monthly resource review belongs in every Azure governance routine.

7. Use the free quotas deliberately. 1,000 semantic ranker requests and 50M agentic retrieval tokens per month are free — often entirely sufficient for pilots and small agents.

Conclusion: check the data source first, then choose the architecture

The hidden costs of RAG in Azure are real: a productive custom index realistically costs an SME CHF 3,000–6,000 per year and a mid-sized company CHF 14,000–17,000 — plus build and operations that often exceed the infrastructure bill. None of it is hidden if you read the price list. But in many projects, that happens too late.

The good news: the biggest cost block is often entirely avoidable. If your knowledge base lives in Microsoft 365, indexing, permissions and freshness come from Microsoft — included in the license price. The custom RAG index is the right tool for the data that lives outside that world. No more, no less.

How CNEXT can help

We do the math with you before you build:

• RAG cost assessment: we model your scenarios with current list prices — you see what operations will cost before the project starts.
• Architecture decision: native grounding, custom index or a mix — justified by your data sources, not by architecture fashions.
• Cost optimization of existing RAG systems: tier review, caching, indexing strategy — 30–60% savings without quality loss are typical.

Before you build an index that shouldn't cost anything: Get in touch →

Sources (all retrieved August 2026): Azure AI Search / Foundry IQ pricing · Microsoft Learn: SKU selection · Azure OpenAI pricing · Microsoft 365 Copilot pricing · Copilot Studio pricing · TechInformed: Microsoft adds metered AI billing (Aug 7, 2026)

Further reading:

• What Does a Copilot Studio Agent Really Cost? Credits, Baselines and Cost Levers
• What Does Microsoft 365 Copilot Really Cost? Licenses, Baselines and Cost Levers
• What Does an Azure AI Foundry Agent Really Cost? Components, Baselines and Cost Levers
• Building a Company Brain: Copilot Studio, Azure AI Foundry or Replit?
• AI Company Brain: Architecture, Components and Data Sources
• Agentic AI services