RAG (Retrieval-Augmented Generation) connects LLMs with your enterprise knowledge. Azure AI Foundry provides all the tools for this.

What is RAG?

RAG extends LLMs with external data:

1. 1User question
2. 2Search in knowledge base
3. 3Create context for LLM
4. 4Answer with current knowledge

Benefits:

• Current information
• Source citations possible
• Fewer hallucinations
• Domain-specific knowledge

Architecture in Azure

Components

Documents → Azure AI Search (Index) → User Question → Vector Search → Relevant Chunks → LLM (Azure OpenAI) → Answer

Azure AI Search

Powerful search:

• Vector search for semantics
• Hybrid search (vector + keyword)
• Facets and filters

Embedding Models

Convert texts to vectors:

• text-embedding-ada-002
• text-embedding-3-small/large
• Multilingual models

Implementation Steps

1. Data Preparation

Prepare documents:

Chunking Strategy:

• Fixed size (e.g., 500 tokens)
• Semantic chunks
• Overlap for context

Metadata:

• Source
• Date
• Category
• Permissions

2. Create Index

Configure fields in Azure AI Search for ID, content, vector, and source with appropriate search profiles.

3. Generate Embeddings

Create embeddings for each chunk with models like text-embedding-3-small.

4. Retrieval Logic

For user queries, embed the question and search the index with vector queries.

5. Prompt Assembly

Assemble context with found chunks and system prompt for the LLM.

Best Practices

Improve Quality

• Reranking – Resort results
• Query Expansion – Add synonyms
• Feedback Loop – Learn from ratings

Scaling

• Incremental updates
• Batch processing
• Cache frequent queries

Optimize Costs

• Smaller models for embedding
• Efficient chunking
• Set token budgets

SharePoint Integration

Particularly relevant for M365 customers:

1. 1Graph Connector for SharePoint
2. 2Automatic synchronization
3. 3Consider permissions
4. 4Use metadata

CNEXT Expertise

We offer:

• RAG architecture design
• Data pipeline development
• SharePoint integration
• Performance optimization

Conclusion

RAG turns LLMs into true enterprise assistants. Azure AI Foundry provides the platform – CNEXT the expertise.