Milvus Documentation AI Assistant: How Milvus Powers RAG for Its Own Technical Ecosystem

1. Executive Summary

Zilliz and Inkeep built a retrieval-augmented generation (RAG) AI assistant for Zilliz and Milvus technical documentation. The system addresses a practical developer-experience problem: documentation becomes harder to navigate as features, versions, APIs and examples grow, while conventional keyword search can struggle with nuanced technical questions. In the documented architecture, Inkeep handles ingestion and generation, while Zilliz supports indexing and retrieval; Milvus is the vector database used to store embeddings and retrieve relevant context.

The pipeline collects technical documentation, support/FAQ material and GitHub content, cleans and chunks the text, attaches metadata, generates sparse and dense embeddings, stores those representations in Milvus, performs hybrid retrieval, reranks candidates, and sends selected context with the user's question to an LLM. The published implementation names BM25 and SPLADE/BGE-M3 for sparse retrieval, and MS-MARCO, MPNET and BGE-M3 for dense embeddings. OpenAI and Anthropic models are identified for final generation. Primary case source.

What is not publicly disclosed: this case does not publish vector count, QPS, p50/p95 latency, recall, infrastructure cost, conversion uplift or revenue impact. The report therefore does not fabricate those metrics. The verified outcome is primarily a product/UX result: developers can ask natural-language questions and receive contextual answers with citations or relevant pages.

2. Company Background

Milvus supports dense and sparse vectors, metadata filtering, ANN search and hybrid retrieval. Its architecture separates access, coordination, worker nodes and storage, enabling distributed scaling. Milvus architecture documentation.

3. The Business & Technical Problem

Technical documentation becomes harder to navigate as a product accumulates APIs, configuration options, versions, examples, GitHub code and support material. The case specifically describes nuanced questions such as combining sparse and dense vectors during retrieval or dynamically scaling a cluster. Basic keyword search can fail to understand the user's intent and context.

• Information fragmentation: answers can be distributed across docs, FAQs, support material and repositories.
• Semantic complexity: developer questions may use different wording from the documentation.
• Version/context requirements: the right answer may depend on version, source, code language, URL/path or date.
• LLM grounding: RAG provides current product-specific context to the generator.

4. Why the Company Chose/Used Milvus

The published implementation needed a vector database for RAG indexing and retrieval. Milvus supports dense and sparse representations and hybrid search, matching the design requirements described by Inkeep and Zilliz.

Alternatives: the case does not document a formal procurement comparison against Pinecone, Qdrant, pgvector, Elasticsearch or Weaviate. They must not be presented as evaluated-and-rejected alternatives.

5. AI Use Case

RAG-powered technical documentation assistant

A developer asks a natural-language question. The system retrieves relevant technical material, then an LLM generates an answer grounded in that context. The assistant can provide citations or relevant documentation pages for verification.

6. AI Architecture

1. Ingestion

Inkeep collects technical documentation, support/FAQ material and GitHub repository content. Data is cleaned and chunked. Metadata includes source type, version/record type, text/code information, programming language, hierarchy, URLs/tags/paths and dates.

2. Embeddings

The case source does not disclose the production vector dimensions or exact model versions.

3. Milvus retrieval

Embeddings are stored in Milvus. Query embeddings are compared with stored vectors. Dense and sparse retrieval can be combined. Milvus supports multiple vector fields, sparse/dense hybrid search and metadata filtering.

4. Reranking

The implementation describes weighted scoring and reciprocal rank fusion (RRF) to combine candidate results from different retrieval signals.

5. Generation

The original query plus retrieved context is sent to an LLM. The case identifies OpenAI and Anthropic models. Answers can include citations/relevant documentation pages.

7. Milvus Implementation

Milvus currently offers Lite, Standalone and Distributed deployment options. Distributed is designed for Kubernetes and horizontal scaling, but this does not prove which deployment mode the assistant uses.

8. Data Scale

9. Performance & Business Impact

The source does not provide a controlled before/after benchmark or financial ROI for this specific assistant.

The case describes responses as arriving “in seconds,” but does not publish a measured p50/p95 latency benchmark. This wording should not be converted into a fabricated performance statistic.

10. Why Milvus Matters

The LLM is not the retrieval system. Milvus supplies the retrieval layer that gives the LLM product-specific evidence.

• Vector similarity: finds semantically related chunks even when wording differs.
• Sparse retrieval: preserves exact terminology.
• Hybrid search: combines semantic and lexical signals.
• Metadata filtering: constrains retrieval using attributes.
• Distributed architecture: supports horizontal scaling for larger workloads.

11. Technical Challenges

12. Technical Lessons

1. Do not treat an LLM as the authoritative database of current product knowledge.
2. Semantic and exact-term retrieval are complementary.
3. Metadata such as version, source, path and dates matters.
4. Hybrid retrieval needs a ranking strategy.
5. Evaluate the entire RAG stack: ingestion, chunking, embeddings, retrieval, reranking and generation.
6. Keep citations visible for technical/professional applications.

13. Business Lessons

• Infrastructure affects UX: users do not see Milvus, but retrieval quality affects assistant usefulness.
• Dogfooding can demonstrate product value: Milvus is used in an assistant for Milvus documentation.
• Developer experience is a growth lever: reducing documentation friction can help users learn a complex platform.
• Do not confuse technical capability with ROI: the public case documents architecture and UX, not revenue uplift.

14. Alternatives & Competitive Context

The following are architectural alternatives, not technologies that the case claims were evaluated.

15. Case Study Timeline

16. Key Numbers

17. Executive Summary in 8 Points

• Milvus is the vector retrieval layer, not the LLM.
• The self-use example involves Zilliz/Milvus documentation and Inkeep.
• Inkeep handles ingestion and generation; Zilliz supports indexing/retrieval.
• The system combines sparse and dense representations.
• BM25, SPLADE/BGE-M3, MS-MARCO, MPNET and BGE-M3 are named in the source.
• Milvus stores embeddings and supports hybrid search.
• Weighted scoring and RRF are used for reranking.
• OpenAI and Anthropic models are identified for generation; production scale and ROI are not publicly disclosed.

18. Sources

1. Zilliz — Inkeep Built an Efficient RAG-driven AI Assistant using Milvus
2. Milvus Documentation — Architecture Overview
3. Milvus Documentation — Hybrid Search with Milvus
4. Milvus Documentation — What is Milvus
5. Milvus Documentation — Deployment Options
6. Milvus: A Purpose-Built Vector Data Management System — SIGMOD 2021
7. Zilliz — How Zilliz Built Vector Databases for Production AI

Research standard: primary/official sources were prioritized. General Milvus capabilities are not presented as proof of the specific deployment configuration. Missing figures are explicitly marked “Not publicly disclosed.”