Semantic Search & RAG: How Your AI Understands Before Answering

Ask ChatGPT something about your company. About your own articles. About your products, pricing or internal policies. The answer will be polite, elaborate… and quite likely made up. Language models know nothing about your business — they know about everything else.

RAG (Retrieval-Augmented Generation) solves exactly this. Instead of expecting the LLM to "remember" your content, you first search for relevant information in your own knowledge base and then pass it to the model as context. The result: precise answers, with sources, free of hallucinations.

In this article we explain the entire pipeline — from what embeddings are to how we implemented the Codex chatbot you're using right now (yes, the button at the bottom right). Feel free to try it while you read.

An embedding is the numerical representation of a text's meaning. Instead of treating words as character strings, an embedding model converts them into vectors — lists of numbers with hundreds or thousands of dimensions — where texts with similar meanings end up close together in vector space.

"cat"        → [0.23, -0.41, 0.87, 0.12, ..., -0.33]  // 1024 dimensions
"feline"     → [0.25, -0.39, 0.85, 0.14, ..., -0.31]  // Very close to "cat"!
"automobile" → [-0.67, 0.52, -0.11, 0.83, ..., 0.44]  // Far from both

// Cosine distance:
// cat ↔ feline:     0.97 (nearly identical semantically)
// cat ↔ automobile: 0.12 (no semantic relationship)

The magic is that this works beyond synonyms. "How do you protect my customer data?" and "CRM security and compliance" end up close in vector space even though they share almost no words. The model understands the intent, not the letters.

Traditional search (SQL LIKE '%keyword%' or full-text search) looks for exact word matches. It works well when you know exactly which words to use. But humans don't think in keywords — we think in questions.

RAG (Retrieval-Augmented Generation) is an architecture pattern that combines two phases: first you retrieve relevant information from your knowledge base, then you pass it to the generative model as context. The model doesn't need to "know" your data — it just needs to read it at the right moment.

There are three ways to "teach" an LLM about your business. Each has its place, but for most enterprise use cases, RAG is the right choice.

A vector database stores embeddings and lets you search for the most similar ones to a query vector. The standard algorithm is ANN (Approximate Nearest Neighbors) — it doesn't compare against every vector one by one (that would be way too slow with millions), but uses index structures to find the closest ones in milliseconds.

Cloudflare Vectorize is Cloudflare's native solution. It lives at the edge (alongside Workers and D1), has a generous free tier, and integrates directly with Workers AI for generating embeddings — without leaving the ecosystem, without inter-service network latency.

[[vectorize]]
binding = "VECTORIZE"
index_name = "codex-knowledge"  # 1024 dimensions, cosine
# Free tier: 200K vectors, 5M dims stored/month
# Enough for ~195 articles × ~30 chunks = ~5,850 vectors

You can't generate an embedding of a full 4,000-word article — the meaning gets diluted. You need to split it into fragments (chunks) small enough to be semantically precise, but large enough to retain context.

The chatbot you see on this website is a complete RAG system running 100% on Cloudflare. Here's each component and why we chose it:

// 1. Embed the question (~50ms)
const embedding = await env.AI.run('@cf/baai/bge-m3', {
  text: [question]
});

// 2. Vector search in Vectorize (~30ms)
const results = await env.VECTORIZE.query(embedding.data[0], {
  topK: 5,
  namespace: lang,         // 'es' or 'en'
  returnMetadata: 'all',
  filter: { minScore: 0.55 }
});

// 3. Fetch full text from D1
const chunks = await db.prepare(
  'SELECT chunk_text, title, url FROM codex_chunks WHERE id IN (...)'
).all();

// 4. Generate response with DeepSeek
const response = await fetch('https://api.deepseek.com/chat/completions', {
  body: JSON.stringify({
    model: 'deepseek-chat',
    messages: [
      { role: 'system', content: systemPrompt + context },
      ...history,
      { role: 'user', content: question }
    ]
  })
});

One of RAG's biggest advantages over other architectures is cost. By using services with free tiers for embeddings and vector storage, the only variable cost is the generative LLM. Here's the real breakdown:

A RAG system is only useful if the interface is good. The Codex chatbot includes features that go beyond a simple text field:

The Codex pipeline currently indexes 24 articles. But the architecture is designed to scale several orders of magnitude without changing technologies:

Vectorize uses HNSW (Hierarchical Navigable Small World) as its indexing algorithm — the same family of algorithms used by Pinecone or pgvector. Search latency grows logarithmically, not linearly: going from 100 to 100K vectors barely adds 10ms.

RAG is not a trend — it's the pattern that solves the fundamental problem of LLMs: they know a lot about everything, but nothing about your business. With embeddings, semantic search and generation based on real context, you can have an assistant that answers precisely about your content, cites sources and costs less than $1/month.

The Codex chatbot is the living proof. Ask it anything about Cadences — it searches 98 chunks from 24 articles, finds the most relevant ones by semantic similarity, and generates a grounded response with DeepSeek. All at the edge, all serverless, all for ~$0.0008 per question.