Agentforce Data Library: Search Index & Retrievers Explained

💬 In plain words: There are three main terms people mix up when building AI in Salesforce. The Data Library is the storehouse holding your files. The Search Index makes those files searchable. The Retriever actually runs the search. The best part? Creating a Data Library automatically builds the other two for you.

🔑 Key Points

• Data Library: A centralized repository in Data Cloud for unstructured grounding content (like PDFs or Knowledge Articles).
• Search Index: Translates documents into searchable formats. Salesforce offers two types: Vector (matches on meaning) and Hybrid (matches on meaning + exact keywords).
• Retriever: The engine that executes the search. It fetches context to ground your AI prompts.
• Citations: AI source citations are configured at the Retriever level, not on the Agent itself.
• Automation First: Creating a Data Library automatically generates data streams, data model mappings, indexes, and a default retriever, saving you days of manual work.

🎬 Real-Life Example: The Week Spent Building What Was Already There

The Manual Trap (Old Way): A development team spent a full week manually building a Search Index, configuring a Retriever, and writing complex mapping logic to ground their Agentforce bot.

The Automated Path (New Way): That entire week of work is now obsolete. When you upload files to a Data Library, Salesforce automatically generates the data streams, mappings, a hybrid search index, and a retriever. You should only use the manual path if you need highly granular control over document chunking, specialized parsing, or customized field returns.

🏗️ Architecture Hierarchy

Here is how these three components connect inside Salesforce Data Cloud:

Data Library (The Storehouse)
 └─ Auto-generates ↓
    ├─ Search Index
    │   ├─ Vector Search (Matches by meaning/embeddings)
    │   └─ Hybrid Search (Meaning + exact keywords) ← DEFAULT
    │
    └─ Retriever
        ├─ Individual Retriever (Searches 1 index / 1 object)
        ├─ Ensemble Retriever (Searches multiple sources)
        └─ *NOTE: Citation settings live HERE on the Retriever

🧠 Core Takeaway: Let the library do the heavy lifting. Always create the Data Library first, review what it automatically built, and only resort to hand-building components if your retrieval accuracy scores are poor.

⚠ INTERVIEW TRAP: Do not describe "Keyword Search" as its own distinct index type in Salesforce Data Cloud. There are only two index types: Vector and Hybrid. Keyword matching (BM25) is simply one half of the Hybrid index equation.

🧭 360 Card: Data Library Strategy

• Rule: Take the fast path first. Point a Data Library at your files, let it auto-generate the architecture, test it, and only customize manually if it fails to fetch the right answers.
• Gain: You save days of implementation time and get a highly optimized Hybrid Search setup by default.
• Reach for manual when: You absolutely need custom document chunking strategies, strict metadata filters, or specific field return payloads.
• Limits: An Individual Retriever only covers a single search index (which maps to a single Data Model Object). Web search data libraries also require Enterprise or Unlimited editions plus specific Einstein add-on licenses (meaning you won't see them in a standard Developer org).

💡 Core Q&A

Q: How would you set up document grounding for an Agentforce Agent?

🎯 Say this first: I would start by creating a Data Library. It automatically handles the heavy lifting by generating a Search Index and a Retriever using Hybrid Search by default.

Here is the exact workflow for success:

• Step 1: Create the Data Library. This pushes your content into Data Cloud, creates data objects, and generates both a hybrid search index and a default retriever.
• Step 2: Inspect the auto-generated components. Check how the documents were chunked.
• Step 3: Test retrieval accuracy by running about ten real-world questions against the retriever before attaching it to an agent.
• Step 4: If the accuracy score is poor, you can then hand-build a custom retriever. Building manually allows you to add strict metadata filters (e.g., filtering by region or product category).

Q: Why are Metadata Filters on the Retriever so important?

Filters serve two critical purposes. First, they improve relevance (so a user asking about a "metering issue" doesn't get a "billing department" document). Second, they enforce data boundaries, ensuring users do not retrieve data meant for a different department or geography.

Q: Where do you turn on AI Citations so users can see where the Agent got its answer?

This catches many developers off guard: Citation toggles are configured on the Retriever settings, not on the Agent itself. The retriever is responsible for passing the source document metadata (like the file name or URL) alongside the text chunk to the LLM.