The Ultimate Guide to Agentforce Testing Center & Custom Evaluations

💬 In plain words: Testing Center is your dedicated AI validation hub. Located directly inside Agentforce Studio (right next to Agent Builder and Observability), it replaces manual trial-and-error testing. It can automatically generate test cases, simulate full multi-turn conversations using different user personas, and mathematically score the results against rules you define.

⚡ Key Points

• Location: Testing Center is a native, dedicated tab in Agentforce Studio, completely removed from traditional Salesforce Setup.
• Expected vs. Actual: Every test case requires you to define an expected subagent and an expected action, allowing for direct comparison.
• Scale with AI: Instead of writing hundreds of tests by hand, use AI-generated cases to instantly build synthetic interactions.
• Custom Evaluations: Score your agents using strict deterministic rules (Strings/Numbers) or an LLM-as-a-Judge.
• Cost Awareness: Running these automated test suites consumes Einstein Requests, which are tracked in your Digital Wallet.

🗺️ Module Map

MODULE 11 root: 'How do you prove it works, repeatably?'
├─ 11.1 Why agent testing is statistical
├─ 11.2 Testing Center and custom evaluations 📍 (Current)
└─ 11.3 The noise floor

📌 Real-Life Example:

Meridian, an AI developer, built a robust test suite containing:

• 12 routing test cases (ensuring the correct subagent picks up the prompt).
• 10 adversarial test cases (ensuring the agent refuses malicious requests).
• 3 custom evaluations:
  1. Checking that a specific legal disclaimer string appears verbatim.
  2. Checking that system latency stays below a specific numeric threshold.
  3. An LLM-as-a-judge checking if the response sounds "polite and empathetic" based on a plain English prompt.

⚙️ Core Concepts of Testing Center

Testing an autonomous agent requires a fundamentally different toolkit than testing standard Apex code. Here is how Salesforce has structured the Testing Center:

• Expected Outcomes: You don't test for exact sentences. Test cases carry an expected Subagent and an expected Action. Results dashboard shows expected vs. actual side-by-side.
• AI-Generated Cases: Bootstrapping a test suite is hard. Agentforce can generate large volumes of synthetic interactions based on your instructions to establish a wide testing baseline.
• Persona-Based Testing: Users don't talk in single, perfectly phrased utterances. Persona testing simulates full, multi-turn conversations acting as a specific type of customer, catching context-loss errors that single-line tests miss.
• Custom Evaluations: You are not locked into standard pass/fail metrics. You can create evaluations in two primary forms:
  • Deterministic: string_comparison (contains, equals, starts with, ends with) and numeric_comparison (equals, greater than, less than).
  • LLM Judge: Describe your grading rubric in plain English, and a secondary model will evaluate the primary agent's response.
• CI/CD Integration: Tests can be downloaded via the Salesforce CLI and executed from the command line, allowing you to use them as quality gates in your deployment pipeline.

🧭 360 Card — Testing Center and Custom Evaluations

Rule: Build routing cases, adversarial cases, and custom evaluations together inside Agentforce Studio, then automate their execution from your CI/CD pipeline.
Gain: You catch AI hallucinations and routing regressions safely in the sandbox before they reach your customers.
Reach for: AI-generated cases for rapid breadth, and persona testing for complex, multi-turn behavior that you could never accurately write by hand.
Price: Test runs actively consume Einstein Requests. A massive test suite run frequently will have a tangible ongoing cost tracked in Digital Wallet.
Limits: Custom evaluations are currently limited to string comparisons, numeric comparisons, or an LLM judge.
Mirror (The Wrong Way): Relying on manual spot-checking by typing into the chat preview. It's fast today, but provides zero safety net when the system quietly degrades tomorrow.

🚨 Warning Trap: Unchecked Suite Sizes

Because AI can generate hundreds of synthetic test cases for you in seconds, it is tempting to run massive suites on every minor save. Remember that executing these tests triggers live LLM calls. If you aren't careful, you will burn through your organization's Einstein Request allocation simply by running excessive tests. Size your suites deliberately.

🗣️ Core Q&A

🎯 Say this first: Testing Center provides repeatability and scale. It allows us to set expected subagents and actions, leverage AI to generate bulk synthetic cases, simulate multi-turn personas, and score the agent using custom evaluations.

Q: What does Testing Center give you beyond manually typing messages into the Agent Builder preview?

A: The core difference is repeatability, breadth, and objective measurement.

• Manual testing is anecdotal. Testing Center lets you bind an expected subagent and expected action to every case, providing a mathematical "expected vs. actual" scorecard.
• It allows you to use AI to generate synthetic test cases, giving you a breadth of testing you would never have the time to type out by hand.
• Persona testing simulates full, conversational interactions (multi-turn), which is crucial for finding out where an agent "forgets" context.
• It allows for custom evaluations—like checking that latency stayed under a certain limit or that a required legal string was returned verbatim.

Q: How exactly does an "LLM Judge" evaluation work?

A: Instead of writing a rigid true/false code block, you write a prompt that acts as a grading rubric (e.g., "Score this response a 1 if it is aggressive, and a 5 if it is empathetic"). Salesforce uses a backend LLM to review the agent's output against your rubric and assign a score. This is ideal for testing tone, summarization quality, or conversational flow where exact string matching is impossible.

Q: Can I integrate Testing Center into my DevOps process?

A: Yes. The real power of Testing Center is that tests can be downloaded and executed via the Salesforce CLI. This means you can add agent testing as a required quality gate in your Continuous Integration (CI) pipeline, preventing bad agent configurations from ever being deployed to production.

🔗 Connecting the Dots: Follow-Up Question

"Now that we know how to build a robust suite in Testing Center, how do we establish a 'Noise Floor' so we know if a failed test is a true code regression or just standard AI variance?"

(We will cover the critical concept of the Noise Floor in Module 11.3)