# Test the AI before customers do

Write the questions that matter, say what a good answer does, and let Ha Tang grade the assistant on them after every change.

The **Tests** tab checks the assistant against questions your customers really ask. You write the question and what a good reply does; Ha Tang asks the assistant three times and grades each answer from 0 to 100. Run the tests after you change the knowledge, the instructions or a situation, and you will know whether you made things better or worse.

## Add a test

1. Go to **Customer Support → AI assistant → Tests**.
2. Under **Add a test**, fill in:
   - **Name**: a short label, for example "Shipping fee to Da Nang".
   - **Channel** and **Office hours**: the context to pretend, for example **Zalo OA** and **Outside office hours**.
   - **Earlier chat** (optional): previous messages, one per line, starting with "Customer:" or "AI:".
   - **Customer says**: the message to answer.
   - **Expected**: what a good reply does, in plain words. "States the shipping fee and time to Da Nang; promises no exact date" or "Apologizes and hands off to a person".
3. Click **Add**.

![Three tests for a tea shop: shipping fee, an out-of-season product and a refund request](https://hatang.dev/help/img/en/ai/evals.webp)

## Run the tests

Click **Run all** at the top, or **Run** under one test. Tests show **Queued**, then **Grading**; reload the page after a few minutes to see the results.

For each test you get:

- **the score**, the average of the three grades, in green from 80, orange from 50, red below;
- **the spread**, the gap between the best and the worst of the three answers. A spread of more than 30 points means the assistant is not consistent yet on this question;
- **See 3 answers**: each reply with its grade and a one-sentence reason from the grader.

The page also shows the average score over all tests.

The assistant answers tests exactly as it would answer a customer: same knowledge, instructions, situations, products and tools. A hand-off counts as right only if the expectation asks for one, or the question really needs a person.

> [!NOTE]
> Each test costs three answers and three gradings, and counts toward your AI budget. Ten tests run after each change is a good routine; running them in a loop is not.

## Write good tests

- **Start from real questions**, from **Unanswered questions** and from conversations agents had to correct.
- **One thing per test.** "Gives the right fee" is easy to grade; "gives the fee, the time, the return policy and a discount code" is not.
- **Test the hand-offs too**: refunds, complaints, a customer asking for a person. The expectation is "hands off to a person, without promising a refund".
- **Test what must not happen**: "does not invent a delivery date", "does not offer a discount".

## What to do with a low score

| What you see | What it usually means | What to do |
|---|---|---|
| Low score, small spread | The knowledge is missing or wrong | Fix the source, or answer the question in **Unanswered questions** |
| Good average, large spread | The knowledge is vague or contradicts itself | Make the answer one clear Q/A pair; remove the old version |
| It hands off when it should answer | The passage is not found | Use the customer's words in the question; check **Knowledge used** in **Chat with the AI** |
| It answers when it should hand off | Your instructions or a situation push it to keep going | Add a situation with **Hand off to an agent** |