Check what the AI did and what it cost
The AI runs log shows every answer, draft, hand-off and test the assistant made in the last 30 days, with its outcome and cost.
Updated on
On this page
Every time the AI does something for your business, Ha Tang records it: the customer message it answered, what it decided, how long it took and what it cost. Use the log to understand a strange answer, to see why the assistant handed off, and to keep an eye on spending.
The summary on the Assistant tab
The top of Customer Support → AI assistant shows four figures:
| Card | What it counts |
|---|---|
| Spent today / month | What the AI cost today and this month, with your daily and monthly caps underneath |
| Runs, 30 days | Every AI call in the last 30 days, split into replies and drafts |
| Hand-offs | Conversations handed to your team, with how many runs were stopped by the caps and how many failed |
| Cost, 30 days | The total of the last 30 days and the average time per answer |
The AI runs log
Open the AI runs tab. It lists the latest runs, newest first. Runs are kept for 30 days.
| Column | What it shows |
|---|---|
| When | Date and time |
| Kind | What the run was for: respond (an answer to a customer), draft (a suggestion for an agent), test (the Chat with the AI panel), eval (a test on the Tests tab), followup, workflow, comment or tagging |
| Outcome | Replied, Draft, Email request (it showed the email form), Handoff, Skipped, Capped (the budget was used up) or Error, with the error underneath |
| Conversation | A link to the conversation, or "test" for runs made from the dashboard |
| Reply | The beginning of what it wrote. Actions appear in brackets, for example [handoff_to_human: not_in_knowledge] or [show_products] |
| Tokens | The size of the request and of the answer, and how much was reused from cache (which costs less) |
| Cost | The cost of this run in US dollars |
| Latency | How long the model took |
Use it to fix problems
- A wrong answer: open the conversation from the log, check Knowledge used under the reply, and correct it with Improve this reply. See Answer the questions the AI could not.
- Many hand-offs with
not_in_knowledge: answer the questions in Unanswered questions. - Many hand-offs with
max_turns: customers need more back and forth than the limit allows; raise Max assistant replies before hand-off. - Runs marked Capped: the daily or monthly cap was reached and the assistant stopped answering. Raise the caps in Team budget and key, or wait for the next day or month.
- Runs marked Error: the message under the outcome says why. "no API key" means neither the platform nor the business has a key for the model.
Keep costs predictable
- Set a Daily cap that matches a busy day, and a Monthly cap that matches your budget. Owners and admins get an email the first time a cap is hit each day.
- Tests and the Chat with the AI panel cost money too. They appear in the log with the kinds
evalandtest. - AI follow-ups, AI comment replies and the AI writes a message workflow step count toward the same caps. If spending jumps, the Kind column shows which one grew.
- Answers reuse the fixed part of the request from cache, which is why the Tokens column often shows a cached share.
For charts of AI activity over time, add the AI assistant and AI by day widgets to a dashboard. See Custom dashboards.
Was this article helpful?
Related articles
- Test the AI before customers doWrite the questions that matter, say what a good answer does, and let Ha Tang grade the assistant on them after every change.
- Set up the AI assistantAdd knowledge, try the assistant, choose when it answers and give it a name and tone, in about fifteen minutes.
- Custom dashboards and reportsBuild pages of widgets from every module, such as sales, support, payments and email, for your team's daily view and for period reports you can print.
Not finding what you are looking for?
Our team answers in the chat, in English or Vietnamese.