How accurate are Corebee AI answers?

Before you switch the agent on for customers, the account owner can run a test from the Lab. It replays up to 30 test questions through your agent, using your agent's model, settings and knowledge base search.

• Test questions come from your own resolved conversations from the last 90 days, or you add them one at a time.
• For each answer, a separate AI judge compares the answer with the knowledge base passages the agent was given and scores how well the answer is supported by them.
• The result shows which questions the agent would answer on its own and which it would pass to a person.
• The test runs on its own copy of the question. Nothing is sent to your customers.

For every question, Corebee searches your knowledge base: the documents, website pages, files and answers you have added. The agent then answers from what it finds, under these instructions:

• Only use knowledge base passages that directly address the question that was asked.
• When your knowledge base and the AI model's general knowledge disagree, your knowledge base wins.
• Do not fall back on general knowledge for questions about your business.
• Do not link to a page unless the knowledge base shows that the page exists.

In the website chat widget, an answer drawn from your knowledge base shows the sources it used as small numbered links with the source title, so you and your customer can check where it came from. Sources are only shown when the answer used them or they closely matched the question.

When the knowledge base does not cover a question, the agent is told to say it does not have that detail and give the customer the way to reach your team. Its instructions put it plainly: a confident wrong answer is worse than an honest “I don't have that detail.”

A customer can ask for a person at any point in the chat. The agent then hands the conversation to your team instead of continuing to answer on its own.

After a chat answer, Corebee also checks for signs the question was not really answered: a weak match in the knowledge base, a low-confidence answer, or a reply that reads like “I don't know”. A second AI check confirms it, and the question goes into your answer-gap list, grouped with other wordings of the same question and showing how many times it was asked.

• From the answer-gap list. Write the answer to an open question. Corebee saves it as a new knowledge base entry and reads it in. The question shows as learning until the entry is live, then as resolved, and the agent uses the new entry in later conversations.
• From the pre-launch test. On a test question the agent got wrong, choose “Fix this answer”. Corebee drafts an entry from the question and the current answer, you correct it and accept, and that question re-runs straight away with your fix. The entry is added to your knowledge base for future conversations.
• In your knowledge base. Your knowledge base is the source of truth. Edit or add a document and later answers follow it.

• We do not publish a hallucination rate or an accuracy percentage. The 86% figure is a resolution rate: the share of conversations the AI resolves without a person stepping in. It is not a count of correct answers, and results depend on how complete your knowledge base is.
• The AI can still be wrong. The judge in the pre-launch test is also an AI and can miss a mistake. If the judge is unavailable for a question, that question is not graded for grounding.
• The pre-launch test checks up to 30 questions per run. It is a sample of how your agent answers, not proof about every possible question.
• Source links appear in the website chat widget only, and not on every reply. Greetings, short follow-ups and some saved instant replies show none.
• Answers are only as good as your content. Outdated or contradictory documents lead to outdated or contradictory answers.
• The answer-gap list shows questions the agent could not answer. We do not quote a coverage percentage from it.
• The pre-launch test and “Fix this answer” are available to the account owner.

For security, privacy and compliance facts, see the Trust center and security practices.