How AI Chatbots Use Knowledge
Modern AI support chatbots use Retrieval-Augmented Generation (RAG) to answer questions. Here is the simplified flow:
- A customer asks a question
- The system converts the question into a vector embedding — a numerical representation of the meaning
- The system searches your knowledge base for the most semantically similar content
- The most relevant articles or passages are retrieved
- The AI model reads the retrieved content and generates a natural-language answer
- The answer is delivered to the customer
The critical implication: the AI can only answer questions about topics that exist in your knowledge base. If a topic is not covered, the AI either says "I do not know" (good) or makes something up (bad — this is called hallucination). Your knowledge base is the boundary of your AI's capability.
Key insight: The AI cannot answer what is not in your knowledge base. Every gap in your content is a question the AI will either dodge or hallucinate on.
What Content to Include
Tier 1: Must-Have (Cover These First)
Product how-to guides — Step-by-step instructions for every core feature. Cover the happy path first, then common variations. Include screenshots or specific UI references when helpful.
FAQ answers — Your existing FAQ page is the lowest-hanging fruit. Convert each Q&A into a knowledge base article with enough context for the AI to answer follow-up questions.
Billing and pricing information — Plan details, pricing, billing frequency, refund policy, upgrade/downgrade process, payment methods accepted. Billing questions are high-volume and high-sensitivity.
Troubleshooting guides — For every common issue, document: symptoms, likely causes, and step-by-step resolution. Cover the top 20 support tickets from the past quarter.
Getting started / onboarding — New customer questions are among the most common. Document the setup process, first steps, and common initial questions.
Tier 2: High-Value Additions
Integration guides — How your product connects to other tools. Cover setup steps, common configurations, and troubleshooting.
API documentation — If you have a developer audience, include API docs, code examples, and common error messages with explanations.
Policy documents — Terms of service, privacy policy, data handling practices, SLA commitments. Customers ask about these more than you think.
Release notes — Recent product changes, new features, and known issues. Keep the AI current with what your product does today.
Tier 3: Competitive Advantage
Use case guides — How different customer segments use your product. "How marketing teams use [Product]" or "Setting up [Product] for e-commerce."
Best practice guides — Expert-level content that goes beyond basic how-to. This positions the AI as a knowledgeable advisor, not just a FAQ machine.
Comparison pages — How your product differs from alternatives. When a customer asks "Can your product do X like [Competitor]?", the AI has an accurate answer.
How to Structure Knowledge Base Articles
Write for Retrieval, Not Just Reading
Traditional help articles are written for humans who read from top to bottom. AI-optimized articles need to work for both humans and the retrieval system.
Clear, specific titles — "How to Export Data to CSV" is better than "Data Export Options." The title should match how a customer would phrase the question.
Front-load the answer — Put the direct answer in the first paragraph. The AI retrieval system often pulls the most relevant section, not the entire article. If the answer is buried in paragraph 5, retrieval quality drops.
One topic per article — An article covering "Billing, Pricing, and Account Management" will be retrieved for all three topics but provide diluted answers for each. Split it into three focused articles.
Use headers and structure — Headers help the retrieval system identify which section of a long article is relevant. Use descriptive headers like "Step 3: Configure Notification Settings" rather than "Step 3."
Include variations of common phrasing — Customers ask the same question in different ways. "How do I cancel?" / "How to end my subscription" / "I want to stop my plan." Include natural variations in your content so the retrieval system matches regardless of phrasing.
Article Template
Use this structure for consistency:
- Title — Clear, question-matching title
- Summary — One-paragraph answer to the core question
- Prerequisites — What the customer needs before starting
- Steps — Numbered, specific steps with UI references
- Troubleshooting — Common issues and their solutions
- Related topics — Links to related articles
Training the AI: Step-by-Step
Step 1: Audit Existing Content
Gather all existing support content: help center articles, FAQ pages, internal documentation, canned responses, email templates. Assess each piece for accuracy, completeness, and relevance.
Step 2: Identify Gaps
Compare your content against your top 50 support ticket topics. For each topic, ask: is there an article that fully answers this question? Common gaps include:
- Topics covered superficially (the article exists but does not have enough detail)
- Recent features with no documentation
- Edge cases and error states
- Billing and account management procedures
Step 3: Write and Import Content
Write articles for every gap. Use the structure template above. Import all content into your knowledge base platform. In Corebee, navigate to Knowledge Base > Import to bulk-import existing content or create articles directly in the editor.
Step 4: Test with Real Questions
Before launching, test the AI with 50-100 real customer questions from your recent ticket history. For each:
- Did the AI find the right article?
- Was the AI's answer accurate and complete?
- Did the AI hallucinate or make anything up?
Track accuracy as a percentage. Target 85%+ accuracy before launching. Below that, your knowledge base needs more work.
Step 5: Launch and Monitor
Go live and monitor closely for the first two weeks. Review AI conversations daily:
- Correct answers — The system is working. No action needed.
- Incorrect answers — Trace back to the knowledge base. Is the article wrong, incomplete, or missing?
- "I do not know" answers — The topic is not in the knowledge base. Create an article.
- Hallucinated answers — The AI made something up. This usually means the retrieval found a partially relevant article and the AI filled in gaps. Improve the source article.
Maintaining Knowledge Quality Over Time
Monthly Review Cycle
Schedule a monthly knowledge base review:
- Pull the list of AI "I do not know" responses from the past month — these are your content gaps
- Pull the list of incorrect AI answers — these are your content quality issues
- Check for outdated content (product changes, pricing changes, policy updates)
- Review auto-resolution rate trends — declining rates often indicate knowledge base degradation
Feedback Loop from Agents
When a human agent handles a ticket that the AI could not, ask: why did the AI fail? Create a process for agents to flag knowledge gaps directly from the inbox. In Corebee, agents can click "Flag for KB" on any conversation to create a knowledge base improvement task.
Version Control
Track changes to knowledge base articles. When an article is updated, the AI immediately uses the new version. Version history lets you revert if an update causes accuracy problems.
Training an AI chatbot is not a one-time project — it is an ongoing practice. The initial build gets you to 70-80% accuracy. The ongoing maintenance, feedback loops, and gap-filling push you toward 90%+.
Key insight: Teams that treat their knowledge base as a living product rather than a static library consistently achieve the highest AI resolution rates and customer satisfaction scores.
Ready to see AI support in action? Start your free trial and watch your resolution rates climb.