CrawlChat Deliver your documentation with AI

What is CrawlChat?

CrawlChat offers a solution for making extensive documentation readily accessible and interactive through Artificial Intelligence. It works by scraping the content from a provided website URL. This content is then processed, converted into markdown, and transformed into embeddings stored within a vector database, making it suitable for large language models (LLMs) to understand and utilize.

Once the knowledge base is created, CrawlChat allows users to deploy it across multiple channels. Businesses can embed a chat widget directly onto their website, add a dedicated bot to their Discord community server for automated query handling, or make the documentation available via a Model Context Protocol (MCP) server for integration with other AI systems. The platform focuses on providing answers derived strictly from the source documentation, aiming to minimize inaccuracies and AI hallucinations. It also includes features for customization, conversation monitoring, and analytics to help users understand customer queries and identify areas for documentation improvement.

Features

Website Scraping: Automatically crawls and processes website content into an LLM-ready format.
Embeddable Chat Widget: Allows embedding an 'Ask AI' chat widget on any website.
Discord Bot Integration: Provides a bot to answer questions within a Discord community based on the documentation.
MCP Server Deployment: Offers a Model Context Protocol (MCP) server for documentation integration with external LLM systems.
System Prompt Customization: Enables users to define system prompts to align AI responses with brand voice.
Conversation Viewing: Provides visibility into user chat interactions for monitoring and improvement.
Usage Analytics: Offers insights into customer queries and helps identify documentation gaps.
Source-Based Answers (RAG): Ensures AI responses are grounded in the provided documentation to prevent hallucinations.
Widget Customization: Allows modification of chat widget colors and welcome messages.
Shareable Chat Links: Generates direct links to the AI chat interface.
Reactive Discord Bot: Bot can automatically react to relevant messages in designated Discord channels.
PDF/Document Upload: Functionality to upload PDFs and other documents to the knowledge base (Coming Soon).

Use Cases

Creating an interactive AI chat interface for website documentation.
Providing instant answers to user questions based on existing content.
Automating support responses within a Discord community.
Integrating technical documentation with AI development tools via MCP.
Analyzing user interactions to improve documentation clarity and coverage.
Offering a searchable knowledge base alternative to static documentation pages.

FAQs

How does CrawlChat ensure the AI answers accurately?

CrawlChat uses Retrieval-Augmented Generation (RAG), ensuring the AI only answers based on the provided documentation content, which helps eliminate hallucinations.
Can I customize the appearance of the chat widget?

Yes, you can customize the widget's colors and welcome screen to match your brand identity.
What is an MCP server and why is it useful?

A Model Context Protocol (MCP) server allows external AI systems and LLMs (like Cursor, Windsurf, Claude) to connect and utilize your documentation content. CrawlChat provides this integration capability.
How does the Discord bot interact with users?

The Discord bot answers questions based on the scraped documentation when tagged (@crawlchat). It also has a reactive mode where it checks messages in a channel and reacts if it knows the answer.
Can I use CrawlChat without embedding the widget on my site?

Yes, CrawlChat provides shareable links to the chat page, allowing you to share the AI interaction interface directly with users.