What is Preprocess?

Preprocess is a solution designed to streamline the creation of ingestion pipelines for Retrieval-Augmented Generation (RAG) applications. It focuses on converting and splitting complex documents, such as PDFs, Word files, Powerpoint presentations, Excel spreadsheets, HTML, OpenOffice files, and plain text, into optimally sized chunks of text suitable for vector databases. The platform aims to simplify the often complex task of document preprocessing, allowing developers and teams to bypass significant development and maintenance overhead.

By handling the nuances of various file types and their internal elements like tables and images, Preprocess ensures high-quality data preparation. It offers features like intelligent parsing, chunking, and extraction of text, tables, and images (including text within images). This specialized preprocessing enhances the performance of downstream RAG systems by providing them with well-structured, relevant text segments. The service can be integrated via API or SDKs, fitting into existing development workflows.

Features

Intelligent Parsing and Chunking: Automatically processes and divides documents into optimal segments.
High-Quality Table Extraction: Accurately extracts tabular data from documents.
Image Extraction: Isolates images embedded within documents.
Image Text Extraction: Extracts text content found within images (OCR).
Full Document Extraction: Processes the entire content of supported documents.
Multi-Format Support: Handles PDF, Scanned PDF, Excel, Slides, HTML, Plain text, and OpenOffice files.
Parallel Task Processing: Enables efficient processing of multiple documents simultaneously.
API & SDK Access: Provides integration options through an API and Python SDK.

Use Cases

Building efficient Retrieval-Augmented Generation (RAG) pipelines.
Preparing complex documents for vector database ingestion.
Automating document preprocessing for AI applications.
Extracting text and structured data (tables, images) from various document formats for AI models.
Streamlining data preparation workflows for large language models.