Open-source LLM development tools - AI tools

Laminar is an open-source platform that enables developers to trace, evaluate, label, and analyze Large Language Model (LLM) applications with minimal code integration.

• Freemium
• From 25$

Hegel AI provides a developer platform for building, monitoring, and improving large language model (LLM) applications, featuring tools for experimentation, evaluation, and feedback integration.

• Contact for Pricing

Flowise is an open-source low-code platform that enables developers to build customized LLM orchestration flows and AI agents through a drag-and-drop interface.

• Freemium
• From 35$

Langfuse provides an open-source platform for tracing, evaluating, and managing prompts to debug and improve LLM applications.

• Freemium
• From 59$

Missing Studio is an open-source AI platform designed for developers to build and deploy generative AI applications. It offers tools for managing LLMs, optimizing performance, and ensuring reliability.

• Free

PromptsLabs is a community-driven platform providing copy-paste prompts to test the performance of new LLMs. Explore and contribute to a growing collection of prompts.

• Free

Phoenix accelerates AI development with powerful insights, allowing seamless evaluation, experimentation, and optimization of AI applications in real time.

• Freemium

BenchLLM is a tool for evaluating LLM-powered applications. It allows users to build test suites, generate quality reports, and choose between automated, interactive, or custom evaluation strategies.

• Other

Agenta is an LLM engineering platform offering tools for prompt engineering, versioning, evaluation, and observability in a single, collaborative environment.

• Freemium
• From 49$

EleutherAI is a research institute focused on advancing and democratizing open-source AI, particularly in language modeling, interpretability, and alignment. They train, release, and evaluate powerful open-source LLMs.

• Free

LM Studio is a desktop application that allows users to run Large Language Models (LLMs) locally and offline, supporting various architectures including Llama, Mistral, Phi, Gemma, DeepSeek, and Qwen 2.5.

• Free

LMQL is a programming language designed for large language models, offering robust and modular prompting with types, templates, and constraints.

• Free

LiteLLM provides a simplified and standardized way to interact with over 100 large language models (LLMs) using a consistent OpenAI-compatible input/output format.

• Free

Rig is a Rust-based framework for building modular and scalable LLM applications. It offers a unified LLM interface, Rust-powered performance, and advanced AI workflow abstractions.

• Free

Keywords AI is a comprehensive developer platform for LLM applications, offering monitoring, debugging, and deployment tools. It serves as a Datadog-like solution specifically designed for LLM applications.

• Freemium
• From 7$

LLMStack is an open-source development platform that enables users to build AI agents, workflows, and applications by integrating various model providers and custom data sources.

• Other

Aider is a command-line tool that enables pair programming with LLMs to edit code in your local git repository. It supports various LLMs and offers top-tier performance on software engineering benchmarks.

• Free

Gentrace is an LLM evaluation platform designed for AI teams to test and automate evaluations of generative AI products and agents. It facilitates collaborative development and ensures high-quality LLM applications.

• Usage Based

Unify provides tools to build, test, and optimize LLM pipelines with custom interfaces and a unified API for accessing all models across providers.

• Freemium
• From 40$

ModelBench enables teams to rapidly deploy AI solutions with no-code LLM evaluations. It allows users to compare over 180 models, design and benchmark prompts, and trace LLM runs, accelerating AI development.

• Free Trial
• From 49$

LMSYS Org is a leading organization dedicated to developing and evaluating large language models and systems, offering open-source tools and frameworks for AI research and implementation.

• Free

Mistral AI is a research lab developing state-of-the-art open-source and commercial Large Language Models (LLMs) for developers and enterprises.

• Other

Neural Magic offers enterprise inference server solutions to streamline AI model deployment, maximizing computational efficiency and reducing costs on both GPU and CPU infrastructure.

• Contact for Pricing

Langbase is a serverless AI developer platform that enables developers to build, deploy, and manage AI products with composable infrastructure, featuring BaseAI - the first Web AI Framework.

• Freemium
• From 20$

Rubra provides a collection of open-weight large language models (LLMs) enhanced with tool-calling capabilities, ideal for building AI agents.

• Free

PromptMage is a Python framework that streamlines the development of complex, multi-step applications powered by Large Language Models (LLMs), offering version control, testing capabilities, and automated API generation.

• Other

Rhesis AI offers an open-source SDK to generate comprehensive, context-specific test sets for LLM applications, enhancing AI evaluation, reliability, and compliance.

• Freemium

OpenRouter provides a unified interface for accessing and comparing various Large Language Models (LLMs), offering users the ability to find optimal models and pricing for their specific prompts.

• Usage Based

Inductor enables developers to rapidly prototype, evaluate, and improve LLM applications, ensuring high-quality app delivery.

• Freemium

Weights & Biases (W&B) Weave is a comprehensive framework designed for tracking, experimenting with, evaluating, deploying, and enhancing LLM-based applications.

• Other