AI application monitoring tool - AI tools

Flowtest.ai uses an AI Agent to continuously monitor your website like a real user, providing instant alerts and detailed reports for any issues.

• Free Trial
• From 20$

Monity.ai offers AI-powered website monitoring, providing real-time alerts and automated data extraction to keep users informed about website changes.

• Freemium
• From 12$

Arize is a comprehensive platform designed to accelerate the development and improve the production of AI applications and agents.

• Freemium
• From 50$

Raindrop provides AI application monitoring, sending alerts for misbehavior and enabling rapid root cause analysis and issue resolution.

• Free Trial
• From 65$

Keywords AI is a comprehensive developer platform for LLM applications, offering monitoring, debugging, and deployment tools. It serves as a Datadog-like solution specifically designed for LLM applications.

• Freemium
• From 7$

AppMap enhances AI development tools by providing real-time runtime context of application behavior. It captures code execution, data flow, and performance, enabling smarter suggestions, faster debugging, and fewer defects.

• Freemium
• From 15$

Evidently AI is a comprehensive AI observability platform that helps teams evaluate, test, and monitor LLM and ML models in production, offering data drift detection, quality assessment, and performance monitoring capabilities.

• Freemium
• From 50$

OpenLIT is an open-source observability platform designed to streamline AI development workflows, particularly for Generative AI and LLMs, offering features like prompt management, performance tracking, and secure secrets management.

• Other

AI Monitor is a Generative Engine Optimization (GEO) platform helping brands track visibility and reputation across AI platforms like ChatGPT and Google AI Overviews.

• Contact for Pricing

llm.report is an open-source analytics and logging platform for OpenAI API usage, providing real-time monitoring, cost tracking, and usage optimization for AI applications.

• Freemium
• From 20$

WhyLabs provides AI Control Center for observing, securing, and optimizing AI applications, offering tools for LLM security, ML monitoring, and AI observability.

• Freemium
• From 125$
• API

HoneyHive is a comprehensive platform that provides AI observability, evaluation, and prompt management tools to help teams build and monitor reliable AI applications.

• Freemium

Palzin Monitor is a comprehensive infrastructure monitoring platform that combines uptime monitoring, incident management, and AI assistance to help teams detect and resolve issues before they impact users.

• Freemium
• From 8$

UsageGuard is an enterprise-ready AI development observability platform that provides a unified API for multiple AI models, comprehensive security controls, and real-time monitoring capabilities with 99.9% uptime and sub-150ms latency.

• Contact for Pricing

HawkFlow.ai is a monitoring tool designed for engineers, product owners, and CTOs, integrating seamlessly with machine learning infrastructure to provide valuable insights and facilitate efficient decision-making.

• Freemium
• API

Retack AI is an error monitoring tool that tracks application errors in real-time across various programming languages and provides AI-powered code fixes.

• Contact for Pricing

MonitUp AI Time Tracker helps you track computer activity, gain insights into work habits, and boost productivity using AI-powered suggestions.

• Paid
• From 6$

Mona provides a Model Performance Insights Platform™ for proactive monitoring of AI, ML, and other data-driven systems in high-stakes environments.

• Contact for Pricing

Distributional is an enterprise platform for AI testing, designed to give teams confidence in the reliability of their AI and ML applications. It offers a proactive approach to mitigate the risks associated with unpredictable AI systems.

• Contact for Pricing

Braintrust provides an end-to-end platform for developing, evaluating, and monitoring Large Language Model (LLM) applications. It helps teams build robust AI products through iterative workflows and real-time analysis.

• Freemium
• From 249$

Tiktokenizer simplifies AI app development by automatically tracking OpenAI API token usage per user, enabling transparent billing. It integrates seamlessly with OpenAI's Chat and Moderations APIs.

• Contact for Pricing

PerfAgents is an AI-powered synthetic monitoring platform that leverages existing web automation scripts to monitor application availability and response time metrics globally. It supports multiple frameworks and offers AI-powered script creation for continuous testing.

• Paid

Velatir provides a platform for managing AI-driven tasks, ensuring responsible AI operation with human oversight, compliance, and real-time monitoring.

• Contact for Pricing

AI Studio is a comprehensive MLOps platform that provides enterprise-level tools for machine learning governance, monitoring, and deployment. It enables companies to streamline their ML operations with real-time insights and automated workflows.

• Freemium

Prompt Inspector is an AI-powered analytics platform that helps businesses analyze and optimize their AI prompts through user behavior tracking, pattern analysis, and comprehensive reporting capabilities.

• Paid
• From 12$

Portkey is a comprehensive AI operations platform offering AI Gateway, Guardrails, and Observability Suite to help teams deploy reliable, cost-efficient, and fast AI applications.

• Freemium
• From 49$

Decipher AI uses advanced vision LMs to analyze session replays, identifying customer issues, understanding feature usage, and providing actionable insights.

• Free Trial

BrandBeacon.ai monitors your brand's presence and visibility across major AI search platforms like ChatGPT and Perplexity. Track mentions, compare competitor visibility, and gain insights for data-driven marketing decisions.

• Freemium

Graphsignal monitors, profiles, and accelerates hosted LLM inference and model APIs, providing full visibility and deep insights for AI optimization.

• Freemium
• From 375$

AgentOps is a comprehensive developer platform that enables testing, debugging, and deployment of AI agents with support for 400+ LLMs, Crews, and AI agent frameworks.

• Freemium
• From 40$