Arch Gateway Intelligent gateway for agents

What is Arch Gateway?

Arch Gateway functions as an intelligent proxy specifically engineered for AI agents. It streamlines the development of agentic applications by managing complex tasks such as prompt handling, security enforcement against jailbreaks, intelligent routing, and integrating user requests with backend functions or APIs. Built by contributors to the Envoy Proxy, it aims to provide enterprise-grade reliability and performance for AI interactions.

The platform offers robust observability features, providing detailed tracing, metrics, and logs compatible with OpenTelemetry destinations. It supports integration with various Large Language Models (LLMs) through a consistent interface, enabling developers to experiment and deploy different models without extensive code changes. Arch Gateway facilitates the transition from basic Retrieval-Augmented Generation (RAG) systems to more complex, multi-turn conversational agents and automated task execution, allowing developers to focus on core application logic.

Features

Intelligent Prompt Routing: Directs prompts to appropriate agents or functions based on intent.
Agentic Task Handling: Translates natural language user requests into backend function calls.
Centralized Safety Guardrails: Prevents jailbreak attempts and enforces custom safety policies.
Multi-LLM Support: Provides a single interface to integrate and experiment with numerous LLMs.
Standards-based Telemetry: Offers rich tracing, metrics, and logs compatible with OpenTelemetry.
Function Calling: Enables dynamic execution of backend functions based on user prompts.
Open Source: Provides flexibility to use, extend, and deploy the gateway freely.
Built on Envoy Proxy: Leverages proven, enterprise-grade technology for reliability.
Purpose-Built LLMs: Utilizes efficient LLMs optimized for specific tasks like parameter extraction.

Use Cases

Building accurate information retrieval systems by transforming prompts into precise queries.
Developing agentic applications that execute transactional operations (e.g., placing orders, processing payments).
Implementing agentic routing for tasks requiring multiple agent interactions (e.g., scheduling meetings).
Creating unified conversational interfaces to interact with multiple SaaS applications.
Improving the speed and accuracy of Retrieval-Augmented Generation (RAG) scenarios.
Simplifying the handling of multi-turn conversations in AI applications.

FAQs

What is Arch Gateway built on?

Arch Gateway is built on and extends Envoy Proxy, leveraging its HTTP connection management, filtering, and telemetry capabilities.
Is Arch Gateway open source?

Yes, Arch Gateway is an open-source project, offering flexibility to use, extend, and deploy it freely.
How does Arch Gateway handle different LLMs?

It provides a single, consistent interface to centralize access controls and manage traffic for 100+ LLMs, allowing rapid incorporation and experimentation.
What are Prompt Targets in Arch Gateway?

Prompt Targets are a core concept allowing developers to define how different user prompts are processed and routed, separating business logic from prompt handling complexities.
How does Arch Gateway ensure safety?

It centralizes guardrails to prevent jailbreak attempts and ensure safe interactions, with options to configure custom guardrails for specific application requirements.