Gateway

A thin proxy between your agents and your LLM providers. It captures traces, optionally enforces guardrails, and leaves the analysis to the platform.

The Harbor gateway sits between your agents and your LLM providers, and it runs on Harbor's infrastructure. Adoption is one change: point your framework's base URL at the gateway instead of the provider. That single swap is enough to be monitored, to turn on guardrails, and to start building the evidence that insurance is priced on. Every call is forwarded unchanged, streaming included, and your provider credentials pass through. The gateway holds no keys of its own.

Where it lives

The gateway is deliberately thin: it captures and, if configured, enforces. Analysis, dashboards and configuration live in the Harbor platform, never in the request path.

What it does

Capture. One OpenTelemetry span per LLM call: model, tokens, latency, messages, using the standard gen_ai.* conventions. This is the trace stream the platform turns into risk signals. Capture is fire-and-forget and fails open: your traffic never breaks because Harbor is down. OpenAI-compatible and Anthropic APIs are supported today, with Bedrock, Azure OpenAI and Vertex next, since that is how enterprises consume these models.

Guardrails, if configured. The gateway checks tool calls against your verified policies before your agent executes them. Streaming is handled properly: text streams through untouched, and tool calls, which arrive at the tail of a stream, are held for the check and then released or blocked. The perceived cost is roughly one policy check, and the check runs in microseconds. A blocked call is returned as a valid, machine-readable refusal the agent can adapt to, not a dropped connection. Enforcement fails closed. Same policies, same observe and enforce modes as everywhere else: see Guardrails.

The two halves fail in deliberately opposite directions:

	Capture	Enforcement
In the request path	Fire-and-forget	Blocking, microseconds
If Harbor is unreachable	Fails open — your traffic is unaffected	Fails closed — the call does not proceed
Configured	Always on	Per agent, per policy

That is the whole feature list for now. No routing logic, no caching, no provider marketplace. A thin gateway is easy to trust in the request path.

Integration

The integration is the base URL. Any framework that lets you set one, which is nearly all of them, works unchanged:

# illustrative
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-5",
    base_url="https://gateway.harbor.ai/v1",   # the only change
)

Frameworks that read OPENAI_BASE_URL from the environment need no code change at all:

export OPENAI_BASE_URL=https://gateway.harbor.ai/v1

Your provider API key stays exactly where it is and passes through untouched.

Deployment

The gateway is hosted by Harbor. Swapping the base URL is the whole integration: no binaries to run, no infrastructure to own, and you are set up for monitoring, guardrails and, eventually, coverage.

If your use case demands it, the same gateway can also run in your environment:

Mode	Where it runs	When
Hosted (default)	Harbor's infrastructure	Almost everyone; one URL swap
Your cloud	A single container in your VPC	Raw prompts and responses must stay inside your boundary
Sidecar	Next to each agent deployment	Air-gapped or latency-critical setups

It is the same static binary in every mode, with no dynamic dependencies. When it runs in your cloud, the default is that raw prompts and responses stay inside your environment and only traces and policy verdicts leave for the Harbor platform. What egresses is configurable per deployment.

Gateway or SDK

The gateway is the fastest way to get coverage across many agents at once; the SDK sees more. The overview has the full comparison.

They share the same policies and feed the same platform. Running both is the highest-assurance setup, not a conflict: the SDK enforces at ground truth while the gateway guarantees capture.