For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Open WebUI

Deploy Open WebUI in Kubernetes and route its LLM traffic through agentgateway.

Deploy Open WebUI in Kubernetes and route its LLM traffic through agentgateway to keep API keys server-side and capture audit logs for every chat.

Before you begin

1. Set up an agentgateway proxy.
2. Set up access to the OpenAI LLM provider.

Get the gateway URL

export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-proxy \
  -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

echo "Gateway address: $INGRESS_GW_ADDRESS"

export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-proxy \
  -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

echo "Gateway address: $INGRESS_GW_ADDRESS"

kubectl port-forward -n agentgateway-system svc/agentgateway-proxy 8080:80

Set up the OpenAI backend

1. Export your OpenAI API key.

   export OPENAI_API_KEY="sk-your-key-here"

2. Create a Kubernetes Secret for your API key.

   kubectl apply -f- <<EOF
   apiVersion: v1
   kind: Secret
   metadata:
     name: openai-secret
     namespace: agentgateway-system
   type: Opaque
   stringData:
     Authorization: $OPENAI_API_KEY
   EOF

3. Create an AgentgatewayBackend for OpenAI.

   kubectl apply -f- <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayBackend
   metadata:
     name: openai
     namespace: agentgateway-system
   spec:
     ai:
       provider:
         openai: {}
     policies:
       auth:
         secretRef:
           name: openai-secret
   EOF

4. Create an HTTPRoute to forward traffic to the backend.

   kubectl apply -f- <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: openai
     namespace: agentgateway-system
   spec:
     parentRefs:
       - name: agentgateway-proxy
         namespace: agentgateway-system
     rules:
       - matches:
         - path:
             type: PathPrefix
             value: /
         backendRefs:
         - name: openai
           namespace: agentgateway-system
           group: agentgateway.dev
           kind: AgentgatewayBackend
   EOF

Deploy Open WebUI

Deploy Open WebUI and point it at the agentgateway service.

kubectl apply -f- <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: open-webui
  namespace: agentgateway-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: open-webui
  template:
    metadata:
      labels:
        app: open-webui
    spec:
      containers:
      - name: open-webui
        image: ghcr.io/open-webui/open-webui:main
        ports:
        - containerPort: 8080
        env:
        - name: OPENAI_API_BASE_URL
          value: "http://agentgateway-proxy.agentgateway-system.svc.cluster.local/v1"
        - name: OPENAI_API_KEY
          value: "placeholder"
        volumeMounts:
        - name: open-webui-data
          mountPath: /app/backend/data
      volumes:
      - name: open-webui-data
        emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  name: open-webui
  namespace: agentgateway-system
spec:
  selector:
    app: open-webui
  ports:
  - port: 8080
    targetPort: 8080
EOF

The following table describes each environment variable:

Verify the connection

1. Port-forward to Open WebUI.

   kubectl port-forward -n agentgateway-system svc/open-webui 8080:8080

2. Open http://localhost:8080, create the initial admin account, select a model, and send a message.

3. Confirm the request appears in the agentgateway proxy logs.

   kubectl logs deployment/agentgateway-proxy -n agentgateway-system --tail=5

   You should see a log entry showing the request was forwarded to the OpenAI endpoint with the configured model:

   info  request gateway=agentgateway-system/agentgateway-proxy listener=http route=agentgateway-system/openai endpoint=api.openai.com:443 http.method=POST http.path=/v1/chat/completions http.status=200 protocol=llm gen_ai.operation.name=chat gen_ai.provider.name=openai gen_ai.request.model=gpt-4o gen_ai.usage.input_tokens=4569 gen_ai.usage.output_tokens=10 duration=2242ms

Next steps