The Gateway Reports On Its Own Denominator

An eBPF program on the socket can redirect your agents' model calls into a proxy without touching a line of their code, and the proxy can then read the prompt, swap the model, cap the tokens and refuse the response. That works. The question this lesson is about is the one the word "transparent" hides: which calls actually arrive at the proxy, and how would you know they were the only ones.

This assumes why the decision has to leave the kernel in the first place — the short version is that a syscall filter can see a connection but not a prompt, so the semantics have to happen in a process that can parse JSON. eBPF stands for the extended Berkeley Packet Filter, the in-kernel virtual machine Linux runs small verified programs on. JSON is JavaScript Object Notation, the encoding every model API speaks.

The reference implementation attaches to a cgroup hook called connect4. In its author's own words during the code walkthrough: "if you look at the area where it says SEC, that's the section header, and connect4, that is a TCP IPv4 connection event, which is what happens when you connect out to something." Read that again as a coverage statement rather than an implementation note. A connect4 program runs on connect() for IPv4 sockets. It is not strictly TCP-only — the kernel calls the same hook from udp.c and ping.c, so an IPv4 UDP connect() raises it too — but every model call in this fleet is TCP, and the list of what the hook never sees is the part that matters. It does not see IPv6, it does not see a unix domain socket, it sees only the processes inside the cgroup it was attached to, and it fires on new connections, so anything already established was never offered to it. IPv4 is version 4 of the Internet Protocol; IPv6 is version 6, and a resolver that returns an AAAA record will hand your agent one.

Below is one hour of a small agent fleet: 600 model calls, generated from six call shapes that differ in transport, in which process made them, and in whether the connection already existed when the gateway attached. Configure the hooks. Then read the two numbers at the top against each other — the count of calls the gateway actually governed, and the number its own dashboard would show you.

Before changing anything, read those two readouts together. The dashboard says 100.0% of 357. The gateway governed 357 of 600 calls — 59.5%. Neither number is wrong. The first is a ratio over the calls that reached the proxy, and the 240 that never did are not violations in it, they are simply not in it. Now tick connect6 and watch which number moves.

The number that is right and useless

Reset the panel: connect4 ticked, connect6 unticked, forced reconnect ticked, the hook attached to the annotated agent process only, the model in the cluster over plain HTTP, 18 ms of proxy overhead, fail open, the restart ticked, prompt rewriting unticked. Every figure below assumes that state unless the paragraph moves something.

The dashboard reads 100.0% of 357. The gateway governed 357 of 600 calls, which is 59.5%. Both numbers are correct. The first is computed over the calls the proxy handled, and the 240 calls that never arrived are not failures in it — they are absent from it. This is the whole lesson, and it is visible at the default configuration, before you have changed anything.

Look at the lanes to see where the 240 went. Shape D, 81 calls, is the agent doing exactly what it always does, except the resolver handed it an AAAA record and the connection went out over IPv6 to a connect6 program that does not exist. Shape C, 94 calls, is a subprocess the agent spawned — a retrieval helper, an evaluation script — outside the cgroup the hook was attached to. Shape E, 45 calls, is a different container in the same pod. Shape F, 20 calls, is a local runtime reached over a unix domain socket.

Now close the gaps you can close. Tick connect6: governed goes to 438. Set the cgroup selector to "that process and its descendants": 530. Set it to "the whole pod cgroup": 575. The dashboard has said 100.0% at every step — it said it when the gateway was governing 59.5% of the fleet and it says it now that the gateway is governing 95.8%. It is the same number describing two very different systems.

Twenty calls remain uncovered, and the "calls no hook here could catch" readout is showing them separately from the rest for a reason. Untick and re-tick every box: shape F never moves. A unix domain socket connect is AF_UNIX — a different address family, a path on the filesystem rather than an IP address and port — so it never becomes a connect4 or connect6 event, so no setting on this panel reaches it. The other 220 were a configuration change. These 20 are a different hook, or a different design.

Leave the panel with connect6 ticked and the cgroup selector on "the whole pod cgroup" for the next section.

Reading the request is a separate privilege from catching it

State: both connect hooks on, forced reconnect on, the whole pod cgroup, the model in the cluster over plain HTTP, 18 ms, fail open, restart on, no rewriting. Governed 575, dashboard 100.0% of 575, time to first token 418 ms.

That configuration works because the model is inside the cluster and nothing encrypted the request. This is not a detail of the demo, it is the precondition for the whole pattern, and it is worth seeing what happens without it. Set "where the model lives" to "hosted over HTTPS, TLS left alone".

Governed drops to 0. The 575 calls are still intercepted — the socket hook works exactly as before, the proxy still receives every connection and still forwards it. The "redirected but unreadable" readout now says 575. The proxy has a TLS stream and no key, so it can count a connection and route it and do nothing else. The dashboard now reads 0.0% of 575, which is at least honest, though a real one would more likely report 575 connections proxied and no policy decisions and let you draw your own conclusion.

Set it to "hosted over HTTPS, proxy terminates TLS with your CA". Governed returns to 575 and time to first token goes to 424 ms. That is the deal, and it is a real one that people take: Cilium documents the same mechanism for its own Layer 7 visibility, where you create an internal certificate authority so the proxy can, in its words, "act like the destination service and terminate the TLS connection". What you have bought is readability. What you have taken on is a certificate authority your pods trust and a process holding a key that decrypts every prompt and every completion your organisation produces. That is a reasonable thing to own deliberately and a terrible thing to acquire by accident because a demo worked.

Leave the endpoint on "hosted over HTTPS, proxy terminates TLS with your CA".

The proxy is in the data path, and the failure mode is the real cost

State: both hooks on, forced reconnect on, whole pod cgroup, TLS terminated with your CA, 18 ms, fail open, restart on, no rewriting. Governed 575, time to first token 424 ms.

Take the overhead slider to 120 ms. Time to first token goes to 526 ms, which is +31.5% on the delay before the first word appears. Against a whole streaming response of a couple of seconds that is a few percent and nobody will file a ticket; against time to first token it is the difference between an interface that feels immediate and one that does not. If you are going to quote one latency number when you propose this, quote that one. Put the slider back to 18 ms.

The bigger cost is not latency, it is that you have added a component every model call depends on. The restart checkbox is one config reload, thirty seconds, the most ordinary event in the life of a proxy. With fail open, the readout says 5 calls bypassed: they went straight out, succeeded, and are not in the dashboard. Coverage silently dropped for thirty seconds and the compliance figure did not move, because bypassed calls are handled by nobody and counted by nobody.

Switch "if the proxy is unavailable" to "fail closed". The readout says 5 calls failed. Read the log line under it, because the detail is the point: the calls that failed are the covered ones. Every call on an uncovered path sailed through the restart without noticing, because it was never going to the proxy. Fail-closed makes your best-covered traffic your least reliable traffic, and does nothing at all to the traffic you were worried about. That is not an argument for fail-open — it is an argument for knowing your coverage number before you choose, because fail-closed on 59.5% coverage is the worst of both.

Put "if the proxy is unavailable" back to "fail open" before the next section.

A gateway that rewrites prompts is writing into the model's context

State: both hooks on, forced reconnect on, whole pod cgroup, TLS terminated with your CA, 18 ms, fail open, restart on. Governed 575.

Tick "the gateway rewrites prompts before forwarding". The readout "calls carrying text the application never wrote" goes from 0 to 575.

In the reference talk this capability is presented as a benefit, and in a narrow sense it is: rewriting the prompt is how you enforce a house style, or strip a field, or pin an instruction the application forgot. But describe the mechanism without the framing and it is exact: a process on the network path modifies the instructions a language model receives, and the application that composed those instructions is not told. That is the same mechanism as indirect prompt injection. The difference is entirely in who is doing it and whether you meant to.

Three consequences follow, and none of them is a reason not to do it — they are reasons to treat the rewrite rule as production code rather than configuration.

• It is a single point that reaches every context. The readout is 575 because rewriting applies to every governed call. Whoever can edit that config map can edit the input of every agent in the fleet at once, without touching a repository any of those agents' owners watch.
• It breaks the provenance story. When an agent does something surprising, the first question is what was in its context. If a gateway rewrote the prompt, the application's logs and the model's actual input are two different documents, and only one of them is in your trace.
• It composes badly with the thing it is often bought to stop. A common motivation for inspecting prompts is detecting injected instructions in retrieved content. A gateway that both looks for injected text and inserts its own has to be very clear with itself about which is which.

Untick the rewriting box before the next section.

What is actually being standardised, and what is not

You will see this pattern described as being on a standards track. It is worth being precise, because the standards work is real and is about something adjacent.

Kubernetes announced WG AI Gateway on 9 March 2026 — not, as it is sometimes called, an "AI egress working group". Its egress work is proposal 10 in kubernetes-sigs/wg-ai-gateway, and that proposal's goal is to "define the standards for Gateways that route and manage traffic destined for external resources outside of the cluster", along with "the standards by which token management for Gateways can be employed to enable access to backends that require auth". That is an explicit gateway: a declared endpoint your workloads are configured to route through, which holds the provider credentials so the workloads never do.

Transparent interception is not that, and the proposal says so directly in its “Out of Scope” section: "Forward-proxy egress (dynamic routing to arbitrary external hostnames), network-level egress (L3/L4 CIDR-based routing), and mesh-attached egress (sidecar-enforced policy without a Gateway) are not covered by this proposal." A shared goal, then, and two different mechanisms. The socket-redirect approach is a proof of concept — its author calls it one — and a good one. It is not the thing being standardised, and saying it is will not survive the first person who opens the repository.

The distinction matters practically, not just pedantically. The explicit model trades the thing this whole lesson is about — coverage you did not have to ask for — against coverage you can actually enumerate, because a workload either routes to the gateway or it does not, and a NetworkPolicy that denies direct egress turns "did we cover this path" into a question with a yes-or-no answer. Transparent capture gets you started without touching the application. Explicit routing gets you a denominator.

Checking this on a system you actually run

1. Get the denominator, and get it from somewhere other than the gateway. The gateway cannot tell you what it missed. Count model calls at a source it does not control: your provider's own usage or billing records, per-agent token counters emitted by the application, or the flow log from your network layer. Then compare that count with the proxy's handled count for the same window. The gap is the number this lesson is about, and nothing on the gateway's own dashboard will ever show it to you.
2. Enumerate the transports before you trust the hook. Check whether your model endpoint resolves to an AAAA record — getent ahosts or dig AAAA — because if it does and you only loaded connect4, a share of your traffic has never been to the proxy. Then look for unix sockets: ss -xp inside the pod will show a local runtime that no connect hook of either family will ever see.
3. Work out the cgroup you attached to, and list what is in it. The hook's scope is the cgroup, so the question "does this cover the retrieval subprocess" is answered by cat /proc/<pid>/cgroup for that subprocess, not by reading the annotation. Do this for every process in the pod, including sidecars, and do it after the agent has been running long enough to have spawned its helpers.
4. Decide the failure mode with the coverage number in front of you. Fail-closed is defensible at high coverage and indefensible at low, because at low coverage it only breaks the traffic you had already secured. Whatever you choose, alert on the proxy's handled-call rate rather than its compliance rate: a fail-open bypass shows up as a dip in the first and not at all in the second.
5. Treat prompt rewriting as code, with a review and a trace. If the gateway edits prompts, the model's real input is not in the application's logs. Emit the post-rewrite prompt, or a hash of it, into the same trace as the application's version, so that when something goes strangely you can diff the two rather than argue about them.
6. Consider whether you need the payload at all. Terminating TLS to read prompts is a large, permanent commitment. Orange Innovation's agent platform constrains agent egress with a network policy keyed on per-agent identities issued by cert-manager, which needs no payload visibility to work and no certificate authority of its own. If your actual requirement is "these agents may only talk to these model endpoints", that buys it far more cheaply than an interception gateway does.

Transparent interception is a genuinely good way to get control over model traffic you do not own the source of, and the eBPF half of it is elegant. The failure it invites is not technical. It is that the gateway's own numbers are the most reassuring artefact in the system and the least equipped to tell you they are incomplete. Every question worth asking about this pattern is a question about the calls that are not in them.