DeepConcepts

Security / security / transport / pki

mTLS Tells You Which CA Signed the Caller, Not Which Caller It Is

The misconception

That switching on mutual TLS authenticates the calling service. It authenticates the issuer. RFC 8446 §4.4.2.4 states that detailed certificate validation is out of scope for TLS, and there is no client-side equivalent of the server-name check RFC 9525 defines — nothing in the handshake compares the client certificate against an expected identity. With nginx's ssl_verify_client on and one internal CA in ssl_client_certificate, every workload in the fleet presents a certificate that verifies, so any of them can call any route as any other. Widen the trust file from the issuing CA to the root, or paste in /etc/ssl/certs/ca-certificates.crt, and the set of peers that pass the handshake grows without a line of application code changing. The check most teams then write — $ssl_client_verify == "SUCCESS" — is true of every one of them and therefore rejects nobody.

16 min

Mutual TLS — mTLS, where the server demands a certificate from the client as well as presenting one of its own — is sold as authentication for service-to-service calls. What the handshake actually establishes is that the peer holds the private key for a certificate that chains to something in your trust file. Which peer, out of everything that trust file admits, is a question TLS never asks.

That is not an implementation gap. RFC 8446, the TLS 1.3 specification, says it in §4.4.2.4: "In general, detailed certificate validation procedures are out of scope for TLS." There is a whole separate standard for checking that a server certificate names the host you dialled — RFC 9525 — and there is no counterpart for clients, because there is no equivalent of a hostname on the client side. The name in a client certificate means whatever your application decides it means, and if your application decides nothing, it means nothing.

The panel below is one nginx server in front of a payments API. Eleven peers open a connection and POST to /transfer, a route only the payments service is supposed to reach. Nine of them are things you would recognise from a real estate: other workloads, a web server presenting the wrong certificate, an employee laptop, a build runner, an attacker with a free public certificate. For each one the panel runs OpenSSL's chain check, then nginx's own gate, then whatever your handler checks, and prints which of the three decided.

Start with the defaults. ssl_verify_client on, your internal issuing CA in ssl_client_certificate, and a handler that treats a request arriving at all as authorisation. Eight of the eleven get a 400 at the door. Two of the three that get through are not the payments service. Now change only the trust file to your internal root — the same PKI, one level up. It becomes four.

SPIFFE ID is the Secure Production Identity Framework For Everyone's name for a workload identity written into a certificate as a URI subjectAltName, like spiffe://prod.internal.test/ns/pay/sa/payments-api. nginx exposes the subject and issuer distinguished names as variables but not the subjectAltName, so reading that one means passing $ssl_client_escaped_cert upstream and parsing it yourself.

peers that reach /transfer and are not the payments service
legitimate callers refused
$ssl_client_verify = SUCCESS
refused by nginx with a 400
Eleven peers, one trust file

reached /transfer and should not have · a legitimate caller refused · the intended outcome. Chain building, the depth limit and the purpose check are modelled as OpenSSL implements them; revocation and certificate lifetimes are not simulated here.

Two results from that panel are worth sitting with. The first is that the trust file is the entire membership decision, and it is a file path in a config, not a security review. The second is what happens when you select $ssl_client_verify must be SUCCESS — the check almost every team writes first. The hero number does not move. It is a true statement about every certificate that got this far, which is exactly why it separates nobody from anybody.

What the handshake proves, sentence by sentence

The client-authentication half of TLS 1.3 is four messages and about a page of specification. Reading it is the fastest way to see how little it promises.

The server sends CertificateRequest. RFC 8446 §4.3.2 lets it carry a certificate_authorities extension — a list of distinguished names the server would like the client to chain to — and that list is advisory. §4.4.2.3 says only that "at least one of the certificates in the certificate chain SHOULD be issued by one of the listed CAs". Lowercase should, in a specification where the word is defined to mean there may be valid reasons to ignore it.

The client sends Certificate and CertificateVerify. The second is a signature over the handshake transcript with the private key matching the first, and it is the only unforgeable thing in the four messages: it proves possession of a key. §4.4.2 also requires that a client with nothing suitable send a Certificate message containing no certificates at all, which is why "no certificate" is a normal outcome rather than an error — row C8 in the panel.

The server then decides. §4.4.2.4 is worth quoting exactly, because it is where the responsibility is handed back to you: "if some aspect of the certificate chain was unacceptable (e.g., it was not signed by a known, trusted CA), the server MAY at its discretion either continue the handshake (considering the client unauthenticated) or abort the handshake." Both are conforming. ssl_verify_client optional_no_ca is nginx choosing the first one.

Nowhere in those four messages is there a field for "the identity I expected". For servers there is a whole standard about that comparison — RFC 9525, which tells a client to build its list of acceptable names independently of what the server presents. For clients there is nothing, and RFC 9525 says so itself: §1.4.2 puts "client or end-user identities" on its out-of-scope list, noting that "certificates representing client identities … are beyond the scope of this document". There is no protocol-level notion of what a client is called. That is the gap the rest of this lesson lives in.

One consequence worth knowing before you debug an mTLS problem: in TLS 1.3 the client's Certificate message is encrypted. It travels inside the handshake encryption, where TLS 1.2 sent it in the clear. A packet capture will not show you which certificate a client offered, and a network appliance that used to log client subjects no longer can. Related: renegotiation-based client authentication is gone. A server that wants to demand a certificate part-way through a connection must use post-handshake authentication, and §4.6.2 permits that only "when the client has sent the post_handshake_auth extension" — a ClientHello extension, which §4.2.6 backs with "servers MUST NOT send a post-handshake CertificateRequest to clients which do not offer this extension". OpenSSL does not send it unless the application calls SSL_set_post_handshake_auth. "Ask for a certificate only on the admin routes" is a TLS 1.2 pattern that quietly stopped working.

The trust file is the membership list

Move the trust-file control through its three values with everything else at its default. The hero number goes 2, 4, 5. Your application did not change. Your certificates did not change. A file path did.

This is the part that surprises people, so state it plainly: in ssl_client_certificate you are not listing clients. You are listing issuers, and you are accepting everything each of them has ever signed or will ever sign. Pointing it at your internal root instead of the specific issuing CA admits every other CA underneath that root — including the corporate device CA that hands a certificate to every laptop in the company (rows C5 and C11). Pointing it at /etc/ssl/certs/ca-certificates.crt, which people do because that is the path they know, admits everything the operating system trusts plus whatever anyone has ever dropped in with update-ca-certificates (row C10).

There is a second, easier-to-miss effect. nginx's documentation notes that the certificates in ssl_client_certificate "will be sent to clients" — the names go out in the CertificateRequest as the certificate_authorities list. Pasting a system bundle in there means a few hundred distinguished names in every handshake. RFC 8446's implementation-pitfalls appendix, C.3, asks implementers whether they fragment oversized handshake messages, and notes that "in particular, the Certificate and CertificateRequest handshake messages can be large enough to require fragmentation". nginx says the same thing from its own side: the certificates in ssl_client_certificate are sent, and "if this is not desired, the ssl_trusted_certificate directive can be used". That is the same trust file without the advertisement, which is what you usually want.

Now leave the trust file on internal-root.pem and move ssl_verify_depth between 1 and 2. Row C3 — a real workload in your own fleet, issued under a regional CA so its chain carries two intermediates — flips between 400 at nginx and a completed handshake. At depth 1 the error is X509_V_ERR_CERT_CHAIN_TOO_LONG, raised before the subject is ever looked at. This is the "works from my laptop, fails from the new region" ticket. The depth counts intermediates only: OpenSSL's own documentation says it is "a limit on the number of certificates between the end-entity and trust-anchor certificates" and that "neither the end-entity nor the trust-anchor certificates count against depth". So depth 1 means one intermediate, which is fine for a flat internal CA and wrong for every two-tier PKI. nginx picked 1 as its default; OpenSSL's own default is 100.

Row C4 shows the one identity check the stack does perform for you, and almost nobody knows it is there. The checkout web server presents a perfectly valid certificate from your own CA, and it is rejected with X509_V_ERR_INVALID_PURPOSE. OpenSSL's TLS code calls X509_STORE_CTX_set_default(ctx, s->server ? "ssl_client" : "ssl_server"), which installs check_purpose_ssl_client, which rejects a certificate whose extendedKeyUsage extension is present and does not contain id-kp-clientAuth. Note the condition carefully: present and. A certificate with no extendedKeyUsage extension at all sails through, because the macro that decides this, xku_reject, tests the extension flag first.

That check is also the reason row C6 fails today and did not fail last year. Let's Encrypt removed id-kp-clientAuth from its default profile on 11 February 2026 and stopped issuing it entirely on 8 July 2026, following Chrome's root-program requirement to split client and server authentication into separate PKIs. If you have ever used a publicly-trusted certificate as a client certificate, that is why it stopped working, and the error you got was unsupported certificate purpose rather than anything about expiry or trust.

The check that rejects nobody

Set the trust file to the root plus the OS bundle and leave the handler checking nothing: six peers reach /transfer, five of which should not. Now select $ssl_client_verify must be SUCCESS. The number stays at five.

That is not a bug in the simulation. $ssl_client_verify is nginx reporting SSL_get_verify_result(), and by the time a request reaches your handler under ssl_verify_client on, nginx has already refused everything for which that value is not SUCCESS. Reading it in the application re-asks a question the proxy answered. It feels like authorisation because it mentions a certificate; it is a tautology.

It is not entirely useless — under optional it is the only thing separating an authenticated caller from an anonymous one, and if your handler can be reached by any path that does not go through the mTLS listener, it is a real check. But it is a check for whether, never for who.

Walk down the remaining options and watch which row each one catches.

  • CN allowlist. Catches the metrics scraper and the employee laptop. Misses row C11: a certificate from the corporate self-service portal with payments-api typed into the common-name box. The common name is a free-text field. Whether an attacker can choose it depends entirely on your CA's enrolment policy, and enrolment policy is not visible from the certificate. This is why RFC 9525 §1.3 tells implementers not to check domain-name-like strings in the common name at all, and why Istio stopped putting the identity there — see the four years of comments on istio/istio#42114 from people whose authorisation rules broke when it did.
  • URI subjectAltName equals the expected SPIFFE ID. Catches C11 as well, because a subjectAltName of type URI is a structured field your CA controls. Misses row C10: the decommissioned staging CA in the OS bundle will mint any SPIFFE ID you ask it for, including a production one. A name is only as trustworthy as the narrowest authority that can issue it.
  • …and the issuer must be the internal issuing CA. Catches C10. This is the pairing that actually works — a name, and the specific authority allowed to assert that name. Envoy's configuration reference states the same rule as an admonition on its SAN matcher: subject alternative names "are easily spoofable and verifying only them is insecure, therefore this option must be used together with trusted_ca."

With all three on, one impostor is left, and no further certificate check will remove it.

Where the certificate stops helping

Row C9 is the payments service's own certificate and own private key, presented from a host that is not the payments service. Every check in the panel passes it, at every setting, because every check is a statement about the certificate and the certificate is genuine.

This is the boundary of the mechanism, not a gap in your configuration. A TLS client certificate is a bearer credential with a slow expiry. It proves possession of a key, and possession is exactly what an attacker acquires from a leaked volume mount, a core dump, a debug endpoint, a compromised sidecar, or a backup of /etc/ssl/private. mTLS is often chosen because it is supposed to be better than a shared secret, and against network interception it is. Against a host compromise it is a secret in a file.

What actually shrinks that window is lifetime and binding, not more checking. Short-lived certificates — the SPIFFE model issues them in hours, not years — mean a stolen key stops working on its own, which matters because revocation for private PKI is usually theoretical. If your certificates last a year, you are relying on a certificate revocation list or an OCSP responder that your proxy may well be configured to soft-fail on; the same freshness problem that makes bearer-token revocation hard applies here, with a worse tooling story. Binding the key to hardware — a Trusted Platform Module, or a cloud instance identity — means the key cannot be copied out at all, which is the only thing that closes row C9.

There is also a boundary in the other direction, and it is where most mTLS outages come from. Every setting that makes the hero number smaller also makes the "legitimate callers refused" number twitchy. The handshake is the hard part of a deployment: a certificate that renews at 03:00 with a new chain, a new region whose CA sits one level deeper, a proxy that terminates TLS and forwards a header your handler trusts unconditionally. That last one deserves saying out loud — if your application reads X-Client-Cert or X-SSL-Client-S-DN from a request, then anyone who can reach the application without going through the proxy can set it. The header is a claim; the connection is the evidence.

Checking your own deployment

Every question below is answerable in a terminal today, without a scanner and without a pen test.

  • Print your actual membership list. openssl crl2pkcs7 -nocrl -certfile "$(nginx -T 2>/dev/null | awk '/ssl_client_certificate/{print $2}' | tr -d ';')" | openssl pkcs7 -print_certs -noout gives you one line per issuer that can authenticate a client. If that output is longer than the number of CAs you can name from memory, you do not know who your callers are. Anything with a public CA's name in it is a finding on its own.
  • Grep your handlers for the tautology. Search for ssl_client_verify, SSL_CLIENT_VERIFY, verified_chains, peerCertificates != null. Every hit that is not followed by a comparison against a specific name is a route that accepts your whole trust file.
  • Test it with a certificate you own. Take any other workload's certificate and key — a staging service, a metrics agent, anything from the same CA — and run curl --cert other.pem --key other.key https://api.internal.test/transfer. If you get anything other than a 403, the panel's default configuration is your configuration. This is a five-minute test and it is the only one that settles the argument.
  • Find out whether your CA lets a requester choose the subject. Ask for a certificate with CN=payments-api from whatever enrolment path a developer or a laptop uses. If it issues, every CN-based allowlist in your estate is decoration. Certificate-management services differ enormously here, and the answer is a policy setting, not a property of X.509.
  • Log the issuer, not just the subject. Add $ssl_client_i_dn to your access log format alongside $ssl_client_s_dn. One week of that tells you which CAs are actually presenting certificates to you, which is almost never the list you expected, and it is the data you need before you can narrow the trust file without an outage.
  • Count the chain before you set the depth. openssl s_client -connect host:443 -cert c.pem -key k.key -showcerts from a caller prints the chain it offers. Intermediates between the leaf and your trust anchor must be at most ssl_verify_depth, and the default of 1 is one.
  • Check the purpose on every certificate you use as a client certificate. openssl x509 -in c.pem -noout -ext extendedKeyUsage. If it prints an extension that does not include TLS Web Client Authentication, that certificate stopped working as a client certificate — or will, at its next renewal.
  • Confirm the proxy is unavoidable. From a pod in the same namespace, call the application's port directly, bypassing the mTLS listener, with a forged X-Client-Cert header. If that works, mTLS is a decoration on one network path.

Your service mesh issues every workload a certificate from one internal CA. ssl_verify_client is on, that CA is the only entry in ssl_client_certificate, and your handler checks that $ssl_client_verify is SUCCESS. Which of these reaches /transfer?

The same distinction between "the credential verifies" and "this principal may do this" is the whole content of what a JSON Web Token signature check does and does not establish, and the client-side mirror image of this lesson — a client deciding to trust fewer issuers than its operating system does — is certificate pinning, and why pinning the wrong field breaks at renewal. If any part of the chain-building above was new, the four separate checks behind "the certificate is valid" is the lesson underneath this one.

Why this concept is on the site

Topics are chosen from places engineers visibly get stuck, and the sources are kept with the lesson so the claim is checkable.