routstr-confidential is an experimental confidential mode for Routstr, a marketplace for AI inference where you pay a node with Cashu ecash or Lightning and the node forwards your request to a provider such as Venice with its own API key. In confidential mode the node can’t read your prompts or the answers, and you never see its API key. It needs no enclaves or other special hardware: it relies on the provider’s TLS certificate, a two-party computation and zero-knowledge proofs. I announced it in Confidential inference on Routstr without trusted hardware, which also covers the caveats and the measurements.
Try it, or read the code:
- the demo PWA: fund it with Lightning or a Cashu token and chat (source: routstr-confidential-pwa)
- routstr-confidential: the protocol, the paper and the setup guide
- routstr-core (branch
confidential-2pc): the node side - routstr-sdk (branch
confidential-2pc): the client transport - routstrd (branch
confidential-upstream): the opt-in switch in the local daemon - routstr-confidential-web: the library for using it in your own web app
The programmer’s guide
If you want the protocol itself rather than the magic show, I wrote a programmer’s guide: How routstr confidential mode works (PDF, 72 pages). It goes through the protocol step by step and describes the code in the repositories above. There is also a Slovak edition.
When you send a prompt to an AI model through a broker, the broker sees everything. In the routstr network the broker is called a node. It has an account with the provider and an API key, you pay it in small amounts of bitcoin e-cash or from a prepaid balance, and it forwards your request with its own key. That is convenient: you don’t need an account with the provider, and you can choose between many nodes. It also makes the node a perfect man in the middle. It reads your prompt and the answer, it could swap the model you asked for with a cheaper one or change a tool call, and it tells you what you owe.
The node has its own worry. Its API key lets anyone spend its money at the provider, so it can never hand that key to you.
The confidential version of routstr fixes both sides at once. The node forwards your request using its secret key and never sees your prompt or the answer. You talk to the provider over TLS, encrypted end to end, and never see the key. The bill is the number the provider itself reported. It needs no enclave or special hardware. It leans on the provider’s TLS certificate, the same one behind the padlock in your browser.
I told the protocol as a podcast episode, a sequence of magic tricks where each trick has one small job. The video above is that episode, illustrated (35 minutes). This post is the short version.
The setting: two people holding one pen
Your computer becomes the TLS client of the provider. The node carries the encrypted packets back and forth, like a postman who can’t open envelopes, and gets to write exactly one piece of the conversation: the request head with “Authorization: Bearer” and its API key. Everything else is encrypted with keys only you have.
Trick one: the family tree of keys
TLS 1.3 doesn’t use its shared secret directly. It grows a tree from it: a handshake secret, handshake keys, a master secret, and finally the keys for the conversation. Each step is a one-way hash, so you can go from parent to child but never back up, and one child tells you nothing about its sibling.
TLS 1.3 also has KeyUpdate: the client can switch to a new sending key, derived one step further down. The old key and the new key are siblings. Give the node only the old one and give you only the new one plus the key for the server’s replies. The node writes its header, sends the KeyUpdate (the handover of the pen), and from then on you write your prompt under a key the node doesn’t have. The provider sees a client that sent a header, rotated its key as TLS allows, and sent the rest.
Somebody still has to compute that tree, and nobody on stage is trusted by both sides. So you compute it together, and nobody ever holds the root.
Trick two: the sealed envelope
You and the node each pick a secret number and a public value, and the sum of the two public values goes to the provider. If the node showed its value first, you could pick yours to cancel it out and own the whole tree. So you commit first: you send a hash of your value mixed with randomness, a sealed envelope, and open it only after the node has shown its value. The node also checks your first TLS message and refuses to forward it unless it carries exactly the combined value. (This addition trick is why the protocol uses the P-256 curve; X25519 doesn’t allow it.)
Trick three: the secret nobody holds
The provider answers with its public value. A small conversion protocol leaves you and the node with two random-looking numbers that add up to the shared secret. The secret itself is never written down anywhere.
Trick four: the locked boxes
To compute the rest of the tree from two halves, the protocol uses a garbled circuit. The key computation is written as about 650,000 AND gates. The node invents two random labels for every wire, one meaning zero and one meaning one, and for every gate it builds four locked boxes, each holding the right output label, shuffled. You hold one label per wire, open exactly one box per gate, and walk through the circuit without ever knowing what any label meant.
At the end the node has the key for its single record, and you have the key for your request body and the key for the answer. The root, the handshake secret and the master secret existed only inside the boxes. The circuit also publishes hashes of the three keys, anchors that later proofs are tied to.
Trick five: choosing a card the magician can’t see
How do you get the labels for your own secret inputs without telling the node which ones? Oblivious transfer: the magician holds two face-down cards, you take the one you chose, you learn nothing about the other, and the magician doesn’t learn which you took. A few expensive base transfers get stretched (with algorithms called KOS and Ferret) into millions of cheap ones.
Trick six: catching a cheating magician
The node builds the boxes, so it could build a different circuit, say one that copies your key into its own output. This is the newest part of the protocol: authenticated garbling where only the node garbles and you check its work. The node commits to a secret mask for every wire, each stamped with a tag made from a key only you hold, like a wax seal pressed with your own signet ring. The stamps are linear, so you can check many gates at once. The node sends all the boxes (around 16 MB), and only then do you send a random seed that mixes every gate into one check. If even one gate is wrong, the check fails unless the node guesses a 64-bit number blind. You use no output until the check passes.
There is small print. A cheating node can make the check fail only in certain cases and learn something from your abort. Counting the possible endings of a session gives at most about 2.3 bits, and they are about values drawn fresh for that one session and thrown away. The API key never enters the circuit. This particular check has no published proof and no external review yet, and the project says so.
Trick seven: proving something without showing it
Zero-knowledge proofs are the “Where’s Waldo” trick: a big sheet of cardboard with a small hole, so you see Waldo but not where he is on the page. The protocol uses three of them, interactive ones that convince the other person in the session and nobody else.
- The node proves that its sealed record is the published request template with a key of the right shape in the key slot, encrypted with the key from the joint computation.
- You prove that your request ends with the agreed suffix: model name, maximum tokens, the price switches. The provider’s JSON parser lets the last field win, so the suffix overrides anything earlier. The node holds your last record back until the proof passes, and the provider does nothing until the body is complete.
- At the end you reveal only the provider’s usage event and prove it is the real end of the answer. No model output can fake one, because anything the model writes ends up inside an escaped JSON string. The project tried it.
Before proving anything, you check every parameter the node hands you against your own records, so the node can’t aim a proof at some other part of your traffic and use the yes or no as a hint.
The bill and the oven
Billing works like a hotel deposit. The node can’t see your prompt, so it reserves the worst case and then charges what the provider reported. In one example the deposit was 2,282 millisatoshis and the charge was 63. You get a signed receipt and check it against your own copy of the usage event.
All this costs bandwidth: about 22 MB from the node and 1.2 MB back per request, plus a few MB for the proofs. From a cold start the first token took 14 to 16 seconds. None of that depends on your prompt, though, so the client pre-heats sessions in advance and stops just before saying hello to the provider. With such an armed session, the first token came after about one second, most of it the provider’s own thinking.
What the magic doesn’t hide
The provider still sees everything: your prompt, the answer, and the node’s key. The node still sees the sizes and timing of records, and published attacks have reconstructed parts of AI answers from such patterns. The node could use a different account for its own record, but it can’t touch your content. And if it cuts the stream before the usage event, it keeps the deposit, which is the same trust you give a node today, capped at that deposit.
To the provider, the whole show looks like one ordinary TLS client. It has no idea two people are holding the pen.