I recently wrote about hardware attestation from the angle of your phone: how a chip’s factory-signed key lets a server verify it’s talking to a genuine, unmodified app running on a genuine, unmodified, non-rooted device, and how that same mechanism is quietly locking you out of your own hardware. That post ended with Signal’s SGX-based contact discovery as proof the technology can cut the other way, toward privacy, not just lock-in. This is the follow-up: what happens when you point that same idea at AI inference.
The problem with renting someone else’s GPU
Services like Claude, OpenAI’s GPT, OpenRouter and friends run very good models. But you don’t actually know what they do with your prompts. Most of the time that’s fine. Sometimes it isn’t: you can’t run inference on a sensitive project, say a security audit of unreleased code, through a service whose only promise is a policy document. And when you’re under an NDA, you can’t rely on a pinky promise from the inference provider not to log.
The extreme solution: buy the hardware yourself
Many people turn to the obvious fix: buy inference hardware. For a genuinely good model like the new DeepSeek v4 Flash 0731, that probably means 2x DGX Spark, a beefy Mac Mini, or a professional GPU cluster. The problem is you can only run one model at a time, you can’t just switch, and every second that hardware isn’t doing inference is money burning quietly in a rack.
It’s also not very capital efficient. For a genuinely cheap model like DeepSeek, work out the return on investment: API cost versus tokens versus running your own hardware full tilt, and when the hardware would actually pay itself back. DeepSeek tokens are extremely inexpensive on the API. Running your own hardware isn’t a strategy to save money on tokens; it’s not economically sound on that basis alone. And once you try to make it economical by sharing the machine (through a cloud provider, since these are open-weight models that can run anywhere), you’re back to the same problem: you don’t know whether that provider is logging.
The middle ground: rented hardware you can verify
A good option is to gain access to inference hardware you couldn’t afford to buy, run different models, switch between them, use it from opencode (a terminal-based AI coding agent) or whatever else, but still get encryption. Maple AI and Phala Network specialize in this space; Venice, a broader AI API service, offers it as one tier among several.
You have inference hardware, and what runs on it is a fixed inference container called an enclave, like a virtual machine. In this case it runs isolated from the rest of the machine, so even the hypervisor cannot see into the container’s memory. You can verify its hash, so you know what is running there. The keypair is generated inside that enclave. The CPU attests that the container ran, generated its own keys, and that the resulting public key really was generated inside the enclave. The private key exists nowhere else. If the machine reboots, the key is gone; the next boot generates a fresh one.
So when you start a session with the provider, you negotiate keys and verify the attestation, meaning the public key you’re handed really was generated inside an enclave running a known, published hash. It’s as if you were running the model yourself, except you’re not: you just know exactly which container is running, because it’s open source and published, and you can be sure that once you encrypt your prompt to that public key and hand over a key for the response, nothing fishy happened. The container cannot log from inside the enclave, and you can verify that from the attestation.
That’s the theory, and it’s how it’s supposed to work. In practice there’s a lot more plumbing.
Where it actually gets complicated: Venice
Venice needs API key verification, usage accounting so it knows how many tokens and how much credit you’ve spent, a router to the provider, and a provider that routes to the actual inference machine. All of that needs to be end-to-end attested too, ideally the whole path, ending with your prompt on the inference machine itself.
That’s not quite what happens with Venice today. Encryption terminates at the router, not the inference machine. It can still work, though. The router is attested, and it opens a TLS connection to the inference machine, which is also attested, even though you personally only see the router’s verification of that. You trust it because the router’s own code, which performs that check and fails if the check fails, is itself attested. You know the check happens because you verified the code that performs it. On top of that, the call to the GPU is verified as going to a genuine Nvidia chip, not some logging proxy sitting in between. Chain enough of these hops together, verifying one machine and having it vouch for the next, and you can be fairly confident nothing was logged end to end.
It’s possible to do this right, and it would be genuinely useful. Apple is one example of a company providing its own confidential compute hardware. Other inference providers could use Apple’s stack, or confidential compute based on AMD or another CPU vendor. In that case, the inference provider is not Apple. Apple, AMD, Intel, Nvidia and the other hardware companies provide the attested compute layer; a provider such as Venice, Phala or Maple could build a service on top of it.
You might not like Intel, Apple, Nvidia, AMD or any of these companies. That is a reasonable position. Breaking the trust of an enclave still requires either a security bug or active collusion. Security bugs have happened before, and when one is found, you know not to trust attestations from the affected hardware until it is fixed. The other possibility is that a CPU vendor gives an inference provider keys or certificates that can sign arbitrary attestations. I don’t see Intel taking that call from Phala, Venice or Maple. These are small players compared with Intel and the others, and if such a scheme were discovered, the hardware vendor’s reputation would be destroyed. That is a risk none of them can sensibly take.
What the audit found
I’m the author of venice-e2ee-proxy, built on the Elkim venice-e2ee library. I spent a few days improving the verification and testing what Venice actually exposes.
The proxy encrypts prompts on your machine to a key carried in an Intel TDX (Trust Domain Extensions) attestation. Venice’s network and infrastructure see ciphertext. Decryption happens inside a Phala-operated enclave, the gateway, which forwards the plaintext to an attested inference router and then to a GPU node. The gateway checks the next machine’s attestation and binds the TLS connection to the key inside that attestation. I verified the chain through the second hop. The full security audit also found important limits: the gateway runs a development dstack (an open-source framework for deploying TEE applications) image and builds its code at boot using a persistent cache, rather than running a pinned binary whose exact code is measured (hashed and recorded in the attestation, so anyone can verify what actually ran); a root SSH key may be injected into the enclave, and the attestation does not tell you whether one was; the measured settings allow public logs and debug error logging that could expose fragments of failed requests; upstream verification is recorded but not fail-closed on Venice’s route; and the serving software and model weights remain outside the measured claims. The attestation therefore proves a narrower configuration than the phrase “private inference” might suggest, but it still gives you substantially more evidence than an ordinary provider’s promise not to log.
I’m not the only one poking at this. Andrew Miller (as he mentioned on X) runs awesome-private-inference, a live dashboard that probes roughly twenty confidential-inference providers daily and checks what their attestations actually prove versus what they claim. His underlying methodology and per-provider write-ups live in the devproof-audits-guide case studies, including a dedicated report on Venice. It confirms independently what I found: attestation covers pieces of the pipeline, not the whole thing, so a provider’s “private inference” claim is worth checking rather than accepting outright.
Two ways to actually use this
Integration comes in two shapes. An app can use the verification directly as a library. This is what getbased.health does, and the attestation check runs inside your browser in JavaScript as the page loads. The browser performs the verification itself, without inserting another service between you and the proof.
Or, for anything that expects a normal OpenAI-compatible API, opencode, your own scripts, whatever, you run a local proxy that spawns an OpenAI-shaped endpoint on localhost, handles the attestation and key exchange under the hood, and encrypts/decrypts transparently. That’s the venice-e2ee-proxy approach: same guarantee, different integration point, trading browser-native simplicity for compatibility with existing tools.
The usability gap, and it’s Venice’s fault
The real usability gap is tool calling. The E2EE path can’t carry structured tool call parameters, because the API doesn’t support encrypting them. The library has to fall back to inferring tool calls out of the model’s plain text response, which is flaky by nature. Run something agentic like opencode through it and it holds together for a few exchanges, then breaks. This is the single highest leverage fix available: solve encrypted tool calling and the E2EE path stops being a novelty and starts being something you could actually run agents through daily.
Why the top tier isn’t the obvious answer
Buying your own hardware, the extreme option from earlier, sounds like the obvious answer until you run the numbers. Take two DGX Sparks clustered together at roughly $10,000 for the pair, pooling enough unified memory to run something in the 200B+ parameter range. Realistically you’re looking at a two year window before next generation hardware or next generation model requirements make that setup feel dated. This stuff moves fast enough that “buy once, run forever” isn’t the world we’re in. That’s roughly $417/month in capital alone, before power. Even light to moderate use at a few hundred watts adds another $40 to $60/month. Call it $460 to $480/month, fixed, whether you use it or not.
Now compare that to actually cheap API inference, something like DeepSeek at well under $1 per million tokens blended. To make $460/month in hardware costs pencil out against that, you’d need to be burning somewhere north of half a billion tokens a month, sustained. That’s an enormous amount of inference. Most power users, even ones running multiple daily agents and pipelines, are nowhere close to that. Which means for the vast majority of people the hardware never actually pays itself back before it’s obsolete.
And there’s a second trap layered on top. If your usage is light enough that the economics don’t already favor the API, the hardware sits idle most of the time, which is the worst possible use of $10,000. But if you push your usage up high enough to actually justify owning it, you run into the opposite problem: this class of hardware isn’t built for serving many concurrent requests in parallel, so under real load it queues and slows down exactly when you need it to keep up. You either own expensive hardware that mostly sits idle, or you own hardware that’s finally earning its keep and is too slow to be pleasant to use. The economics do not offer a comfortable middle.
Conclusion
The options form a rough hierarchy. At the bottom are providers that openly log prompts and hand them over when legally required. Above them are anonymized inference services and providers that promise not to log. Venice’s private tier belongs here: the protection is mostly contractual, so you still trust the provider’s policy. Above that are TEE and E2EE systems, where enclaves and attestation let you check more of the claim yourself. At the top is your own hardware, where another operator is not involved.
Your own hardware gives you the cleanest trust model, but it costs about $10,000 for two DGX Sparks, requires power and maintenance, and becomes outdated quickly. For many people, the TEE and E2EE tier might be the sweet spot: cheaper and more flexible than ownership, with more evidence than a promise from an API provider.
It isn’t perfect, but it’s better than just sending the prompts. And hopefully, after a blog like this one, it will become a better when providers fix bugs.