TL;DR: Google TorchTPU is now open source. Announced on October 20, 2026 at PyTorch Conference 2026, it is a native PyTorch backend for Google’s Tensor Processing Units, co-developed with Meta’s PyTorch team. Google says existing PyTorch code can move to TPUs with minimal changes, often just switching .to("cuda") to .to("tpu"). The launch came a day after Amazon released TorchNeuron for its Trainium chips, and both moves aim to loosen Nvidia’s grip on how AI apps are built.
For most of the AI boom, PyTorch code has effectively meant Nvidia code. Google TorchTPU is the search giant’s latest and most serious attempt to change that. Here is what was released, what it supports, what Google and its partners are claiming, and what it could mean for developers and for Nvidia.
What Google TorchTPU is
In the Google Cloud announcement, Google describes TorchTPU as its native PyTorch stack for TPUs, now available as open source on GitHub. “With TorchTPU, TPUs join GPUs and CPUs as a native option for PyTorch,” the post says.
The PyTorch Foundation blog adds the history. Google first offered PyTorch on TPUs in 2020 through PyTorch/XLA, but that approach required code changes to mark compilable regions, suffered long compile times and lacked support for PyTorch’s normal “eager” execution mode. TorchTPU, started in mid-2025, was built instead as a proper TPU backend with eager execution as a top priority.
Key features at a glance
| Feature | What Google and PyTorch say |
|---|---|
| Install | pip install torch_tpu, per Google Cloud’s documentation |
| Execution modes | Eager mode for debugging, torch.compile for production performance |
| Operator coverage | About 450 ATen ops, including forward and backward passes for core ops |
| Model testing | Validated out of the box on the 5,000 most downloaded Hugging Face models |
| Custom kernels | Written via JAX/Pallas or the Helion DSL |
| Serving | vLLM, SGLang and llm-d integrations |
| Training | TorchTitan, VeRL (preview) and Ray support |
| Hugging Face | Transformers v5 deploys to TPUs with device_map="tpu" |
The ecosystem Google lined up
Google did not ship TorchTPU alone. Its announcement lists a series of partner integrations:
- vLLM: a native TorchTPU backend developed with Inferact, the company behind vLLM, supporting continuous batching, quantization and multi-chip parallelism.
- SGLang: a TPU backend developed with RadixArk for structured and agentic workloads.
- llm-d 1.0: released with Red Hat, IBM and, Google says, more than 500 contributors across 25 organizations, for routing inference across fleets of TPUs and GPUs on Kubernetes.
- TorchTitan: recipes for training large dense and mixture-of-experts models on TPUs.
- Ray: native TorchTPU support for orchestrating mixed CPU, GPU and TPU pipelines.
- LiteRT: Google’s on-device runtime, formerly TensorFlow Lite, joins the PyTorch ecosystem with a one-call converter for phones, laptops and embedded devices.
- TPU agent suite: an early preview of coding-agent plug-ins that help with onboarding, diagnostics and inference tuning.
Google’s performance claims
Google says TorchTitan training paths on TorchTPU reached up to 52% Model FLOPs Utilization on Qwen 3 8B, with near-linear scaling across up to 8,192 Ironwood TPUs (TPU v7x). It also quotes customer Zenteiq.ai, which says it achieved more than 4x higher aggregate inference throughput using GKE Inference Gateway powered by llm-d. These are company and customer figures, not independent benchmarks.
Partners were similarly upbeat. Meta’s Director of PyTorch Engineering, Jana van Greunen, is quoted by Google saying, “We cannot be constrained by a single hardware architecture,” and that TorchTPU supports major LLM, ads and recommendation workloads on key production use cases at Meta.
Why this is really about Nvidia
As TechCrunch reported, most AI apps are written by default to run on Nvidia GPUs, partly because PyTorch grew up on Nvidia hardware. Any company that wants to rent out its own AI chips has to make switching painless.
That is exactly what Amazon tried a day earlier. As we covered, Amazon TorchNeuron lets PyTorch code run on Trainium with a one-line change, and TechCrunch says Amazon is also submitting code to the main PyTorch project to decouple features from Nvidia’s chips. Google’s answer leans on breadth: it says it worked directly with Meta, Hugging Face, Inferact and RadixArk so that the tools developers already use behave the same on TPUs.
What it does not mean
TechCrunch also cautions that easier porting will not make developers abandon Nvidia en masse. Nvidia’s CUDA software, tooling and installed base remain deep advantages, and many teams will simply keep the option to move rather than move.
How to try TorchTPU
Google’s documentation and the TorchTPU getting-started guide describe a simple path for developers who already know PyTorch:
- Create a Cloud TPU VM in Google Cloud, which the guide calls the simplest way to start.
- Install the package with
pip install torch_tpu. - Select the device in your script with
torch.device("tpu")and move models and tensors with.to("tpu"). - Debug in eager mode first, then wrap the model with
torch.compileonce it runs correctly. - For Hugging Face models, pass
device_map="tpu"in Transformers v5, or use the vLLM and SGLang backends for serving.
According to Google’s engineering deep dive from April, TorchTPU uses PyTorch’s built-in interface for third-party devices rather than wrappers or tensor subclasses, the tensors you work with are ordinary PyTorch tensors, which is what makes the one-line device switch possible.
What it means for developers
- More bargaining power: if the same PyTorch code runs on GPUs, TPUs and Trainium, teams can shop on price and availability.
- Less rewrite risk: eager-mode support means debugging on TPUs should feel like normal PyTorch rather than a separate framework.
- Check your ops: roughly 450 supported ops is broad, but unusual custom layers may still need work or custom kernels.
- Cloud lock-in shifts, not disappears: TPUs are still mainly a Google Cloud product, so portability between clouds is what really improves.
The release also fits a busy week for open inference software, including DeepSeek’s open-sourced DeepInfer engine. And TPUs keep turning up in unexpected places, as shown by Google’s TPU in orbit.
The bottom line
Google TorchTPU is the most complete attempt yet to make TPUs a first-class PyTorch target, and its open-source release plus a long list of ecosystem integrations makes it easier to try than any previous Google effort. Whether it dents Nvidia’s lead will depend on real-world performance and pricing, which developers can now test for themselves.
The Teck Hustlers Editorial Team covers technology, business, finance and gaming news for readers who want the story behind the headline. Every article is researched, written and edited by our in-house team, checked against company announcements, filings and reputable reporting, and updated when new facts emerge. We focus on what each development means for founders, professionals, investors and everyday users. Have a tip or a correction? Reach us through our contact page.
