The key facts

  • Google released TorchTPU as an open-source project — a PyTorch-compatible environment for TPU chips
  • Amazon introduced TorchNeuron for Trainium chips a day earlier and is contributing changes to the main PyTorch project
  • By some estimates, PyTorch is used in 85% of AI applications; both companies seek to weaken Nvidia's position

Google released TorchTPU

On Tuesday, October 20, Google announced that TorchTPU — its long-awaited version of PyTorch — is now available as an open-source project. With this step, Google joins Amazon in an attempt to weaken Nvidia's dominance in the AI industry, where most AI applications are written by default to run on Nvidia chips (GPUs).

Context: PyTorch and its connection to Nvidia

PyTorch is a popular open-source toolkit that AI application developers and startups use to build, train, and run models. By some estimates, it is used in 85% of AI applications. PyTorch was created by Meta (then still Facebook) in 2016, when Nvidia chips were already the standard for AI research, so many of the library's features were originally developed for them. In 2022, Meta transferred PyTorch to the Linux Foundation.

What TorchTPU offers

TorchTPU allows AI applications to run on Google's chips — Tensor Processing Units (TPUs) — and simplifies porting existing PyTorch applications to TPUs and, accordingly, to Google Cloud. Developers only need to change the configuration; there is no need to rewrite applications.

Amazon's answer: TorchNeuron

On Monday, Amazon introduced its own version of PyTorch — TorchNeuron, which allows AI applications to be ported to its own Trainium chip and to AWS, also without rewriting code. Amazon said it is going further: the company is contributing code to the main PyTorch project that decouples various functions from Nvidia chips, making the library more open to any hardware by default.

How Google strengthens its position

According to TechCrunch, Google may surpass Amazon in some respects: the company says it worked directly with Meta, Hugging Face, Inferact (the developer of vLLM), RadixArk (the developer of SGLang), and others to ensure that TorchTPU's features and libraries work correctly with popular development tools, just like the original PyTorch. In addition, Google launched a special set of tools, the TPU agent suite, for porting applications to its chips and cloud with error fixing along the way, and also updated the LiteRT tool for running AI on phones, laptops, and embedded devices so that it works better with PyTorch applications.

Caveat

As TechCrunch notes, all of this does not mean that AI application developers will automatically abandon Nvidia hardware en masse simply because it is now easier for them to switch to alternative chips and clouds. However, Amazon and Google are clearly working to eliminate Nvidia's advantage.