The key facts

  • AWS released TorchNeuron: PyTorch applications run on Trainium with a single line of configuration changed, according to Amazon
  • Amazon announced plans to make changes to PyTorch that decouple the framework from being tied to specific chips
  • AWS cites the example of Deepgram: 70% more audio processed per dollar at production quality

A new tool against Nvidia's software advantage

On Monday, October 19, Amazon Web Services introduced the TorchNeuron tool, which allows AI startups to port their applications to Amazon's proprietary Trainium chip. With this move, the company is attacking Nvidia's software advantage, which has made its GPUs the default choice for AI applications.

How TorchNeuron works

TorchNeuron is compatible with PyTorch — a free open-source set of tools for building, training, and running AI models. According to Amazon, a PyTorch application can be run on Trainium by changing just one line of configuration, without rewriting code. Amazon claims that about 85% of AI applications are written in PyTorch. The framework was originally developed by Facebook, and since 2022 it has been part of the Linux Foundation. Nvidia has provided seamless PyTorch support in its CUDA software infrastructure for years, so its hardware is effectively built into the framework by default.

Not the first porting effort

Amazon is not the first company to simplify porting to its chips: PyTorch users could previously port applications to AMD GPUs or Apple chips simply by changing the configuration. However, AWS emphasizes that porting to Trainium gives startups access to Amazon's entire cloud ecosystem, unlike 'non-cloud' providers tied to Nvidia — such as Coreweave, Nebius, and Firmus.

AWS's arguments

“You build on AWS. You don't bet on a narrowly specialized compute provider and frantically build everything else around it,” Colin Brace, vice president of technology software at AWS Annapurna Labs, the division that develops Amazon's proprietary chips, told TechCrunch by email. According to him, the infrastructure for moving from prototype to production already exists, allowing founders to move fast and scale. AWS claims that since the same chip handles both training and inference and is tightly integrated with its cloud infrastructure, its cloud will save startups money even if the price per token is not the lowest. “The value of Trainium is the price-to-performance ratio plus a single platform for training and inference, which improves the economics of the full workload,” Brace added. He cited the example of voice AI application provider Deepgram (a competitor to ElevenLabs), which moved to Trainium with native PyTorch and processes 70% more audio per dollar at production quality.

Changes to PyTorch itself

Amazon said it is not limiting itself to convenient porting to its own cloud: the company is making changes to the core PyTorch project to remove code that ties the framework to specific chips. “Amazon's approach goes beyond its own hardware. The company is making upstream changes to PyTorch itself, working with the PyTorch community to remove code that ties the framework to specific chips. The goal is to make PyTorch more hardware-agnostic for the entire developer community, not just Trainium users,” Amazon's blog states. Brace called this “the real story — eliminating the lock-in that denied developers real choice.”

Open questions

TechCrunch notes that one of PyTorch's tasks is to hide the complexity of running AI code on hardware, and it remains unclear whether pushing Nvidia out of the framework will give developers more choice and more affordable options, as Amazon promises, or merely additional configuration difficulties. At the same time, if Amazon succeeds in decoupling PyTorch from Nvidia, the consequences for the GPU market leader could extend far beyond chip-level competition.