Shortly after Amazon CEO Andy Jassy introduced AWS’s groundbreaking $50 billion funding cope with OpenAI, Amazon invited me on a personal tour of the chip improvement lab on the coronary heart of the deal, at (principally*) its personal expense.
Industry experts are watching Amazon’s Trainium chip, created at that facility, for its implications for lower-cost AI inference and, probably, a dent in Nvidia’s close to monopoly.
Curious, I agreed to go.
My tour guides for the day have been the lab’s director, Kristopher King (pictured under proper) and director of engineering Mark Carroll (under left), in addition to the staff’s PR one that organized the go to, Doron Aronson (pictured with yours really later within the story).
AWS has been Anthropic’s main cloud platform for the reason that AI lab’s early days — a relationship vital sufficient to outlive Anthropic later including Microsoft as a cloud associate as effectively, and Amazon’s rising partnership with OpenAI.
The OpenAI deal makes AWS the unique supplier of the mannequin maker’s new AI agent builder, Frontier, which might grow to be an vital a part of OpenAI’s enterprise if brokers grow to be as massive as Silicon Valley thinks they may. We’ll see if that exclusivity stands precisely as introduced. The Monetary Instances reported this week that Microsoft could consider OpenAI’s cope with Amazon violates its personal cope with OpenAI, specifically with Redmond getting entry to all of OpenAI’s models and tech.
What makes AWS so interesting to OpenAI? As a part of this deal, the cloud big has agreed to produce OpenAI with 2 gigawatts of Trainium computing capability. It is a big dedication, on condition that Anthropic and Amazon’s personal Bedrock service are already consuming Trainium chips quicker than Amazon can produce them.
Techcrunch occasion
San Francisco, CA
|
October 13-15, 2026
There are 1.4 million Trainium chips deployed throughout all three generations, and Anthropic’s Claude runs on over 1 million of the Trainium2 chips deployed, the corporate stated.
It’s value noting that whereas Trainium was initially geared towards quicker, cheaper mannequin coaching (a much bigger precedence a few years in the past), it’s now tuned and used for inference as effectively. Inference — the method of truly operating an AI mannequin to generate responses — is presently the most important efficiency bottleneck within the trade.
Living proof: Trainium2 handles nearly all of the inference visitors on Amazon’s Bedrock service, which helps the constructing of AI functions by Amazon’s many enterprise clients and permits the apps to make use of a number of fashions.
“Our buyer base is simply increasing as quick as we are able to get capability on the market,” King stated. “Bedrock might be as massive as EC2 at some point,” he added, referring to AWS’s behemoth compute cloud service.
Trainium vs. Nvidia
Past providing an alternative choice to Nvidia’s backlogged, hard-to-acquire GPUs, Amazon says its new chips operating on its new specialty Trn3 UltraServers value as much as 50% much less to run for comparable efficiency than utilizing traditional cloud servers.
Together with Trainium3, launched in December, this AWS staff additionally constructed new Neuron switches, and Carroll says that combo is transformative.
“What that provides us is one thing large,” Carroll stated. The switches enable each Trainium3 chip to speak to each different chip in a mesh configuration, decreasing latency. “That’s why Trainium3 is breaking all types of information,” notably in “worth per energy,” he stated.
When trillions of tokens a day are concerned, such enhancements add up.
In actual fact, Amazon’s chip staff was lauded by Apple in 2024. In a uncommon second of openness for the secretive firm, Apple’s director of AI publicly described the way it used one other of the staff’s chips — Graviton, a low-power, ARM-based server CPU and the primary breakout chip this staff designed. Apple additionally lauded Inferentia — a chip particularly designed for inference — and gave a nod to Trainium, which was new on the time.
These chips characterize the traditional Amazon playbook: See what individuals need to purchase, then construct an in-house various that competes on worth.
The catch for chips, traditionally, has been switching prices. Purposes written for Nvidia’s chips should be re-architected to work with others — a time-consuming course of that daunts builders from switching.
However the AWS chip staff proudly instructed me that Trainium now helps PyTorch, a well-liked open supply framework for constructing AI fashions. That features lots of the ones hosted on Hugging Face, an enormous library the place builders share open supply fashions.
The transition, Carroll instructed me, requires “mainly a one-line change, after which recompile, after which run on Trainium.” In different phrases, Amazon is making an attempt to chip away at Nvidia’s market dominance wherever attainable.
AWS has additionally this month introduced a partnership with Cerebras Systems, integrating that firm’s inference chip on servers operating Trainium for what Amazon guarantees can be superpowered, low-latency AI efficiency.
However Amazon’s ambitions transcend the chips themselves. It additionally designs the server that hosts the chips. Apart from the networking parts, this staff has designed “Nitro,” a hardware-software combo that gives virtualization tech (which permits many situations of software program to run individually on the identical server); new state-of-the-art liquid cooling know-how; and the server sleds (pictured under) that host this gear.
All of that’s to manage value and efficiency.
Working 24/7 on the “bring-up”
Amazon’s {custom} chip-designing unit was born when the cloud big bought Israeli chip designer Annapurna Labs in January 2015 for about $350 million. So this staff has now had greater than 10 years designing chips for AWS. The unit has retained its Annapurna roots and title — its brand is all over the place within the workplace.
This chip lab is positioned in a shiny, chrome-windowed constructing in Austin’s upscale “The Area” district, a walkable space full of outlets and eating places that’s typically referred to as Austin’s Silicon Valley.
The places of work have your traditional tech company vibe: desks in cubicles, gathering spots, and convention rooms. However tucked away in the back of a excessive flooring within the constructing is the precise lab, with sweeping views of the town.
The shelving-filled lab, concerning the measurement of two massive convention rooms, is a loud industrial area due to the followers on the tools. It seems like a cross between a highschool store class and a Hollywood set for a high-end lab, besides the engineers are wearing denims, not white lab coats.
Notice that this isn’t the place the chips are manufactured, so no white hazmat fits have been vital. The Trainium3 is a state-of-the-art 3-nanometer chip, produced by TSMC, arguably the chief in 3-nanometer manufacturing, with different chips produced by Marvell.
However that is the room the place the magic of the “bring-up” happens.
“A silicon bring-up is once you get the chip for the primary time, and it’s like an enormous in a single day occasion. You keep right here, like a lock-in,” King explains. After 18 months of labor, the chip is activated for the primary time to confirm it really works as designed. The staff even filmed among the Trainium3 bring-up and posted it on YouTube.
Spoiler alert: It’s by no means problem-free.
For Trainium3, the prototype chip was initially air-cooled, like earlier variations. The present chip is now liquid-cooled, which presents vitality benefits and was fairly an engineering feat.
Throughout the bring-up, the scale for the way the chip connected to the air-cooling warmth sink have been off, so the chip couldn’t be activated.
Unfazed, the staff “instantly obtained a grinder and simply began grinding off the metallic,” King stated. As a result of they didn’t need the noise disrupting the bring-up pizza occasion ambiance, they snuck off and did the grinding in a convention room.
Staying up all night time and fixing issues “is what silicon bring-up is all about,” King stated.
The lab even has a welding station, the place {hardware} lab engineer and grasp welder Isaac Guevara demonstrated welding tiny built-in circuit parts by way of a microscope. That is such insanely troublesome work that senior chief Carroll brazenly admitted he couldn’t do it, to the guffaws of Guevara and the remainder of the engineers within the room.
The lab additionally accommodates each custom-made and industrial instruments for testing and analyzing points with chips. Right here’s sign engineer Arvind Srinivasan demonstrating how the lab assessments every tiny part on the chip:
Sleds are the star of the lab
However the star of the lab is a complete row showcasing every technology of the “sleds” the staff designed.
Sleds are the trays that home the Trainium AI chips, Graviton CPU chips, and supporting boards and parts. Stack them collectively on a rack with the networking part, additionally custom-designed by this staff, and also you get the methods which might be on the coronary heart of Anthropic Claude’s success.
Right here’s the sled that was proven off throughout the AWS re:invent convention in December:
Confirmed by Anthropic and OpenAI
I anticipated my guides to crow concerning the OpenAI deal throughout the tour. However they didn’t.
The reticence might have been associated to the aforementioned potential authorized haze that may dangle over the deal. However the sense I obtained was that these boots-on-the-ground engineers (who’re presently designing the following model, Trainium4) haven’t had a lot likelihood to work with OpenAI but. Their day-to-day work has thus far been centered on Anthropic’s and Amazon’s wants.
Presently, the most important chunk of Trainium2 chips is deployed in Challenge Rainier — one of many world’s largest AI compute clusters — which went dwell in late 2025 with 500,000 chips. It’s utilized by Anthropic.
However there was a wall monitor in the principle workplace displaying a quote about how OpenAI can be utilizing Trainium. The delight was there, if refined.
Along with this lab, the staff additionally has its personal non-public information heart for high quality and testing functions. A brief drive away, it doesn’t run buyer workloads, so it’s housed at a co-location facility, not an AWS information heart.
Safety is tight: There are strict protocols to enter the constructing and to entry Amazon’s space inside.
The information heart’s cooling system is so loud that earplugs are obligatory, and the air is thick with the acrid odor of heated metallic. It’s not a nice place for the common particular person to hang around.
At this information heart, there are rows and rows of servers full of sleds that combine all of Amazon’s latest {custom} chips: Graviton CPU, liquid-cooled Trainium3, Amazon Nitro, all fortunately computing away. The liquid runs on a closed system, which means it’s reused, which also needs to assist cut back the environmental affect, the engineers stated.
Right here’s what a present Trn3 UltraServer seems like: A number of sleds are on prime and backside, with the Neuron switches within the center. {Hardware} improvement engineer David Martinez-Darrow is seen right here performing upkeep on a sled:
Whereas consideration on the staff has at all times been excessive, the scrutiny has actually ratcheted up as of late.
Amazon CEO Andy Jassy retains a detailed eye on this lab, publicly bragging about its merchandise like a proud dad. In December, he stated Trainium was already a multibillion-dollar enterprise for AWS and called it one piece of AWS tech he’s most enthusiastic about. He additionally gave the chip a shout-out when saying the OpenAI settlement.
The staff feels the stress, too. Engineers will work 24/7 for 3 to 4 weeks round every bring-up occasion to repair any points so the chips might be mass-produced and put into information facilities.
“It’s crucial that we get as quick as attainable to show that it’s truly going to work,” Carroll stated. “To this point, we’ve been doing very well.”
*Disclosure: Amazon supplied airfare and coated the price of one night time at a neighborhood resort. Honoring its Leadership Principle of Frugality, this was a back-of-the-plane center seat and a modest room. TechCrunch picked up the opposite related journey prices like Ubers and baggage charges. (Sure, I checked a bag for an in a single day journey. I’m excessive upkeep that method.)
