Elon Musk’s just fired up Colossus—the world’s largest Nvidia GPU supercomputer built in just three months from start to finish

[Pelican Press 0]

This is the hidden content, please

• Sign In
• or
• Sign Up

Elon Musk’s just fired up Colossus—the world’s largest Nvidia GPU supercomputer built in just three months from start to finish

Say what you will about Elon Musk, but when the technological disrupter sets his mind to something, he plays to win.

Founded only in July of last year, his

This is the hidden content, please

• Sign In
• or
• Sign Up

, xAI, just brought a new supercomputer dubbed Colossus online during the Labor Day weekend designed to train its large language model (LLM) known as Grok, a rival to Open AI’s better known GPT-4.

While Grok is limited to paying subscribers of Musk’s

This is the hidden content, please

• Sign In
• or
• Sign Up

social media platform, many

This is the hidden content, please

• Sign In
• or
• Sign Up

experts speculate it will eventually form the artificial intelligence powering the EV manufacturer’s humanoid ******

This is the hidden content, please

• Sign In
• or
• Sign Up

.

Musk estimates this strategic lighthouse project could eventually earn Tesla $1 trillion in profits annually.

Located in Tennessee, the new xAI data center houses 100,000

This is the hidden content, please

• Sign In
• or
• Sign Up

benchmark Hopper H100 processors, more than 

This is the hidden content, please

• Sign In
• or
• Sign Up

.

“From start to finish, it was done in 122 days,” Musk wrote, calling Colossus “the most powerful AI training system in the world.”

This weekend, the

This is the hidden content, please

• Sign In
• or
• Sign Up

team brought our Colossus 100k H100 training cluster online. From start to finish, it was done in 122 days.

Colossus is the most powerful AI training system in the world. Moreover, it will double in size to 200k (50k H200s) in a few months.

Excellent…

— Elon Musk (@elonmusk)

This is the hidden content, please

• Sign In
• or
• Sign Up

It doesn’t end there for xAI, either: Musk estimates he will double Colossus’s compute capacity in a matter of months once he can procure 50,000 chips of Nvidia’s new, more advanced H200 series, which are roughly twice as powerful.

Musk and xAI did not respond to requests from Fortune for comment.

Built to train Grok-3, potentially the next leader in AI models

The pace at which Colossus has been set up is blistering given xAI had selected its Memphis site only

This is the hidden content, please

• Sign In
• or
• Sign Up

.

Moreover, several heavy-hitting tech firms, including

This is the hidden content, please

• Sign In
• or
• Sign Up

,

This is the hidden content, please

• Sign In
• or
• Sign Up

, and

This is the hidden content, please

• Sign In
• or
• Sign Up

, are competing to acquire Nvidia’s prized Hopper series AI chips in the current AI gold rush alongside Musk.

But the AI entrepreneur is a

This is the hidden content, please

• Sign In
• or
• Sign Up

of Nvidia and pledged to spend $3 billion to $4 billion this year on CEO Jensen Huang’s hardware—just for Tesla alone.

Moreover, xAI enjoyed a head start by 

This is the hidden content, please

• Sign In
• or
• Sign Up

already delivered to the EV manufacturer.

Story continues

The Memphis cluster will train Musk’s third generation of Grok.

“We’re hoping to release Grok-3 by December, and Grok-3 should be the most powerful AI in the world at that point,” he told ************* podcaster Jordan Peterson

This is the hidden content, please

• Sign In
• or
• Sign Up

.

An early beta of Grok-2 just rolled out to users 

This is the hidden content, please

• Sign In
• or
• Sign Up

.

It was trained on only around 15,000 Nvidia H100s graphic processors, yet by some standards, it is already among the most capable AI large language models, according to

This is the hidden content, please

• Sign In
• or
• Sign Up

.

Upping that GPU count nearly sevenfold suggests Musk has no intention of surrendering the race to develop artificial general intelligence to OpenAI, which he helped cofound in late 2015 after becoming worried that

This is the hidden content, please

• Sign In
• or
• Sign Up

was dominating the technology.

Musk later fell out with CEO Sam Altman and is now

This is the hidden content, please

• Sign In
• or
• Sign Up

the company a

This is the hidden content, please

• Sign In
• or
• Sign Up

.

To help even the odds, xAI

This is the hidden content, please

• Sign In
• or
• Sign Up

This is the hidden content, please

• Sign In
• or
• Sign Up

in a Series B funding round in May, with the help of venture capitalists like Andreessen Horowitz and Sequoia Capital, as well as deep-pocketed investors like Fidelity and Saudi Prince Alwaleed bin Talal’s Kingdom Holding.

Tesla could be the next company to invest in Musk’s xAI

Musk has also indicated he would

This is the hidden content, please

• Sign In
• or
• Sign Up

a vote on whether to invest potentially $5 billion into xAI as well, a step welcomed by a number of shareholders.

Of the roughly $10B in AI-related expenditures I said Tesla would make this year, about half is internal, primarily the Tesla-designed AI inference computer and sensors present in all of our cars, plus Dojo.

For building the AI training superclusters, NVidia hardware is about…

— Elon Musk (@elonmusk)

This is the hidden content, please

• Sign In
• or
• Sign Up

xAI’s supercomputer cluster has caused alarm in Memphis, however, given the extreme haste with which city officials agreed to the project, which brings economic activity back to a part of the city that last housed an

This is the hidden content, please

• Sign In
• or
• Sign Up

factory for white goods.

One main concern is the strain it will create on the city’s resources. 

This is the hidden content, please

• Sign In
• or
• Sign Up

 of municipal utility MLGW estimate that Colossus requires up to 1 million gallons of water per day to cool the servers and will consume as much as

This is the hidden content, please

• Sign In
• or
• Sign Up

.

But Musk is someone who only thinks big, and anything worth doing is worth doing fast—otherwise you risk falling behind the competition.

Speaking to Lex Fridman after the podcaster visited xAI’s rapidly growing operations, Musk said speed was a key part of his five-step management process.

“Any given thing can be sped up. However fast you think it can be done,” he said

This is the hidden content, please

• Sign In
• or
• Sign Up

, “it can be done faster.”

This story was originally featured on

This is the hidden content, please

• Sign In
• or
• Sign Up

This is the hidden content, please

• Sign In
• or
• Sign Up

#Elon #Musks #fired #Colossusthe #worlds #largest #Nvidia #GPU #supercomputer #built #months #start #finish

This is the hidden content, please

• Sign In
• or
• Sign Up

This is the hidden content, please

• Sign In
• or
• Sign Up