AI Has the Potential to Make One of Technology’s Most Valuable Resources Accessible to All

Nvidia remains the unrivaled leader in AI chip technology. However, as a result of its own innovations, the industry champion may soon encounter increasing competition.

Today’s AI relies heavily on Nvidia designs, a scenario that has catapulted the company’s market cap to over $4 trillion. Each successive Nvidia chip generation enables firms to train increasingly robust AI models by utilizing hundreds or thousands of processors interconnected within vast data centers. A significant factor in Nvidia’s triumph is its provision of software that facilitates programming each new chip generation. This advantage might not remain unique for long.

A startup named Wafer is developing AI models to tackle one of the most challenging and essential tasks in AI—optimizing code for efficiency on specific silicon chips.

Emilio Andere, Wafer’s co-founder and CEO, explains that the company employs reinforcement learning on open-source models to teach them to generate kernel code, which directly interacts with hardware in an operating system. Wafer also integrates “agentic harnesses” with existing coding models such as Anthropic’s Claude and OpenAI’s GPT to enhance their capability to produce code that operates directly on chips.

Numerous leading tech companies now possess their own chips. Apple and others have long utilized custom silicon to boost performance and efficiency for software operating on laptops, tablets, and smartphones. On a larger scale, companies like Google and Amazon create their own silicon to enhance their cloud-computing services. Recently, Meta announced it would allocate 1 gigawatt of compute power using a new chip developed with Broadcom. Creating custom silicon necessitates a substantial amount of coding to ensure it runs smoothly and efficiently on the new processor.

Wafer is collaborating with companies like AMD and Amazon to optimize software for effective performance on their hardware. To date, the startup has secured $4 million in seed funding from notable figures such as Google’s Jeff Dean and OpenAI’s Wojciech Zaremba.

Andere is confident that his company’s AI-driven approach could pose a challenge to Nvidia’s market strength. Various high-end chips now showcase comparable raw floating point performance—a crucial industry metric for a chip’s calculation capabilities—to Nvidia’s premium offerings.

“The top AMD hardware, the finest [Amazon] Trainium hardware, and the best [Google] TPUs all deliver similar theoretical flops to Nvidia GPUs,” Andere mentioned recently. “Our goal is to maximize intelligence per watt.”

Performance engineers skilled in optimizing code for reliable and efficient usage on these chips are both costly and in high demand, states Andere, while Nvidia’s software ecosystem simplifies writing and maintaining code for its chips. This challenge makes it difficult for even the largest tech companies to operate independently.

For instance, when Anthropic teamed up with Amazon to develop its AI models on Trainium, it had to completely rewrite its model’s code to ensure it operated as efficiently as possible on the hardware, according to Andere.

Of course, Anthropic’s Claude is now one of many AI models that excel at writing code. As a result, Andere believes it won’t be long before AI begins to diminish Nvidia’s software advantage.

“The moat exists in the programmability of the chip,” Andere observes, referencing the libraries and software tools that facilitate optimizing code for Nvidia hardware. “I believe it’s time to start reconsidering whether this is truly a strong moat.”

In addition to simplifying code optimization for various silicon, AI may soon revolutionize chip design itself. Ricursive Intelligence, a startup established by two former Google engineers, Azalia Mirhoseini and Anna Goldie, is innovating new methodologies for designing computer chips utilizing artificial intelligence. If successful, their technology could enable a wider array of companies to venture into chip design, producing custom silicon that executes their software more efficiently.