The Surge in AI Is Driving a Demand for Faster Chip Connectivity

A new chapter in Silicon Valley is shaping up—not the networking you link to on LinkedIn.

With the tech sector pouring billions into AI data centers, both large and small chip manufacturers are intensifying their efforts around the technologies connecting chips to each other, as well as server racks to one another.

Networking tech has existed since the inception of computers, crucially linking mainframes for data sharing. In the semiconductor landscape, networking is integral at nearly every layer—from interconnects between transistors on the chip to external links between clusters or racks of chips.

Chip leaders like Nvidia, Broadcom, and Marvell already boast strong credentials in networking. However, amid the AI surge, some firms are exploring fresh networking methodologies aimed at accelerating the enormous volumes of digital data traversing data centers. This is where deep-tech startups such as Lightmatter, Celestial AI, and PsiQuantum come into play, employing optical technology to boost high-speed computing.

Optical technology, or photonics, is experiencing a renaissance. Once deemed “lame, expensive, and marginally useful” for 25 years, its relevance has been revived by the AI boom, according to PsiQuantum co-founder and chief scientific officer Pete Shadbolt, who participated in a recent panel co-hosted by WIRED.

In hopes of riding the next wave of chip innovation or uncovering a prime acquisition target, venture capitalists and institutional investors are channeling billions into startups that have devised novel strategies for enhancing data throughput. They argue that traditional interconnect technologies, reliant on electrons, cannot keep up with the rising demands of high-bandwidth AI applications.

“Historically, networking was quite dull to report on, due to its focus on switching packets of bits,” explains Ben Bajarin, a veteran tech analyst and CEO of Creative Strategies. “Now, with AI, it must handle significantly heavier workloads, driving innovation around speed.”

The Power of Chips

Bajarin and others recognize Nvidia’s foresight regarding the significance of networking, particularly following two key acquisitions in the sector. In 2020, Nvidia invested nearly $7 billion to acquire Israeli firm Mellanox Technologies, which specializes in high-speed networking solutions for servers and data centers. Shortly thereafter, Nvidia bought Cumulus Networks to enhance its Linux-based computer networking software. This marked a pivotal moment for Nvidia, which correctly predicted that the GPU and its parallel-computing strengths would become far more effective when clustered with other GPUs in data centers.

While Nvidia leads in vertically-integrated GPU solutions, Broadcom has emerged as a dominant player in custom chip accelerators and high-speed networking technology. The $1.7 trillion firm collaborates closely with companies like Google, Meta, and more recently OpenAI, on chips intended for data centers. It is also spearheading advancements in silicon photonics. Recently, Reuters reported that Broadcom is preparing a new networking chip named Thor Ultra, aimed at providing a “critical link between an AI system and the remainder of the data center.”

During its earnings call last week, semiconductor design behemoth ARM disclosed plans to acquire the networking enterprise DreamBig for $265 million. DreamBig develops AI chiplets—compact modular circuits intended for integrative packaging in larger chip systems—collaborating with Samsung. ARM CEO Rene Haas mentioned the startup possesses “intriguing intellectual property … which is essential for scalable networking” during the earnings call. (This involves connecting components and transmitting data within a single chip cluster, as well as linking racks of chips with each other.)

Illumination Ahead

Lightmatter CEO Nick Harris noted that the computing power required for AI is now doubling every three months—far outpacing Moore’s Law. Computer chips are growing larger. “When you reach the maximum size for chip production, all subsequent performance enhancements stem from interconnecting the chips,” Harris states.

His company’s innovative approach does not depend on traditional networking. Lightmatter designs silicon photonics to interlink chips. It claims to have created the world’s fastest photonic engine for AI chips, essentially a 3D silicon stack linked via light-based interconnect technology. Over the past two years, the startup has secured more than $500 million in funding from investors like GV and T. Rowe Price, achieving a valuation of $4.4 billion last year.