What will future chips look like?

2024-06-15

What will future chips look like? Who will manufacture them and what new technologies will they unlock?

The world of chips is at the forefront of a massive shift due to the booming development of artificial intelligence. The demand for chips that can train AI models faster and operate them from devices such as smartphones and satellites, allowing us to use these models without leaking private data, is continuously increasing. Governments, tech giants, and startups are all competing to get a slice of the growing semiconductor market.

Here are four trends to watch in the coming year that will define what future chips will look like, who will manufacture them, and what new technologies they will unlock.

CHIPS activities are taking place worldwide

The world's two largest chip manufacturers, TSMC and Intel, are racing to build campuses in the desert, hoping that these campuses will become the home of American chip manufacturing strength. A common point of these efforts is funding: in March, President Joe Biden announced $8.5 billion in direct federal funding and $11 billion in loans for Intel's expansion across the country. A few weeks later, TSMC announced an investment of $6.6 billion.

Advertisement

These awards are just part of the subsidies that the United States is pouring into the chip industry through the CHIPS and Science Act signed in 2022, which amounts to $280 billion. This funding means that any company involved in the semiconductor ecosystem is analyzing how to restructure its supply chain to benefit. Although most of the funding is aimed at promoting American chip manufacturing, there is still room for other players to apply.The United States is not the only country attempting to localize part of its chip manufacturing supply chain. Japan itself has spent $13 billion, equivalent to the "Chips Act," Europe is set to spend over $47 billion, and earlier this year, India announced an investment of $15 billion to build local chip factories. Chris Miller, a professor at Tufts University and author of the book "Chip War: The Battle for the World's Most Critical Technology," says that the root of this trend can be traced back to 2014. Since then, China has begun to provide substantial subsidies to its chip manufacturers.

"This has created a dynamic where other governments have concluded that they have no choice but to offer incentives or see businesses shift manufacturing to China," he said. This threat, coupled with the surge in artificial intelligence, has led Western governments to fund alternatives.

Miller stated that this funding is unlikely to give rise to entirely new chip competitors, nor is it likely to restructure the largest chip manufacturers. Instead, it will primarily incentivize dominant companies like TSMC to establish roots in multiple countries. However, funding alone is not enough to achieve this quickly—TSMC's efforts to build a factory in Arizona have been mired in missed deadlines and labor disputes, and Intel has also failed to meet its promised deadlines. It is currently unclear whether the equipment and workforce of these factories, whenever they come online, will be able to reach the advanced chip manufacturing standards of these companies abroad.

Miller said, "The supply chain will only change slowly over years or even decades. But things are changing."

More Edge Artificial Intelligence

Currently, most of our interactions with AI models like ChatGPT are completed through the cloud. There are some drawbacks to relying on the cloud: on one hand, it requires internet access, and it also means that some of your data will be shared with the model creators.

This is why there is a great deal of interest and investment in edge computing for artificial intelligence, where the processing of AI models occurs directly on your devices, such as laptops or smartphones. As the industry becomes more committed to AI models that help us understand the future, there is a need for faster "edge" chips that can run models without sharing private data. These chips face different constraints from data center chips: they typically must be smaller, cheaper, and more energy-efficient.

The U.S. Department of Defense is funding a large amount of research for fast, private edge computing. In March of this year, its research division, the Defense Advanced Research Projects Agency (DARPA), announced a collaboration with chip manufacturer EnCharge AI to create an extremely powerful edge computing chip for AI inference. EnCharge AI is working on creating a chip that enhances privacy but operates with very low power consumption. This will make it suitable for military applications such as satellites and off-grid surveillance devices. The company is expected to ship these chips in 2025.

AI models will always rely on the cloud for certain applications, but new investments and interest in improving edge computing can bring faster chips to our everyday devices, leading to more artificial intelligence. If edge chips become small and cheap enough, we may see more AI-driven "smart devices" in homes and workplaces. Currently, AI models are mainly limited to data centers.EnCharge AI co-founder Naveen Verma said: "Many of the challenges we encounter in data centers will be overcome. I anticipate a significant focus on the edge domain. I believe this is crucial for the large-scale implementation of artificial intelligence."

 

Large technology companies enter the chip manufacturing field

Companies are incurring high computing costs to create and train artificial intelligence models for their businesses. For example, models that employees can use to scan and summarize documents, as well as external-facing technologies such as virtual agents. This means that the demand for cloud computing to train these models is increasing sharply.

 

The companies that provide most of the computing power are Amazon, Microsoft, and Google. For many years, these tech giants have dreamed of improving their profit margins by manufacturing chips for their own data centers, rather than purchasing chips from companies like Nvidia. Amazon began its efforts in 2015 by acquiring the startup Annapurna Labs. Google launched its own chip TPU in 2018. Microsoft launched its first artificial intelligence chip in November, and Meta launched a new version of its own artificial intelligence training chip in April.

 

This trend could put Nvidia at a disadvantage. However, in the eyes of large technology companies, Nvidia is not only a competitor: regardless of the internal efforts of the cloud giants themselves, their data centers still need their chips. Part of the reason is that their own chip manufacturing work cannot meet all their needs, but also because their customers hope to be able to use top-notch Nvidia chips.

 

"This is actually to provide customers with choices," said Rani Borkar, head of hardware work at Microsoft Azure. She said she could not imagine a future where Microsoft provides all the chips for its cloud services: "We will continue to maintain a strong partnership and deploy chips from all the chip partners we work with."

 

As cloud computing giants try to take some market share from chip manufacturers, Nvidia is also trying to do the opposite. Last year, the company launched its own cloud service, allowing customers to bypass Amazon, Google, or Microsoft. As the market share battle unfolds, the coming year will be about whether customers will see the chips of large technology companies as similar to the most advanced chips of Nvidia.

 

Nvidia's competition with startupsDespite Nvidia's dominant position, there is still a wave of investment flowing towards startups that aim to surpass Nvidia in certain areas of the future chip market. These startups all promise faster artificial intelligence (AI) training, but they have different ideas about which computing technology can achieve this goal, ranging from quantum computing to photonics to reversible computing.

Many companies, such as SambaNova, Cerebras, and Graphcore, are trying to change the underlying architecture of the chips. Imagine an AI accelerator chip that needs to constantly move data back and forth between different areas: information is stored in a memory area but must be moved to a processing area where it is calculated and then stored back in the memory area for safekeeping. All of this requires time and effort.

Improving the efficiency of this process will provide customers with faster and cheaper AI training, provided that chip manufacturers have good enough software to allow AI training companies to seamlessly transition to new chips. If the software transition is too clumsy, model manufacturers such as OpenAI, Anthropic, and Mistral may stick to large chip manufacturers. This means that companies adopting this approach (such as SambaNova) will spend a lot of time not only on chip design but also on software design.

Eva, a chip startup, founder Onen proposed a deeper change. Instead of using traditional transistors, which have provided higher efficiency by becoming smaller over the past few decades, he used a new component called a proton-gated transistor, which he said was specifically designed for the needs of AI training. It allows the device to store and process data in the same location, saving time and computing energy. The idea of using such components for AI inference can be traced back to the 1960s, but researchers could never figure out how to use it for AI training, partly due to material barriers - it requires a material that can precisely control the electrical conductivity at room temperature, among other qualities.

One day, in the laboratory, "by optimizing these numbers, and very fortunate, we got the material we wanted," said Onen. After months of effort to confirm the data was correct, he founded Eva, and the result was published in the journal "Science."

However, in this field, many founders have promised to overthrow the dominant position of leading chip manufacturers but have failed. Onen frankly said that it will still take several years to know whether the design works as expected and whether manufacturers will agree to produce it.

Leave a comment

Submit