Thanks to the boom in Artificial Intelligence, the world of chips is on the verge of enormous change. There is a growing demand for chips that can train AI models faster and connect them from devices like smartphones and satellites, allowing us to use these models without revealing private data. Governments, technology giants and startups are competing to secure their shares of the growing semiconductor market.
Here are four trends that will define what the chips of the future will look like, who will make them, and what new technologies they will unlock.
CHIPS laws around the world
Just outside Phoenix, two of the world’s biggest chipmakers, TSMC and Intel, are racing to build campuses in the desert. The expectation is that these hubs will become centers of American power in chip manufacturing. One thing the efforts have in common is their funding: In March, President Joe Biden announced $8.5 billion in direct federal funds and $11 billion in loans for Intel expansions across the country. Weeks later, another $6.6 billion was announced for TSMC.
The subsidies are just one part of U.S. contributions to the chip industry through the $280 billion CHIPS and Science Act, signed in 2022. The money means that any company involved in the semiconductor ecosystem is looking at how to restructure its supply chains. to benefit from this amount. While much of the money is aimed at boosting American chip manufacturing, there is room for other players to apply, from equipment manufacturers to niche materials startups.
But the U.S. isn’t the only country trying to insource part of the chipmaking supply chain. Japan is spending $13 billion on its own equivalent of the CHIPs Act, Europe will spend more than $47 billion, and earlier this year India announced a $15 billion effort to build local chip factories. The roots of this trend go back to 2014, says Chris Miller, a professor at Tufts University and author of the book Chip War: The Fight for the World’s Most Critical Technology, when China began to offer huge subsidies to its chipmakers.
“This created a dynamic where other governments concluded they had no choice but to offer incentives or see companies move manufacturing to China,” he says. This threat, along with the rise of AI, has led Western governments to fund alternatives. Next year, this could have a snowball effect, with even more countries starting their own programs for fear of being left behind.
According to Miller, the money is unlikely to lead to new chip competitors or fundamentally restructure who the biggest players in that market are. Instead, it will encourage, particularly dominant players like TSMC, to establish roots in multiple countries. But funding alone won’t be enough to do this quickly — TSMC’s effort to build factories in Arizona has been hampered by missed deadlines and labor disputes, and Intel has also failed to meet its promised deadlines. Furthermore, it is unclear whether, when the factories finally come online, their equipment and labor will be able to maintain the level of advanced chip manufacturing that companies maintain overseas.
“The supply chain will only change slowly, over years and decades,” says Miller. “But it’s changing.”
More AI at the limit
Currently, most of our interactions with AI models like ChatGPT are done via the cloud. This means that when you ask GPT to choose an outfit (or to be your boyfriend), your request is sent to OpenAI’s servers, which processes it and draws conclusions (known as “inference”) before a response is sent back. for you. Relying on the cloud has some disadvantages: it requires internet access, for example, and means that some data is shared with the model creator.
This is why there is a lot of interest and investment in edge computing for AI, where the process of interacting with the model happens directly on your device, such as a laptop or smartphone. With the industry increasingly working toward a future in which AI models know a lot about us (Sam Altman described his ideal AI model as one that knows “absolutely everything about my entire life, every email, every conversation I’ve had”), there is a demand for faster edge chips that can run models without sharing private data. These chips face different constraints compared to data centers: They typically need to be smaller, cheaper, and more power efficient.
The US Department of Defense is funding a lot of research into fast, private edge computing. In March, the Defense Advanced Research Projects Agency (DARPA) research wing announced a partnership with chipmaker EnCharge AI to create an ultra-powerful edge computing chip used for AI inference. EnCharge AI is working to make a chip that allows for greater privacy but can also operate on low power. This will make it suitable for military applications such as satellites and off-grid surveillance equipment. The company expects to ship the chips in 2025.
AI models will always rely on the cloud for some applications, but new investment and interest in improving edge computing could bring faster chips and therefore more AI to our everyday devices. If edge chips get small and cheap enough, we’re likely to see even more AI-driven “smart devices” in our homes and workplaces. Today, AI models are mostly restricted to data centers.
“Many of the challenges we see in data centers will be overcome,” says EnCharge AI co-founder Naveen Verma. “I expect to see a big focus on edge [computing]. I think it will be key to getting AI at scale.”
Big Techs enter the fray for chip manufacturing
In industries ranging from fast fashion to lawn care, companies are paying exorbitant amounts in computational costs to create and train AI models for their businesses. Examples include templates that employees can use to scan and summarize documents, as well as outward-facing technologies like virtual agents that can guide you on how to fix your broken refrigerator. This means that the demand for cloud computing to train these models is through the roof.
The companies providing most of this computing power are: Amazon, Microsoft and Google. For years, these tech giants have dreamed of increasing their profit margins by manufacturing chips in-house for their data centers rather than buying from companies like Nvidia, a giant with a near-monopoly on the most advanced AI training chips and a higher value. than the GDP of 183 countries. Amazon began its efforts in 2015 by acquiring startup Annapurna Labs. Google next moved forward, in 2018, with its own chips called TPUs. Microsoft released its first AI chips in November, and Meta unveiled a new version of its own AI training chips in April. This trend could tip the scales against Nvidia. But the company doesn’t just play the role of rival in the eyes of Big Tech: regardless of their own internal efforts, cloud giants still need Nvidia chips for their data centers. That’s partly because its own chipmaking efforts can’t meet all of its needs, but also because its customers expect to be able to use Nvidia’s top-of-the-line chips.
“It’s really about giving customers power of choice,” says Rani Borkar, who leads hardware efforts at Microsoft Azure. She says she can’t imagine a future where Microsoft supplies all the chips for its cloud services: “We will continue our strong partnerships and deploy chips from all the partners we work with.”
While cloud computing giants try to steal some market share from chipmakers, Nvidia is also trying the opposite. Last year, the company started its own cloud service so customers can bypass Amazon, Google or Microsoft and get processing time with Nvidia chips directly. As this dramatic fight for market share plays out, the next year will be about whether customers view Big Techs’ chips as similar to Nvidia’s more advanced chips, or more like their smaller cousins.
Nvidia fights startups
Despite Nvidia’s dominance, there is a wave of investment flowing into startups that aim to overtake it in certain slices of the future chip market. These startups all promise faster AI training, but they have different ideas about which computing technology will get them there, from quantum to photonics to reversible computing.
But Murat Onen, founder of one such chip startup, Eva, which he created from his doctoral work at MIT, is blunt about what it’s like to start a chip company now.
“The king of the hill is Nvidia, and that’s the world we live in,” he says.
Many of these companies, such as SambaNova, Cerebras and Graphcore, are trying to change the underlying architecture of chips. Imagine an AI accelerator chip constantly having to shuffle data between different areas: information is stored in the memory zone, but must move to the processing zone, where a calculation is made, and then be stored back to the processing zone. memory for security. All of this takes time and energy.
Making this process more efficient would provide faster and cheaper AI training to customers, but only if the chipmaker has good enough software to allow the AI training company to seamlessly transition to the new chip. If the software transition is too complicated, model makers like OpenAI, Anthropic, and Mistral will likely remain major chipmakers. This means that companies taking this approach, like SambaNova, are spending a lot of time not just on chip design, but also on software design.
Onen is proposing changes on a deeper level. Instead of traditional transistors, which have delivered greater efficiency over decades of getting smaller and smaller, he is using a new component called a proton transistor that he says Eva designed specifically for the mathematical needs of AI training. It allows devices to store and process data in the same place, saving time and computational energy. The idea of using this component for AI inference dates back to the 1960s, but researchers have never been able to figure out how to use it for AI training, due to a material obstacle – it requires a material that can, among other qualities, control accurately determine conductivity at room temperature.
One day in the lab, “by optimizing these numbers and getting really lucky, we got the material we wanted,” says Onen. “Suddenly the device is no longer a science fair project.” This raised the possibility of using this component at scale. After months of working to confirm that the data was correct, he founded Eva, and the work was published in Science.
But in an industry where so many founders have promised to upend the dominance of major chipmakers — and failed — Onen freely admits it will be years before he knows whether the design works as intended and whether manufacturers will agree to produce it. Leading a company through this uncertainty, he says, requires flexibility and an appetite for skepticism from others.
“I think sometimes people feel very attached to their ideas, and then they kind of feel insecure that if that goes away, there won’t be anything to follow,” he says. “I don’t think I feel that way. I’m still looking for people to challenge us and tell us we’re wrong.”
( fonte: James O’Donnell/MIT Technology Review)
