Economic Observer reporter Zheng Chenye

Since the beginning of October, stocks in the global AI industry chain have undergone a round of significant adjustments.

As of October 16, the Philadelphia Semiconductor Index in the United States has been declining for 8 consecutive trading days, setting a new record for the longest streak of declines since 2008. On October 22, chip and AI stocks in the Asian market led the decline again; the South Korean KOSPI index closed down 3.52%, SK Hynix fell by more than 5%, and the A-share GEM index dropped 4.37%, with computing power hardware stocks collectively weakening.

According to the guidelines provided by Amazon, Microsoft, Google's parent companies Alphabet and Meta in late July, all four companies' capital expenditures this year will exceed $100 billion, with a major portion being invested in AI infrastructure such as data centers. Whether these investments can be recouped and what the risks of debt expansion are are currently at the center of debate in the global capital markets.

“It will all pass, and the reason is simple: AI will bring about change, affecting every industry and every country in a very positive way.” On October 16th, Jensen Huang, the founder and CEO of NVIDIA who was on a tour of Europe at the time, responded to market fluctuations in an interview with the media in Berlin. In his view, the changes brought about by AI are still at their beginning.

On October 21st, Jensen Huang delivered a keynote speech at the GTC (the NVIDIA GPU Technology Conference) in Berlin, using the logic of manufacturing to discuss the inputs and outputs of AI factories (data centers that train and operate AI models). "We realized quite early on that AI is essentially a form of manufacturing," he said. The way computers function has fundamentally changed; they have evolved from tools used by people to intelligent factories that produce knowledge.

In Jensen Huang's description, the AI factory uses electricity as its raw material to produce Token (which are the basic units for AI models to process and generate information), for use in various industries. He estimates that a 1-gigawatt AI factory would cost about 50 billion US dollars, including land, electricity, factory buildings, as well as computing, networking, storage, cooling equipment, and system integration. It is expected to generate several hundred billion dollars in revenue each year.

According to the manufacturing logic explained by Jensen Huang, whether the investment in the AI factory can be recouped depends, first, on whether the factory can be built quickly and with low power consumption, and second, on whether there will be enough buyers for the Token produced after it is built.

AI Factory Efficiency

When introducing the design scheme for the AI factory, NVIDIA summarized its goal as "producing more intelligence per watt of power, and bringing the first Token to market in less time." In manufacturing terms, the former refers to the output per unit of electricity, while the latter refers to how quickly a new factory can start producing. Both are related to how long it will take for the AI factory to recoup its investment.

How much Token a AI factory can produce depends to a large extent on the performance of the computing devices, and in the past, the improvement in performance mainly relied on advancements in chip manufacturing processes.

Jensen Huang stated in his speech that Moore's Law has essentially come to an end, and even if it remains valid, chip performance can only increase by about 100 times over a decade. He mentioned that in the past 11 years, AI's performance has increased by one million times, thanks to the "extreme collaborative design" of chips, systems, software, and algorithms.

When power supply is limited, how much Token can be converted from each kilowatt-hour of electricity determines the output of the AI factory. Information published by NVIDIA shows that, according to its design scheme, under the same output conditions, the power consumption of the AI factory can be reduced by up to 30%; after testing NVIDIA's power management software on the previous generation of GB300 systems, the British AI cloud service provider Nscale was able to deploy 37% more GPU while keeping the power supply unchanged, and the total amount of Token produced per unit of time increased by nearly 50%.

Jensen Huang gave an example: assuming a 1-gigawatt AI factory generates $40 billion in revenue per year, being delayed by 3 months in production would result in a loss of about $10 billion. To shorten the production time, NVIDIA released a reference design for the entire AI factory at the conference. According to this design, partners first install the equipment in containerized modules, complete the integration and verification, and then transport them to the construction site. With the site facilities ready, the on-site deployment time can be reduced from 6 months to 14 days.

Jensen Huang believes that integrating assembly work into the supply chain represents a complete transformation of the global supply chain, which will change the way data centers are installed and constructed. There will be less idle and wasted computing power and capital investment, and the profitability of all parties in the industrial chain will increase. He also mentioned that NVIDIA was able to supply hundreds of data centers around the world simultaneously by incorporating the assembly of entire cabinets into its supply chain, stating, “We are now producing computing equipment on an unprecedented scale.”

Jensen Huang hopes that Europe will also build its own AI factories. "To become part of the AI economy, one cannot just be a consumer of AI; instead, one must become a producer of AI," he said. "We are fully capable of achieving all this in Europe, and in fact, we have already begun."

Token Use Cases

On the demand side, Jensen Huang believes that infrastructure must be in place before the AI model can be implemented, just as communication networks were necessary before the advent of the internet. In his view, the main consumers of Token in the future will be intelligent agents. These agents will think, reason, iterate repeatedly, and use tools. The more useful they are, the more Token will be consumed, and the demand for GPU will also increase significantly as a result.

For agents to truly be implemented in enterprises, they must also be granted access to files, networks, and various tools. "The usefulness of an agent depends on the level of access permissions you grant it," says Jensen Huang. Of course, the top priority is to ensure the security of the agents; their permissions should be minimized as much as possible and monitored closely. NVIDIA presented a OpenShell security reference architecture at the conference, which runs agents in isolated environments and limits what they can access and what they can do at the chip level, making it easier to deploy agents securely for customers.

According to Jensen Huang's judgment, in the future, every company will use both closed models and customized open models simultaneously, and buyers of Token will come from all walks of life. He also stated that NVIDIA will make the parameters of the open models, the full set of training methods, and the data available publicly, allowing companies to reproduce and customize these models based on that information.

Jensen Huang also mentioned that some people believe that the advancement of AI will lead to the end of software and tool companies, but his view is quite the opposite. He believes that more users of these tools than ever before will emerge, and every company that develops professional software tools will transform into a AI company, that is, an entity-based company. In the case demonstrated by NVIDIA, Dassault Systèmes, a French company specializing in design and simulation software, has already been able to identify issues in vehicle designs through simulation and then improve them accordingly. Of course, such simulations themselves require a substantial amount of computing power.

Farer and greater demands come from physical AI (the technology that enables cars and robots to perceive and act in the real world). According to Jensen Huang, building a physical AI system requires three types of computers: one for training models, another for enabling machines to acquire skills within digital twins, and a third for running on the robots themselves. The first two types of computers are typically located in data centers, which means that physical AI will consume the computing power of data centers even before it is widely implemented on a large scale.

In Jensen Huang's view, driverless taxis represent a huge opportunity, but the market is extremely fragmented. At the aforementioned conference, he announced that the next-generation models of car manufacturers such as Volvo and Mercedes-Benz will be compatible with NVIDIA's standardized autonomous driving architecture Hyperion. According to his vision, with this industry standard and by bringing together companies from the entire ecosystem, a massive "virtual fleet" that anyone can join could be formed.

As for humanoid robots, Jensen Huang made an analogy by comparing them to mobile phones, personal computers, and NVIDIA's own NVL72 rack systems. He stated that their rapid progress is due to standardized configurations that make them accessible to many customers, which in turn leads to increased sales and accelerates technological iteration even further. Based on this, he believes that the form of robots should also be standardized as much as possible. Since humanoid robots can be applied in many scenarios, they are likely to become the most widely sold type of robot.

When the capital market presses for investment returns on AI, NVIDIA presented reference factory designs and various standard architectures at the conference. The true intention behind this may be to maximize the market scale by relying on standardized configurations.