NVIDIA launches its first-ever CPU: But it won't be coming to a laptop near you soon...
First Arm-based CPU from NVIDIA will offer "10x" x86 performance.
NVIDIA has announced its first data centre CPU, with the company revealing a new Arm-based processor dubbed "Grace" that it will start shipping in early 2023 specifically for "giant-scale" AI and High-Performance Computing.
The launch -- revealed in a keynote by founder Jensen Huang at the GPU Technology Conference (GTC) -- makes NVIDIA "now a three-chip company" Huang said, referring to the CPU's addition to his company's existing heritage of graphics processing units (GPUs) and programmable data processing units (DPUs.)
The release comes as NVIDIA's $40 billion proposed buyout of Arm -- a UK-based but Japan-owned semiconductor company with an open licensing model -- faces regulatory headwinds, a protest from rivals, and concern in China, where policymakers fear US ownership of Arm could restrict its access in future to the latest designs.
NVIDIA launches its first CPU: "Niche" release targets giant NLP models.
The CPU's introduction comes as the volume of data and size of AI models surge. As NVIDIA noted: "Today’s largest AI models include billions of parameters and are doubling every two-and-a-half months.
"Training them requires a new CPU that can be tightly coupled with a GPU to eliminate system bottlenecks."
The company suggested it will be used in the kind of systems designed to train natural language processing (NLP) models that have more than 1 trillion parameters. It has built corresponding kit that will allow 900 GB/s of bidirectional bandwidth between the Grace CPU and its GPUs using its wired NVlink for a "unified, cache-coherent memory address space that combines system and HBM GPU memory for simplified programmability."
"When tightly coupled with NVIDIA GPUs, a Grace CPU-based system will deliver 10x faster performance than today’s state-of-the-art NVIDIA DGX-based systems, which run on x86 CPUs", the company said in a release, April 12, as it announced a flurry of new products. If the figures are accurate, the company has clearly done something genuinely special with Arm Neoverse cores. (The release comes after Arm unveiled its own latest architecture: it's first in a decade.)
The Swiss National Supercomputing Centre (CSCS) and the U.S. Department of Energy’s Los Alamos National Laboratory will be early adopters; both announcing plans to build Grace-powered supercomputers.
“With an innovative balance of memory bandwidth and capacity, this next-generation system will shape our institution’s computing strategy,” said Thom Mason, director of the Los Alamos National Laboratory, saying in a canned statement that the CPU will help researchers "deliver advanced scientific research using high-fidelity 3D simulations and analytics with datasets that are larger than previously possible.”
The launch came as NVIDIA also announced a series of collaborations that combine its GPUs and their specialised software with Arm-based CPUs. These include an eye-catching partnership with AWS, which will offer new Amazon EC2 instances that combine its Arm-based Graviton2 processors with NVIDIA GPUs.
The instances will "enable game developers to run Android games natively on AWS, accelerate rendering and encoding with NVIDIA GPUs, and stream games to mobile devices without the need to run emulation software" NVIDIA said.
AWS added: "In addition to mobile gaming, customers running machine learning models in production are continuously looking for ways to lower costs as ML inference can represent up to 90% of the overall infrastructure spend for running these applications at scale. With this new offering, customers will be able to take advantage of the price/performance benefits of Graviton2 to deploy GPU accelerated deep learning models at a significantly lower cost vs. x86-based instances with GPU acceleration."
“As the world’s most widely licensed processor architecture, Arm drives innovation in incredible new ways every day,” said Arm CEO Simon Segars. “NVIDIA’s introduction of the Grace data center CPU illustrates clearly how Arm’s licensing model enables an important invention, one that will further support the incredible work of AI researchers and scientists everywhere.”
The chip has been named after computing pioneer Grace Hopper.
