ARM has just delivered its roadmap for its second-generation Neoverse designs, which must integrate the next processors for servers. The chips concerned are future versions of Altra from Ampere, Graviton from AWS, Rhea from SiPearl and probably Grace from Nvidia. The Neoverse V2 design plans have already been communicated to the manufacturers and those of the Neoverse N2 will be communicated in the course of 2023.
ARM V2s are designed to maximize computing power per core and ARM N2s to maximize the number of individually power-efficient cores. Compared to the N1 cores found in current ARM server processors, the N2 and V2 will support PCIe 5.0 and DDR5 memory buses, which offer twice the bandwidth of current PCIe 4.0 and DDR4. But this is far from being their only interest.
“These supercomputing chips will have more complex processing units than ever before. But, most importantly, given that AI and Machine Learning algorithms take precedence over all other expectations from server processor manufacturers, then our circuits will be nothing like anything you’ve experienced before.” said, during a press briefing, Chris Bergey, the head of infrastructure at ARM.
Processors designed for AI and cloud-native applications
In essence, the next ARM Neoverse chips will implement in their circuits – and in the CMN-700 interconnect of their circuits – everything needed to run machine learning engines like TensorFlow as quickly as possible.
In addition, Chris Bergey announces that ARM will spend most of 2023 with cloud hosts and software publishers to optimize their functional layers according to new designs. This would include better writing the systems related to the container infrastructure, in particular the Kubernetes orchestrator and the Istio network. And for good reason: they form the basis of so-called “cloud native” applications and it is in the cloud domain that the majority of ARM servers should be sold.
“Currently, cloud providers, but also compute providers, are moving more and more towards machines where all the power is based on GPUs. And they want these to be driven by processors with the best speed/power saving ratio. This is why these designs almost only include ARM processors. We therefore worked with people from Red Hat, on OpenShift, from VMware, on Tanzu, but also with those from SAP, on Hana, so that their software would be in line with such servers”, assures Chris Bergey.
V3 and N3 versions should also be unveiled during 2023, for implementation at least in 2024. In 2023, ARM should also announce a new E3 design which consists of maximizing data throughput. This design should succeed the Cortex-A510 cores, themselves successors to the Cortex-A55 found in mobile devices.
According to ARM, it is not a question of building servers with one version or another of Neoverse processors, but rather of combining the three in each server. Each type of core would support a specific type of algorithm. This is good for Machine Learning, which needs intensive computing, parallelization and high throughput depending on the data levels. And it would also be good for “cloud-native” applications that segment themselves into microservices of different types.
Are ARM processors good enough?
According to analysts, the success of ARM architectures among cloud hosts is well established. On the other hand, what these servers would lack today would be performance.
“AWS, Azure and even GCP have been seduced by the opportunity offered by ARM processors to build infrastructure at a lower cost and with low energy consumption. Businesses can now go home and run their applications on ARM VMs that are cheaper than x86 VMs. The problem is that it’s really only for pushing web pages. The heaviest applications remain on Intel or AMD VMs,” observes Jack Gold, an analyst at the consulting firm J. Gold Associates.
Catching up with this performance lag, without sacrificing energy savings, is precisely the challenge of the new Neoverse V2 architecture. An important point of the new design is the use of LPDDR5X memories, those of laptops, potentially on the same chip as the processor cores. This memory alone would make it possible to benefit from twice as much bandwidth per Watt as the DDR5 memories expected on the next generations of Intel and AMD servers.
Are ARM processors a compatibility issue?
Another challenge to overcome: compatibility. Applications compiled by their developers for traditional servers are simply not usable on ARM servers. Their developers must recompile them and, above all, test how they behave when they are executed by an instruction set that has nothing to do with it. Without this, ARM servers will still only be used to run Python, Perl and other web site scripts.
“You don’t have to press a button to make your code produce an ARM binary rather than x86. Changing the instruction set changes how your application reacts. And it’s ultimately a port that’s not that easy to do,” says Jack Gold.
According to analyst Dan Newman, from Futurum Research, the direction of developments towards a cloud-native format (including fragmented applications in containers) should nevertheless reduce development efforts. Especially since it’s more about creating new applications than carrying those that already exist.
“We are talking here about cloud-specific infrastructures for hyperscalers and the services they develop, particularly in SaaS. It’s in no way a replacement for the servers that are running in corporate data centers,” he notes.
That said, he notes that Apple has perfectly mastered the transition of its applications by replacing the x86 processors of its Macs with its own M1 chips, based on ARM architecture.
“Apple had raised concerns about the compatibility of its chip. But through continuous optimization efforts, it managed to make more apps work with it. Will the collaboration efforts with publishers mentioned by ARM be as fruitful? It is likely that they do not give such quick results. Because ARM does not have as much control over the publishers of the market as Apple has over the way of designing applications for its system, ”said Dan Newman.