The November 2021 Top500 listing was very fascinating. At STH we’ve got been a subset of each listing for years now. Particularly, we all the time take a look at the net-new methods since that exhibits rather a lot when it comes to business developments. Many methods will add nodes over time, however we usually like to take a look at the net-new. On this listing, Lenovo dominated. That dominance really brings out some developments that might be ignored in any other case.
For these desirous to make a journey down into the archives, you will discover our earlier items right here: June 2021, November 2020, June 2020, November 2019, June 2019. We’re going to reference the June 2021 fairly a bit on this piece so that could be one to open in one other tab/ window.
Top500 New System CPU Structure Traits
On this part, we merely take a look at CPU structure developments by what new methods enter the Top500 and the CPUs that they use. Allow us to begin by trying on the vendor breakdown.
In contrast to the previous two editions, the November 2021 listing was all AMD and Intel. From this view, it seems to be like Intel led handily, however there may be extra to it than that. In case you recall from the June 2021 listing, this was our video and one might take a look at the above and exclaim that the tables have turned.
After we get into extra of the main points, issues get extra fascinating. Intel Ice Lake launched in roughly the identical timeframe as AMD’s Milan. What we see although is that Intel Ice Lake solely represents 3 of 42 of Intel’s new methods, for a paltry 7%. Supercomputers do are inclined to lag, however AMD noticed a 50% transition to its new chips. Here’s a breakdown by structure:
One thing that can be fascinating is simply that we aren’t seeing any pre-2019 architectures as we’ve got with earlier lists. That is solely the 2 latest architectures from Intel and AMD. Whereas Arm holds the highest spot with Supercomputer Fugaku by Fujitsu and RIKEN and is powering Chinese language supercomputers, it’s not getting new methods onto the listing. That can change as Europe builds Arm supercomputer chips from non-EU Arm IP (UK/ Japan until NVIDIA purchases it, and from what we heard at present slated for TSMC fabs. Nonetheless, a part of efforts just like the Euroexa venture ought to embrace some EU IP.)
Again to the asterisk on the Intel is thrashing AMD. Right here is one other method to take a look at the information, by cores as an alternative of simply by the variety of methods that use every kind of CPU. As we will see, this doesn’t appear to be Intel’s dominance.
One different very fascinating word right here. Of the 42 Intel-based methods, 41 are made by Lenovo and one by Dell. Additionally, solely 10% of the accelerated methods (e.g. with a GPU) make the most of Intel Xeon CPUs.
CPU Cores Per Socket
Right here is an intriguing chart, trying on the new methods and the variety of cores they’ve per socket.
24 cores stays an necessary level. What is de facto fascinating right here is how this has modified. With Ice Lake being much less represented than the earlier listing, we now have the 48 and 64 core AMD methods. 32 cores is blended between the distributors and 24 cores has a single EPYC 7413 system. In any other case, it’s usually Intel on the left and AMD on the best. Only for comparability, we additionally see decrease core rely methods are gone. Right here is the June 2021 listing the place quite a few new methods used <24 cores:
So there definitely has been a whole lot of motion principally utilizing what can be a contemporary low core rely CPU 24-32 cores, or simply going to 64 cores.
Listed here are the precise SKUs used:
The AMD EPYC 7V12 is Microsoft Azure’s customized SKU. The Azure staff is constructing a devoted HPC infrastructure within the cloud and shall be deploying Milan-X quickly.
Maybe the surprising bit is that 14 of the methods use the Intel Xeon Platinum 8280, however none are utilizing the brand new Platinum 8380. It appears a bit unusual that these Platinum 8280’s are on the listing simply given they now really feel dated in a world with the EPYC 7763.
Accelerators or Simply NVIDIA?
In contrast to the June 2021 listing, NVIDIA will not be the one accelerator vendor for the brand new methods. Here’s a breakdown in our accelerator by vendor chart:
Here’s a breakdown of the brand new accelerated methods by accelerator:
First, off, NVIDIA has transitioned to the A100, as we aren’t seeing V100 methods anymore. Fairly a number of of those methods are literally the Dell EMC PowerEdge XE8545 we reviewed. Certainly three of Dell EMC’s 4 entries used this technique:
The larger story is maybe the decline of NVIDIA right here. In June 2021 22 of the 58 new methods used NVIDIA accelerators (down from 28 of 58 in June 2020’s 58 new methods.) Now NVIDIA is nineteen/70. Placing that in perspective, roughly six quarters in the past, ~48% of recent methods on the Top500 used NVIDIA accelerators. Now, that’s all the way down to 27%.
Acceleration remains to be a NVIDIA sport, however with Exascale methods coming quickly, and we learn about AMD with Frontier and El Capitan 2 together with Intel Xe HPC GPUs for that period, we may even see a change over the following few lists as high-end methods get extra various with accelerators.
Above we’ve got what Intel is doing on the HPC facet and we mentioned the AMD Intuition MI200 and noticed the AMD Intuition MI250X OAM at SC21.
The unusual factor is that whereas there are some pretty robust indicators the HPC market has really been pulling again from NVIDIA, Intel and AMD wouldn’t have their GPUs on this listing.
Cloth and Networking Traits
Right here is one which many regulars to this piece will establish with. On the Interconnect facet, Ethernet is by far the commonest answer.
In June 2020 we noticed Ethernet at 53% of the brand new methods. In June 2021 it was solely 19% of recent methods utilizing Ethernet. In November 2021 we are actually again to 54%.
Whereas Omni-Path had some uptake on the November 2020 listing, we once more wouldn’t have any Intel OPA or Cornelis Networks Omni-Path methods right here.
After we take a look at a breakdown by era, here’s what we get:
As a fast word, there have been 4 Russian methods that merely have been marked with “Infiniband” so we’re calling these RU Infiniband. Our greatest guess can be HDR however we simply wished to busket these individually within the occasion that assumption is wrong. Additionally, on the Atos BXI V2, it’s definitely nicely behind the HPE-Cray Slingshot when it comes to adoption.
If we drill into which producers are utilizing 10GbE, 25GbE, 40GbE, and 100GbE we get an fascinating image:
Lenovo tends to be the highest Ethernet person for brand spanking new methods on the listing. That is no totally different besides for 2 components. First, no different vendor is utilizing Ethernet. Second, we lastly have a listing with no 10GbE/ 40GbE. There are nonetheless 38 Ethernet methods however solely 8 are 25GbE.
Lenovo tends to take clusters of non-traditional HPC methods, runs Linpack on them, after which submits them to the Top500 in an effort to declare the most important vendor numbers. All 38 of those Ethernet methods simply occur to even be Intel Xeon Cascade Lake methods.
After we take a look at the seller image, we will get a way of what’s occurring available in the market:
Lenovo is #1 once more, however once more, 38 of the 42 methods it has on this listing are Ethernet-based, Intel Xeon Cascade Lake, non-accelerated methods put in at software program distributors or service suppliers. Lenovo has so many of those methods that they make up 54% of the brand new methods on the Top500 listing. After we focus on Intel, AMD, Arm, or NVIDIA successful within the new Top500 methods, Lenovo’s level is maybe none of them win. As a substitute, something utilizing expertise extra unique than a 2.5-year-old+ CPU and Ethernet is technically a minority system.
Although we all know the listing will not be an ideal illustration of what’s going on, it’s nonetheless enjoyable to do a bit of research across the adjustments occurring within the business. Hopefully, our readers loved this one. Allow us to additionally hope that ISC 2022 brings some large adjustments.