SPEC Releases CPU2017 Benchmarking Suite, Introduces 4 New Test Suites And OpenMP Support

In 2011, I wrote that we were adding SPEC’s CPU2006 to our processor testing suite, and I admit that at that time, I had assumed that the next major iteration of the benchmark would be right around the corner. Welp, I couldn’t have been more wrong, as its successor has just shown up this week, in the form of CPU2017.

To be fair to the suite, its name is the only thing that was truly outdated. The nature of the suite lets users test with their own choice of compilers – compilers which could have been released long after the suite’s last update. Still, 11 years is a long time, so it’s nice to have a more up-to-date, modernized CPU suite – especially since this is one of SPEC’s most important benchmarks.

For the uninitiated, SPEC’s de facto CPU testing suite benchmarks the performance of the processor, memory, as well as the compiler. The compiler chosen to benchmark with doesn’t matter as long as it can compile the tests included in this suite; that includes C, C++, and Fortran, which can be benchmarked on either Linux, Unix, Mac OS X, or Windows. Ultimately, CPU2017 can test both the compilation speed as well as execution speed of those compiled tests.

SPEC CPU2017 Installation in Linux

CPU2017’s tests are split into four different suites, with both integer and floating-point tests available in both SPECspeed, and SPECrate; SPECspeed gives a score based on the amount of time a test takes, while SPECrate gives a result representing the amount of work a system can do in a given unit of time.

One of the biggest new features of CPU2017 is the implementation of OpenMP, to make better use of the available cores and threads available in a system. OpenMP is an option, and would mostly affect the floating-point SPECspeed floating-point suite (where all tests use OpenMP directives), vs the integer counterpart, where only one test (657.xz_s) would be affected.

SPEC’s CPU subcommittee currently features the biggest names in computing, including AMD, Intel, Dell, HPE, NVIDIA, and Oracle. First-time buys of the suite will be able to purchase CPU2017 for $1,000 USD, while those upgrading from CPU2006 can take advantage of a reduced $500 fee. Non-profits can purchase the suite for $250.

As we just finalized our latest CPU test suite a couple of months ago, I am not sure when we’ll be able to implement CPU2017, but I plan to get down to testing with it well in advance, since it’s not exactly an easy benchmark for a non-developer to dive into. When we do begin testing with it, it seems likely that it will become part of our overall Linux testing suite.

CPU2017 been a long time coming, so a big congratulations is in order for the SPEC CPU team!