A Look At Our Tests Ahead Of More CPU & Gaming GPU Reviews

With so many products launching in the near future, and even more that are rumored to launch, we figured we’d take a few minutes to give an update on what we’ve been up to, and what we have in store for the near-future. The past couple of weeks have been dominated by both travel and test suite updating. Our immediate plan at this point is to kick off benchmarking immediately, and start generating results for our reviews of AMD’s upcoming CPUs and GPUs.

At this point, we don’t have anything new from AMD in the lab, but it’s expected to arrive soon, and we don’t plan to waste time in getting down to business with testing. For our CPU testing, the suite has remained largely the same, but some tests have been polished to deliver better results:

Rendering: Adobe Dimension

Encoding: Adobe Premiere Pro

Photogrammetry: Agisoft Metashape

Rendering: Autodesk Arnold in Maya

Rendering: Blender

Rendering: Chaos Czech Corona Renderer in 3ds Max

Rendering: Chaos Group V-Ray Next in 3ds Max

Music Encoding: LameXP

Rendering: Luxion KeyShot

Encoding: MAGIX Vegas Pro

Rendering: MAXON Cinebench

Rendering: MAXON Cinema 4D

Rendering: POV-Ray

Rendering: V-Ray Next Benchmark

CPU Test Suite
• Adobe Dimension (Render)
• Adobe Premiere Pro (Encode)
• Agisoft Metashape (Photogrammetry)
• Autodesk Arnold (Render)
• Blender (Render)
• Chaos Czech Corona Renderer (Render)
• Chaos Group V-Ray (Render)
• Chaos Group V-Ray Benchmark (Render)
• HandBrake (Encode)
• LameXP (Encode)
• Luxion KeyShot (Render)
• MAGIX Vegas Pro (Encode)
• MAXON Cinebench (Render)
• MAXON Cinema 4D (Render)
• POV-Ray (Render)
• SiSoftware Sandra (Math)
• UL 3DMark (Gaming)
• Unigine Superposition (Gaming)

An omission that might not be immediately obvious is AMD’s Radeon ProRender. With the latest version of the plugin, we can’t get CPU-only rendering to work properly with our chosen projects. In both 3ds Max and Maya, a CPU render will use the CPU only for a single pass, at which point the actual rendering process stalls until the rest of the passes pretend to iterate. AMD has told us that this behavior is not typical, but we experience it on two different PCs, and with two different design suites. We’ll test again once the next plugin versions arrive (SIGGRAPH timeframe, we’d assume).

With Linux, we have the full intention to test CPUs with the same suite as before, though we plan to enhance it over time. We’re juggling so many test suites, it’s really challenging to make all of them continually “great”, but we’re getting there. To that end, we’re planning to add DaVinci Resolve to our suite as soon as we can. We acquired the full Studio version of the software last week, and are actively exploring not only the tests to conduct, but also how to advance beyond the “extreme noob” stage of understanding how to use it!

You might have noticed that the CPU suite is lacking in the gaming department. We’re looking to integrate some more real game tests ahead of the first set of CPU swaps, since it would be useful to include. We typically focus on workstation-type workloads a lot, but with Zen 2, there’s been so much attention given to gaming, we can’t realistically ignore it. Thankfully, we have an updated GPU test suite to work with, so we’ll likely integrate a few of the games from it into our CPU testing. We’ll likely use the TITAN RTX for the sake of reducing the GPU bottleneck as much as possible.

Battlefield V

Counter-Strike: Global Offensive

F1 2019

Far Cry 5

Metro Exodus

Monster Hunter: World

Shadow of the Tomb Raider

Tom Clancy's Rainbow Six Siege

Tom Clancy's The Division 2

Total War: Three Kingdoms

UL 3DMark

UL VRMark

Unigine Superposition

Gaming GPU Test Suite
• Battlefield V (DX12)
• Counter-Strike: Global Offensive (DX11)
• F1 2018 (DX12)
• Far Cry 5 (DX11)
• Metro Exodus (DX12)
• Monster Hunter World (DX11)
• Shadow of the Tomb Raider (DX12)
• Tom Clancy’s Rainbow Six Siege (DX11)
• Tom Clancy’s The Division 2 (DX12)
• Total War: Three Kingdoms (DX12)
• UL 3DMark (DX11 and DX12)
• UL VRMark (DX11 and DX12)
• Unigine Superposition (DX11)

Many games from the previous suite are making a return here, as they are still very much appropriate for punishing GPUs with. In all cases, we opt to use DX12 if there’s an option, which leaves the DX11 selection to those games which don’t support anything else.

There are a couple of things to draw attention to here. First is the fact that CS: GO and Rainbow Six Siege are not typical inclusions, since they run so fast, it almost feels ridiculous to test them. However, AMD’s press event discussing these products focused a lot on super-high framerates, since that’s what eSports gamers are eager to see. We might expand eSports performance later, but for now, two of the most popular games should help us get the job done.

There’s an unfortunate lack of a Vulkan game in the list. We had planned to use Rage 2, but in the middle of figuring out test selections, the game received a patch which broke the game on our NVIDIA install. The game is fine on the AMD install, so we’re baffled as to the problem. Even a fresh Windows install didn’t fix the issue, however. If the game somehow begins working before we get down to serious testing, it will be added.

Over the past year, we’ve received many suggestions to adopt 99% and 99.9% percentile results, and we’re pleased to say that we’ll be integrating such results into our upcoming testing. We won’t tackle them for every single game tested, but will with the titles that deserve a more expanded look. Because minimums are super-sensitive at certain resolutions, percentile tests will need to be conducted more than the normal number of times for an increased confidence level. Since this is all pretty time-consuming, we’ll likely focus on just 1080p and 4K frametimes.

While we’re pretty much sold on our current test suites, we always welcome feedback, and are always eager (a little too much) to expand. For gaming, we intend to test AMD’s Navi GPUs with 1080p and 1440p resolutions, but ultrawide is possible if time ends up allowing it. We’re planning to publish a lot of content around both Zen 2 and Navi, so that big “T” word is ultimately going to be the biggest decider of what gets out the door.

We’ve been a bit quiet on the surface lately, but believe us, there’s a lot in store. Almost too much, but that’s the fun of the business sometimes!