A Look at Futuremark’s 3DMark 11

[PAGETITLE=Introduction][Top Ad] Every so often, usually at around the same time we all begin to appreciate the performance from our PCs, a benchmark comes along that turns us against them. We have Futuremark to thank for that, as it has been putting out some of the best PC-busting benchmarks for over ten years. During that time, the company has worked hard to make sure that its products remain in the virtual toolboxes of enthusiasts. So far, the company has done a great job of that, as its products are used by novice to enthusiast PC users alike, and are also used as a baseline for overclocking competitions. To make sure that this never changes, the company has consistently released an updated version of its flagship 3DMark tool every couple of years. Its latest release, and eighth for the series, is 3DMark 11. Like the previous releases in the series, 3DMark 11 has one sole purpose in mind... to give some worth to your PC in the form of a score. The tool uses the latest DirectX 11 techniques to help push even the latest graphics hardware to its limit, and as is typical of all 3DMark products, we're likely to see this latest version heavily used for the next couple of years. This article is meant to be a quick-look into the latest version, so I'm not going to discuss the individual tests at great length, but rather get to the heart of the details. If you are interested in delving deep into the an in-depth explanation of Futuremark's methodology, or the individual tests, I'd recommend you download the reviewer's guide and also the whitepaper. 3DMark 11 Versions. A common complaint regarding 3DMark Vantage was that the "Basic" version was indeed a little too basic, and didn't really give anyone a solid feel for the program. Futuremark remedies that quite a bit with 11, allowing all free users 1 score upload (that can be overwritten), unlimited Performance runs, and access to the demo mode (limited to 720p). The Basic edition can be seen below, and it's definitely in all regards "basic". You are limited to a single resolution, and that's about as far as the functionality will get you. You might notice an "Advanced" tab up top. In the Basic edition, all of the options there are wiped out, but a button is added to encourage an upgrade. Futuremark 3DMark 11 - Basic Edition The Advanced edition once again sells for $19.95 USD, and as you can see, it opens up a lot of tweaking capability. Altering anything here will rid the possibility of achieving a "3DMark Score", but you can still get scores for other sub-tests just fine, including a "GPU Score". Unlike the Basic edition, Advanced allows you to upload an unlimited number of run results, handle result files offline, loop the benchmark (for stress-testing) and it's also advertisement free. Futuremark 3DMark 11 - Advanced Edition The Professional edition, which also retains its previous pricing of $995.00, is designed for business use. It opens up some functionality that's not going to be so important for the enthusiast, such as image quality testing, demo looping, the ability to export a saved result as XML, command-line automation and also priority support. Futuremark 3DMark 11 - Professional Edition Compared to Vantage, the results page for 3DMark 11 is about as simple as it could get. Instead of including the GPU and CPU score on the same page, the program instead lists only the overall 3DMark Score. I consider this to be rather unfortunate, as the GPU score, to me, is just as important as the overall score, if not more so. After all, when we pit ten or more graphics cards against each other, it's the raw GPU power we want to measure, not a grouped result. This is one area where I feel the latest 3DMark makes things a little more difficult than they need to be. In order to see the GPU score result as an Advanced user, you must first upload the score to 3DMark.com first, and if you are benchmarking offline, that's not exactly convenient. In order for us to gather our GPU scores, we had to first save the result to our desktop, then use a feature in the Professional edition that enables us to convert that into an XML, where all of the sub-scores are listed. We benchmarked 12 GPUs twice over on three different settings for this article, and had to go through this process for each one. That's tedious. [PAGEBREAK][PAGETITLE=A Look at the Tests][Top Ad] Being that 2011 is right around the corner, "3DMark 11" seems like an appropriate name, but there's another reason it's so fitting: DirectX 11. For all DX 9 and DX 10 testing, Futuremark stresses that 3DMark 06 and 3DMark Vantage, respectively, are the go-to solutions. GPUs that don't feature DX 11 support cannot function with 3DMark 11. As expected, some of the tests employ tessellation techniques, in addition to compute shaders and multi-threading capabilities enabled by DirectX 11. In addition, there's heavy use of volumetric lighting, depth of field, post-processing and of course, physics. Unlike Vantage, 3DMark 11 does not utilize NVIDIA's PhysX SDK for testing the physics capabilities of a GPU, but rather uses the open-source Bullet engine. This being the case, NVIDIA won't have such a major advantage with the CPU score here as it did with Vantage, making the playing field a bit more fair. Instead of re-hashing what Futuremark explains perfectly, here is a run-down of the six tests that 3DMark 11 offers: Graphics Test 1: This test features lots of light sources with many casting shadows in the scene. Volumetric lighting illuminates the cloudy water. Post processing is used for various camera lens effects. There is no tessellation in this test. Graphics Test 2: This test features a few lights which cast shadows, volumetric lighting through cloudy water and tessellation on the structures, corals, rock and sea bed. Post processing adds depth of field and other camera lens effects. Graphics Test 3: This test features tessellation on the pillars, statues and some of the vegetation. Volumetric lighting from the sun casts shadows on the plane. Post processing adds depth of field and other lens effects. Graphics Test 4: This test features tessellation on the pillars, statues and some of the vegetation. There is volumetric lighting and multiple light sources with several casting shadows. Post processing adds various camera lens effects. Physics Test: This test focuses on CPU performance by simulating rigid body physics with a large number of objects. This test runs at a fixed screen resolution for all presets. There is no post-processing, volumetric lighting or tessellation. Combined Test: This test focuses on CPU and GPU workloads. The CPU handles rigid-body physics while the GPU is tasked with volumetric lighting, tessellation, post processing as well as simulating soft-body physics using DirectCompute. It's clear that Futuremark believes tessellation is important, so it's not just AMD and NVIDIA stating it. Equally important are lens effects and other post-processing techniques. Physics are also important, but don't seem to be weighed quite as heavily into the final score as the other sub-tests. Here are some screenshots from the tests: Time to tackle the benchmark results! [PAGEBREAK][PAGETITLE=Benchmark Results; Final Thoughts][Top Ad] As is common with each new 3DMark release, Futuremark has adjusted the default resolution in each profile for the sake of relevance. The "Entry" profile benchmarks at 1024x600, so it's in all regards useless. The "Performance" profile, which utilized a resolution of 1280x1024 in Vantage, has been dropped down to 1280x720 to suit the 16:9 aspect ratio. The same drop has happened to "Extreme", which now benchmarks at 1920x1080. I am of the belief that testing today's graphics cards at a resolution such as 1280x720 is pointless, as we have a hard enough time pushing a resolution like 1680x1050 sometimes even with modest offerings. Despite that, we will still be running it for our tests as it's the only option available to users of the free version. It'd make less sense for us to limit the comparisons our readers can make. To test the benchmark using 2560x1600, we choose the Extreme preset and then adjust the resolution to 2560x1600. It's a simple change, but as 2560x1600 has almost 2x the number of raw pixels as 1920x1080, it will help us better set the high-end cards apart from all the rest. For the sake of not littering up this page with our methodology and system specs, I'll instead link to a recent GPU review where it's all listed in detail. The testing machine hasn't changed for this article, as we'll be implementing the results seen here into our GPU reviews going forward. The test PC used includes a Core i7-975 overclocked to 4.0GHz, 12GB of Corsair DDR3-1333 memory, a Seagate Barracuda 500GB 7200.11 as the OS drive and sufficient cooling all around. The OS used is Windows 7 Ultimate x64. The drivers used for all AMD Radeon cards was Catalyst 10.11, while GeForce 260.99 was used for NVIDIA's GeForce GTS 450 ~ GTX 470 and 263.09 for the GTX 570 and 580. It's clear that 3DMark 11, with its brute-force graphics techniques and adjusted scoring algorithms, will deliver results much lower than what we've seen with Vantage. While AMD's Radeon HD 5970 scored over 13,000 in the Extreme test in Vantage, is scored a measly 2294 at the similar setting here. No complaints... but it will be fun to see who can hit 10,000 first at the Extreme setting! Overall, 3DMark 11 looks to be a great successor to Vantage, although it does seem a bit lighter in overall testing. Futuremark cut off some excess, but even so, the overall package is nice. If I did have one recommendation, it's that I'd love to see the GPU and CPU scores right along the 3DMark score on the result page. It'd be far, far more convenient. If you decide to give 3DMark 11 a go for yourself, why not talk about your experiences and thoughts in our related thread? Discuss this article in our forums! Have a comment you wish to make on...

A Look at Futuremark’s 3DMark 11

Page 157 –

Rob Williams