Playing the Game

Reviewing and previewing products is a tricky business. To help you better understand what those who do either have to deal with, I offer this run-down of how things can generally play out:

• Receive a product you were told you’d get a week ago, two days before NDA;
• Receive a reviewer’s guide detailing “new features” that the company wants highlighted and “optimal testing strategies” that the company tells you will “stress” the product;
• Plug everything together and hope nothing was damaged in shipping;
• Throw every test you can think of at it, and then shrug your shoulders if results are run-of-the-mill until you run the “recommended” tests;
• End up writing most of your night and into the next morning, trying to include all of the information (both boring and interesting) from the results in time for the NDA;
• Largely have readers read the first page and final conclusion, where you usually end up writing about the places the product differentiated instead of where it middled out, while skimming the rest of the ten to twelve pages.

Yes, it’s that glamorous. Not that I’m having a whine, mind you – we still get to play with great kit and computers all day. But this is what we now call the editorial process – and as the delays for products shipping grow longer and longer, each site and team gets less and less time to truly put a product through the paces and develop an unbiased and understanding view of it. Instead, the writers are pushed to take shortcuts, rely on the press notes, and regurgitate PR drivel in order to hit the fabled launch-day review.

This process has become the norm nowadays, but we all still try. Some sites, like ours, have chosen to give up entirely on NDA release dates – we want our readers to be informed, and so we take a couple extra days with the hardware before putting our opinion out there. Others do their absolute best to try and provide a fair review within the time constraints, while still making sure that their content hits when you expect it to. After all, research has shown that people will read a few early reviews (and only a few parts of those) before making a decision on a product and will rarely return to look for more information later.

Journalists were quick to incite inflammatory rhetoric about how AMD was biasing reviews and trying to show off its good side without displaying the whole picture.

It’s that exact research that has ruffled quite a few feathers with AMD’s new Trinity APU release. Trinity is the followup to AMD’s Llano chip, which comes with a GPU built right next to the CPU on the same die. This “all-in-one” solution takes aim squarely at Intel’s CPUs and motherboards that utilize on-board graphics – which are known to be some of the weakest in the industry. AMD’s hope is that the graphics performance of Trinity will teach people a lesson on what can be done on-board in a cost-effective manner, and so the company has created the first staggered release NDA in history – sites were permitted to review the GPU a week earlier than the CPU.

Now, let’s go back to that research that has shaped our journalism industry so well. Sites rush to release information about the hot new release, but are now limited to only talking about half the picture – quite literally, the picturesque (and picture-making) half. Readers gobble up the information, see how Trinity slaughters Intel on-board graphics (and even many of the truly low-end discrete GPUs) and make their internal decision, likely never to read another word about it. But the reader never got the true picture of the CPU. And since all of the games to test the graphics of Trinity are played on a Trinity die, the only thing that the reader learns about the CPU is that it’s “good enough” to do those tasks.

“Off with their heads,” many cried. Journalists were quick to incite inflammatory rhetoric about how AMD was biasing reviews and trying to show off its good side without displaying the whole picture. Some sites refused to release any content until they could release the total review; others made sure to blast the unusual NDA in their “preview” and chose language specifically designed to highlight the fact that the CPU could not be talked about.

All this journalistic integrity is wonderful, but I think many people missed the boat on something – weren’t reviews already being “shaped” long beforehand by delayed product shipments, paper launches, “recommended” testing procedures, blacklists, and everything else that’s been prevalent in this industry (and getting more so) for years? Nobody cries when their competitor site gets shut out of a release altogether because it didn’t toe the “company line” of PR. Very few, if any, choose to miss an NDA because the product didn’t show up on the doorstep until the day before. Somehow that doesn’t affect the quality of our reviews?

It also doesn’t cover the fact that many of the improvements from one product to the next can be nearly indecipherable to the end user, or that real-world applications will likely see a very different behavior. Chip X is 21% faster than Chip Y at computing this specific bit of code… and in 200 second tasks, that’s relevant. But for the people who are only doing things that require two-tenths of a second at best, running completely different code, 21% faster is not worth much. And that would be assuming it scaled linearly, which it wouldn’t – your tasks might have heavier use of cache or RAM, depending on your software design.

Indeed, as much as we tout the integrity of our numbers, by and large we have to use such specific and obscure metrics that they rarely bear out into real-world performance – especially on the CPU side. That’s not to say that the figures are totally unimportant, but as long as you’re in the relevant “class” of CPUs, you probably won’t notice much real-world difference. Unless you’re aiming for some very specific tasks, by and large a CPU is either “good enough,” or it isn’t.

And suddenly, we start seeing where AMD is coming from.

At its release, Llano was the perfect chip for its purpose – and one-of-a-kind. The GPU beat Intel integrated graphics into the dirt and even put some bottom-rung discrete GPUs out of the running – all in under 100W TDP with the CPU. However, you only need to look at any reviews of the product to see that it loses a lot of points for its CPU being “middling” and lacklustre. Further insult was added to the injury by the fact that nobody includes Llano in their GPU suites, even when comparing low-end GPUs (as it’s not only a GPU and can’t be plugged into many sites’ GPU test rigs).

Unless you’re aiming for some very specific tasks, by and large a CPU is either “good enough,” or it isn’t.

And suddenly, we start seeing where AMD is coming from.

Here we had a product that could completely dominate for home theater systems and lower-resolution gaming – and after its initial “hey, cool!” it was forgotten about. It was too much CPU to be a GPU and too much GPU to be a CPU, so it was just ignored by the journalists. At the end of the day, a great product got lost in the shuffle because AMD trusted the testers and journalists to give Llano more than a passing thought and a pigeon-hole as “not quite” anything… despite HTPCs (and gaming on them) being a growing market.

And now, after the collective “we” of tech reviewers dropped the ball and shunned a product to oblivion simply because it didn’t fit our review mold (and did consumers a disservice in the process), AMD has come up with a way to help make sure we pay attention this time. The company has staggered its NDA to make sure that the GPU and CPU are accounted for and tested separately, knowing that as sites pushed to reach the Trinity GPU NDA, they wouldn’t be able to bias that performance with the CPU being “good enough, but not great.” After all, that’s really all many consumers end up needing, particularly in that market.

Is that data truly misleading you, the reader? I don’t think so. I think it’s right on target – the focus of Trinity is to have competent graphics performance and a decent CPU all within a 100W envelope, cheaper than Intel can sell its cheapest CPU alone.

AMD swung for the fences and nailed it. It leaves Ivy Bridge graphics in the dust and takes a good bite out of a $100 budget discrete GPU (NVIDIA GT 640)… where you’d still need to buy a processor. The situation gets even rosier for AMD when you think of another feature in the Trinity package. Spend $75 on a cheap AMD 6670 GPU instead of that GT 640 and you’ll get even more bang for the buck – Trinity’s dual-GPU technology is something akin to CrossFireX, only better because of the direct CPU and bus tie-in. That means for $200 you’d have an HDTV gaming powerhouse.

Separating the GPU review from the CPU in the case of Trinity wasn’t AMD making us play by it’s rules or hiding facts from consumers – it was making us play fair by our own rules, giving it the same treatment that a GPU should get. Even if the CPU performance was lacklustre (hint: it’s really not, especially for the price), it made us look carefully at what it was designed for and what it should be paired with. I wouldn’t be surprised to see “Trinity A10 + 6670” (or similar) show up in GPU tests down the line – in fact, I’ll be disappointed if I don’t.

If we journalists want to cry foul at the “manipulation,” we only have ourselves to blame. AMD tried with the Llano chips to let us figure out how we’d handle it, and we answered in resounding unison – we wouldn’t. This time, the company has too much at stake and too good of a product at the plate to let us pass it off… and it wins all around – two pieces of serious coverage a week apart, both with great results. Consumers win, too – this will be a launch that won’t be quickly forgotten, and the product is (for once) worthy of memory.

AMD rolled the dice and took a gamble on what is akin to the “Hail Mary” pass in football… and came up a winner.

And to that, all I can say is: good game.

Discuss this article in our forums!

Have a comment you wish to make on this article? Recommendations? Criticism? Feel free to head over to our related thread and put your words to our virtual paper! There is no requirement to register in order to respond to these threads, but it sure doesn’t hurt!