by Rory Buszka on July 14, 2008 in Motherboards
What if you could build a sub-$500 HTPC that was slim, fast, and capable, without the headache of picking bargain-barrel parts to accomplish your design goals? What if it had HDMI and 7.1 audio? With the help of their Pundit series, ASUS hopes to make barebones relevant again. Have they done it?
At Techgage, we strive to make sure the results derived from hardware testing are as accurate as possible. Our rigorous testing focuses not only on testing the hardware in a wide variety of anticipated uses and capacities, but on ensuring that results are both accurate and repeatable, as well as consistent with our expectations for a particular piece of gear.
We feel that while this effort to perfect our benchmarking methods and test suites can be time-consuming, the time spent is well worth it to deliver high-quality, easily-interpreted data for our readers that reflects levels of performance that they can expect to achieve.
Here’s our full-disclosure rundown of all involved hardware, software, and global testing variables. If there’s something we’ve overlooked, or if you wish to make recommendations, please feel free to contact us via email, or create a post in our forums.
To ensure that our hardware test-beds perform consistently, we follow the following system preparation procedures and guidelines.
- The OS installation is left as basic as possible, with the latest service pack applied to bring the operating system software up-to-date in a fashion that’s easily comparable to prior benchmarking runs.
- All required drivers are installed from the motherboard CD-ROM. If there exists an improved version of the driver that is known to provide improved performance, we’ll upgrade to that one.
- No virus scanners, firewalls, or security applications are installed. These programs often run in the background, introducing performance-robbing ‘lurking variables’ to the testing process. However, when possible, we ensure that all installers for drivers and testing software have been scanned to be virus-free beforehand, if they weren’t installed from media supplied by the manufacturer.
- Scrap files from previous testing are cleaned up, and the ‘Recycle Bin’ or ‘Trash’ is emptied. This measure avoids clutter and slowed file system performance from benchmarking runs replacing or overwriting files that already exist.
- Internet access is disabled. Our benchmarking hardware test-beds literally have no link to the outside world. This avoids any CPU cycles being devoted to handling of network or internet traffic.
- The computer has proper airflow, and ambient air temperature is less than 80°F. While it’s rare that a PC operating at its stock specifications will encounter overheating, high ambient temperatures can put an artificially low ceiling on overclocking results.
- Hard drives affected by testing software are defragmented. We use Diskeeper 2008 to defragment the hard drive prior to every fresh benchmarking run, except when individual testing runs require a clean installation of Windows, as in the case of SYSmark 2007.
Test System Hardware
No hardware is changed during performance testing except for the part under test, when comparisons between multiple products must be made. The system configuration used in testing for this article is as follows:
- CPU: AMD Athlon 64 X2 4200+ EE (65W)
- Reference Motherboard: ASUS M2NPV-VM (NVIDIA nForce 430)
- Memory: Crucial Ballistix 2GB DDR2-800 (DDR2-800 5-5-5-18 2T)
- Video Subsystem: Integrated Graphics Processor (AMD Radeon X1250, NVIDIA GeForce 6150)
- Audio Subsystem: Onboard HD Audio Chipset
- Storage: Western Digital Caviar 160GB, 7200RPM, 8MB Cache, SATA 2.0
- Power Supply: Included PSU; Corsair VX450W (reference machine)
- Chassis: Included Chassis; NZXT Hush (reference machine)
- Operating System: Windows Vista Ultimate 32-Bit
- Cooling: Stock Thermal Solution; Noctua NH-U12P (reference machine)
Note: The NVIDIA nForce 430 chipset’s “ForceWare” driver software attempts to install NVIDIA Network Access Manager, a hardware-accelerated software firewall. The installation program is not permitted to install this feature during our driver installations, pursuant to our guidelines regarding system preparation.
For this review, we’ll be using Windows Vista 32-bit in order to maximize driver compatibility. Windows Vista Service Pack 1 (32-bit) is installed, and all screen saver or power-saving options are disabled. The Vista Welcome Center, UAC, and Sidebar are all disabled. A recent driver is used for our reference motherboard, while the ASUS-provided drivers for the P2-M2A690G are used for the barebone system under review.
For all testing with the exception of SYSMark 2007, all benchmarks are run once, and then the system is restarted before running the suite of benchmarks again. The scores are then averaged. All testing is performed with the Aero UI active.
To deliver the most accurate results possible, games that we test with are played manually, with FRAPS 2.9.4 being used to collect FPS (frames-per-second) data. We have determined experimentally that manual gameplay provides more consistent, true-to-life results than time demos for FPS testing, since time demos rely heavily on the CPU’s speed.
Each title we use is thoroughly explored to find the level that’s best-suited to benchmarking, and a route through the level is then chosen and followed for all subsequent test runs. We are not robots, so we cannot make sure that each run is identical, but the duration of the testing run ensures that differences in gameplay decisions have virtually no impact on the scores developed. We play each level for a minimum of five minutes to achieve this consistency.
Let’s be realistic here — because integrated graphics processors are being used in both test systems for this review, we’ll need to dial back the graphics settings quite a bit. We’ll be testing all games at 1024×768 resolution to ensure that meaningful FPS figures can be obtained, and Half-Life 2 will get additional testing at a resolution of 1152×864 with the same settings.
In addition, we’ll be choosing combinations of lower-quality graphics settings developed specifically for each game to allow it to run with satisfactory performance on IGPs and very low-end GPUs, though the settings will be the same across all machines under test Below are the settings matrices for both games used in testing.
Company of Heroes:
Half-Life 2: Episode Two:
On the next page, we’ll kick off the testing portion of this article with SYSmark 2007 Preview and PCMark Vantage results.