Techgage logo

Windows Vista Beta 2 Performance Reports

Date: June 14, 2006
Author(s): Rob Williams

Windows Vista is system intensive, as we knew it would be. We set out to find out just -how- system intensive it is. We run a slew of benchmarks and gaming runs on both the 32-Bit and 64-Bit versions to see how they compare to XP.



Introduction


When Microsoft develops a major software product, the entire world knows about it. Vista is one of the few products that has ever received this much attention, but it’s for good reason. Windows is the leading OS on the market, which leads to many people dependent on it to be a standout product. Since the first incarnation of Longhorn in 2002, the OS has made numerous alterations and cuts, namely WinFS.

In early Alphas of Longhorn, it merely resembled Windows XP with a gray skin. Since that time though, things have changed drastically, and Vista Beta 2 is nothing like the system was even two years ago. The original goals are still there, but since that time, more have been added.

One thing that each new beta of Vista has proven well to do, is become more system intensive than the last. The Aero Glass feature alone can be responsible for eating up over 100MB of ram. The OS in general is quite bulky, especially the Ultimate, which makes people wonder how they are supposed to run it at release. After a fresh WinXP install, you can usually expect near 30 processes to be running, but in Vista, that number skyrockets to 40+. Not only are there more processes though, but each one takes a significant amount of ram.

In the picture above, you can see the processes after I closed everything I didn’t need. The DWM.exe is what’s responsible for the desktop windows, which takes a lovely 50MB by itself. If you turn off the Aero Glass theme, that figure goes down to below 10MB.

While this doesn’t seem like a big deal in the grand scheme of things, it isn’t. However, when you want to get into system intensive applications, it may be. Microsoft is already recommending no less than 1GB for use with Vista, and I personally wouldn’t recommend anything under 2GB if you plan to be doing any gaming. As mentioned in our 4GB memory article last month, companies like OCZ are readying themselves to release 4GB kits of ram. So, it’s not hard to understand that Vista is gearing up to require a very beefy machine.

Not only will the OS test your memory and CPU, it will also require a good video card that supports DX9. Simply running Fraps in Vista will tell you that the entire environment is being rendered, because it actually will keep a FPS counter in the corner, as it would in a game. Do we really need to go to an accelerated environment for an operating system though? I am doubtful, but it sure does look pretty.

I have been planning to run these benchmarks for a while, but had to pick the right time. Since Microsoft has released a public beta of Vista, I feel that means that they are confident enough of it’s quality. Would they release a beta that has -so- many bugs on purpose? Of course, take a look at their retail programs. Either way though, all of these benchmarks are not to be taken that seriously, because this is still a beta. In the next four or five months that they take to iron out the last bugs, anything could happen. As it stands though, this is what you can expect performance wise from Vista.

Incompatibilities with Vista

Before I jump into the testing results, I will explain a few problems that I ran into. There were a few benchmarks that I had wanted to include in this report, but they simply would not run in Vista, even after changing it’s compatibility option. One program was ACDSee, which I wanted to use to resize and convert a mass amount of files at once, but the program just simply would not run.

Another problematic application was Sandra 2007, although it’s 2005 version gave the same issues. The program would install in the 32-Bit version of Vista, with a little trouble, but ran fine afterwards. In the 64-Bit version, it would always install fine, but it would never actually load any of the modules it was supposed to. I had tried re-installing it around 5 times using different installation options, but it simply would not load. This is not due to the version of the program, because a standard install of Sandra includes support for both 32-Bit and 64-Bit.

Everest also gave some various problems, namely a BSOD (Blue Screen of Death) when running memory benchmarks. The settings were fine for the memory though, and this was the only program to cause such an issue. BSOD’s in Vista run through very quickly compared to XP ones, so you barely have time to read the error.

Surprisingly, installing the games didn’t prove much of a problem. However, with Battlefield 2, it was more difficult to load a level than in the 32-Bit version. When loading the level, the application would simply drop. It took 7 times to load the level I needed to benchmark, which is a tad ridiculous.

Serious Sam 2 ran perfectly, though the opening game movies would not load. I could hear the sound, but see no video.

Overall, there were many problems to be had, most of which took an entire day out of my weekend, but what fun would it be if it went smoothly?



Testing Procedures, CPU Benchmarks


To perform the testing, I used a formatted 160GB Western Digital hard drive. In between each new operating system, the drive was reformatted. Only one operating system was installed at a time. In usual reviews that require benchmarking, unneeded services are closed down before testing. However, since this is a direct comparison to a new OS, all of the processes and services that are pre-installed, were left alone to give a more accurate view.

Generally, nothing major was done to the operating system after the install. As I do with all my Windows installations, I select the “Classic Start Menu” mode, which effectively makes the start menu smaller than default. It also places your “My Computer” and other various icons on the desktop. In Vista, the Sidebar was disabled. These are the only modifications done to either OS installation.

I will be splitting up the results into three categories: CPU, Memory and Gaming. Performance based on these factors will be the true deciding factor on whether anyone should upgrade right away. Let’s get underway with a look at the CPU benches.

CPU Benchmarking

Because SANDRA would not function in the 64-Bit version of Vista, there are obviously no results from there. Comparing the 32-Bit version of Vista to the 64-Bit version of XP is surprising. While x64 takes the crown with the Dhrystone, Vista storms ahead in the Whetstone result.

The multi-media test is vice versa. Vistas Int result is far better than what x64 could muster, although x64 performed better in the Floating-Point result.

While the end result of a PC Mark run doesn’t explain much, the higher the number, the better. Although x64 scores 8% higher, I had actually expected the difference to be larger. All of those extra running processes in Vista don’t hurt the overall CPU score here that much.



CPU Benchmarks Cont.

If your computer is a workstation, then Cinebench is a great reference benchmark. The faster your CPU can render the image, the higher the score. The 64-Bit version of the program was used under the 64-Bit OS’, and made a very noticeable difference. Between Vista and x64 though, the performance difference is rather minute.

Super Pi calculates a specified about of digits past the decimal place. In our case, I ran the 8 Million test, which turned out to be in x64’s favor. Overall, x64 shaved 12 seconds off what the 64-Bit version of Vista could do. For minor multi-media benchmarking, I used Nero Recode to recompress a DVD video. I first extracted the video, in this case “Bad Religion – Live At The Palladium”, and used Recode to compress it so that it could be burned to a standard 4.5GB DVD.

The results are odd here. There was a huge hike in 64-Bit Vista with 2GB, but that didn’t happen with 4GB installed. In the end, x64 proved to compress the video more than 2 minutes faster than Vista.

The 32-Bit version of Sciencemark was used through all the benchmarks, due to the 64-Bit one giving very inaccurate results regardless of current OS. While the Cipher benchmark didn’t sway in either OS’ favor, the Moldyn is another story. Under the x64 install, it proved near 50% faster than Vista. Not a small difference!

For the last of our CPU benches, I used EVEREST 2.80 Ultimate Edition. The differences are once again minor, although x64 comes out in top overall.

Overall, we can see that CPU wise, Vista can really hold back your PC. Though it does exceed x64 in some respects, it falls short more often than not. Let’s see how Vista handles our RAM and hard drives.



Memory and Storage

Since Sandra didn’t work in 64-Bit Vista, we had no results to add to the graph. x64 does beat out Vista, though it’s a very small difference. I was doubtful that Vista would weaken HDD performance to a large degree, and it’s looking that way already.

While the maximum read is on par all around, the average read sways again to x64’s favor. This are very small differences either way, and would be impossible to detect during normal PC use.

The same goes for HD Tach. Pretty congruent results all around.

Even though x64 once again came out on top with the Sandra benchmark, I had expected an even larger difference. The 4GB came far behind the others due to the forced 2T timings.

Like the previous four benchmarks, Everests memory benchmarks prove close to one another again. Surprisingly, the 64-Bit version of Vista comes out on top here.

As was really expected, Windows XP has slightly better read and writes over Vista. They are so minor however, that it wouldn’t be a deciding factor in choosing your OS. What would be the deciding factor? Games!



Game Benchmarking, Conclusion


Last, but not least, some of the most important benches. Currently, the Beta 2 of Vista does not go as far to support SLi. There is no real need to get too in-depth with these benchmarks, as it would be more suitable closer to release.

As it’s been mentioned before, some are recommending that 4GB of ram may be the requirement for a hardcore gamer. Of course, we will see about this in the coming months, when we finally see SLi support and the ability to run crazy resolutions. As it stands right now, chances are that 4GB will only slow down your computer, even with gaming, due to the 2T timings.

Throughout all three 3D Marks, Win x64 comes out the definitive leader. Though 3D Mark 01 is showing it’s age [heck it has been since 2002], Vista falls quite short compared to WinXP. 5,000 points is not a small chunk. The routine continues with 03 and 06. 03 scores around 11% better, and 06 around 2% better in Win x64.

For real-time gaming tests, I used BF2, HL2, Far Cry and Serious Sam 2. All four are first person shooters, and all have their fair share of eye candy and great graphics. To create the average FPS results, I manually played through the same level in each respective game, for five minutes. Results were captured with FRAPS, which supports both 32-Bit and 64-Bit versions of Windows.

3D Mark was just a tease at the differences, but these benchmarks straight out prove that Vista is not shaping up to be the ultimate gaming platform. Half-Life turned out to be 30FPS on average fast than in Vista, but Far Cry is an even bigger difference. In Vista, that one proved an average of 37.37FPS, but in Win x64 it jumped up to 76.41FPS. That’s near 40FPS difference. The evidence cannot be any clearer. Drivers can however, make -all- the difference, so hopefully we will see performance increase as the drivers mature.

Conclusion

This has strictly been a performance report, but performance aside, it’s obvious that Vista is not ready for retail yet. Some of the benchmarks I use most often were unable to even run, which goes to show just how fussy Vista can really be. Most of your simple applications should have no problems in running, but if you have programs that use any databases on your PC, such as Sandra, they will not likely work. Why, I have no idea, but I would love to find out.

Even beyond that, the programs that do work can easily cause problems. This was evidenced when I had to reload a level in BF2 7 times before it would actually work. Because of the nature of the crash, Vista needs some tweaking done in order to fully support the applications we use on a daily basis.

Most of our benchmarks proved one thing.. and that’s that current Windows is faster than what Vista can provide. The tested computer is not outdated by any means, but Vista did not work as seamlessly as I would hope by this point in time. The biggest issue lies within games. It’s no surprise… gaming suffered a lot on the new platform. Even after closing many services and turning off themes, games still did not come close to Windows XP performance. If this is how things are going to be, WinXP will long remain the top OS until people are forced to move to Vista due to DX10 or other technologies.

Should you give the beta a go? Of course. If you have an afternoon to goof around with, yes you will have fun experimenting with what’s new. If you are one of the few who planned on using this as their primary OS, despite the BETA tag, then you will regret it fairly quickly. Merely copying a registration key file to a products directory brought up a status bar, which lasted around 3 seconds. That wouldn’t be a big deal usually, but this key file was 1.3 kilobytes.

If Microsoft plans on sticking to the schedule, we should see the RC1 within a few weeks, and the retail version in early 2007. Because of this, that means they have to technically have the entire OS ready to ship by at least the end of September. Not to speculate, but can they rid all these problems within a 4 month period? We’ll have to wait and see what happens.

As more enhancements are made to Vista, or Beta 3 / RC1 becomes available, I will revisit the benchmarking and see if there actually have been ‘enhancements’. Stay tuned.

Discuss in our forums!

If you have a comment you wish to make on this review, feel free to head on into our forums! There is no need to register in order to reply to such threads.


Copyright © 2005-2019 Techgage Networks Inc. - All Rights Reserved.