Go Back   Hardware Canucks > HARDWARE > Video Cards

    
View Poll Results: To Include Vantage or Not??
Yes. We FINALLY have a new 3DMark!! 15 48.39%
Not yet. Wait for the drivers to mature. 8 25.81%
No. Replace it with another game test. 4 12.90%
No. Definately not. Never. 4 12.90%
Voters: 31. You may not vote on this poll

Reply
 
LinkBack Thread Tools Display Modes
  #11 (permalink)  
Old April 30, 2008, 04:55 PM
ipaine's Avatar
Hall Of Fame
F@H
 
Join Date: Apr 2008
Location: Edmonton, AB
Posts: 2,035

My System Specs

Default

I say most definitely add it to the reviews. As FiXT mentions they might not reflect the best real world comparison, but they are a great way to compare one system to another. Well at least to show the differences from one Nvidia to another Nvidia or ATI to ATI, since it seems to vary alot from Nvidia to ATI.

Plus I already have a thread set up for results from people using 3DMark Vantage, which you can find here.

Quote:
Originally Posted by SKYMTL View Post
So it doesn't bother you guys that many of us have seen differences of up to 1800 points in the GPU tests from one run to the next with the SAME hardware?
Really? I don't have the advanced version yet so haven't gotten to play with it too much but the 2 that I have run didn't show any more differences than anything did in 3DMark06. Is that using the newest drivers I take it?
__________________
"Nothing sucks more than that moment during an argument when you realize you're wrong."


Reply With Quote
  #12 (permalink)  
Old April 30, 2008, 05:34 PM
kruzn4evr's Avatar
Hall Of Fame
F@H
 
Join Date: Apr 2008
Location: Ajax, Ontario
Posts: 1,801

My System Specs

Default

My 2 cents worth?...I'd say give it some time for others to try it out 1st and see what the overall opinion is. I have tried it and there are ALOT of features to play with (I should have them all figured by this time next year at my pace!...lol) and even though it may not reflect real world or realistic comparisons I find it a useful and fun tool for experimenting with OC'ing. So far I've scored from P9953 at stock settings then up to P11336 with OC'ing and a few scores in between. All I know is, it IS fun to play with
__________________

Reply With Quote
  #13 (permalink)  
Old April 30, 2008, 06:06 PM
SKYMTL's Avatar
HardwareCanuck Review Editor
 
Join Date: Feb 2007
Location: Montreal
Posts: 11,404
Default

Quote:
Originally Posted by ipaine View Post
Really? I don't have the advanced version yet so haven't gotten to play with it too much but the 2 that I have run didn't show any more differences than anything did in 3DMark06. Is that using the newest drivers I take it?
The Pro version with an ATI HD3870X2 shows between 7700 to about 9000 points from one test to another on the Performance setting.
Reply With Quote
  #14 (permalink)  
Old April 30, 2008, 07:18 PM
MpG's Avatar
MpG MpG is offline
Hall Of Fame
 
Join Date: Aug 2007
Location: Kitchener, ON
Posts: 3,136
Default

Like it or not, it's going to be something that people want to see, if only so they can then start bitching about it not reflecting real world games. If you're getting a high sample variance, then maybe average a batch of runs, then use some sort of averaging mechanism?

It would also be nice to see it run with some options enabled (higher res, filters, etc), which would shift more of the work onto the video card itself. I couldn't venture any stats myself, but I'll wager that the glory days of 12x10 resolutions with no filters are fading fast, with a lot of people grabbing cheap 22" monitors and mid-range GPU's that can handle them without much trouble.
__________________
i7 2600K | ASUS Maximus IV GENE-Z | 580GTX | Corsair DDR3-2133
Reply With Quote
  #15 (permalink)  
Old April 30, 2008, 07:23 PM
SKYMTL's Avatar
HardwareCanuck Review Editor
 
Join Date: Feb 2007
Location: Montreal
Posts: 11,404
Default

I have already figured out a way to take care of the sample variance. I just wanted to know everyone's opinion, ESPECIALLY when considering the sample variance will make it next to impossible for people to compare their systems to others.
Reply With Quote
  #16 (permalink)  
Old April 30, 2008, 07:26 PM
JD's Avatar
JD JD is offline
Moderator
F@H
 
Join Date: Jul 2007
Location: Toronto, ON
Posts: 6,815

My System Specs

Default

To be honest, I think it should be boycotted until it there's a free version that can be run continuously and be able to post on ORB. I believe HWBot is going this way and trying not to support it.

I guess if you do include it (seems like most people want it), make sure you do a few runs and average it out. Also make sure all other benchmarks are done in Vista as well, and that it's clearly stated, as you can't compare Vista benchmarks to XP as far as I'm concerned. Seeing as the Vista adoption rate is rather low, this is probably a key note to mention.

As we all know though, 3DMark really doesn't have much value compared to actual games. Vantage (trial) runs way worse than any of the DX10 games I've played. Also looks quite poor too compared to current games, on "Performance" preset at least as I'm not paying for a program that gives me a number.

Oh, as per the 22" screen thing, I agree there. Should definitely start use 1680x1050 as one of the default resolutions as more and more people are using this now.
Reply With Quote
  #17 (permalink)  
Old April 30, 2008, 07:40 PM
enaberif's Avatar
Hall Of Fame
 
Join Date: Dec 2006
Location: Calgahree, AB
Posts: 10,473
Default

I say no.

To many people take into consideration these synthetic benchmarks and then wonder why their card will perform better or worse after synthetic shows their card to be great.
Reply With Quote
  #18 (permalink)  
Old April 30, 2008, 08:01 PM
CanadaRox's Avatar
Allstar
F@H
 
Join Date: Feb 2008
Location: Scarborough (Toronto)
Posts: 614
Default

I think we should wait until the results are a bit more stable. With results of a single system having such a huge range, its hard to know whats actually best. And even if we average a certain number of results, who is to say what numbers of tests to run which will give an accurate final score.
__________________
Project: Black and White
i7 920 D0 | 3 x 2GB DDR3 | EVGA X58 SLI LE
XFX 4890 | Corsair HX750 | Corsair Obsidian
Reply With Quote
  #19 (permalink)  
Old April 30, 2008, 08:11 PM
MpG's Avatar
MpG MpG is offline
Hall Of Fame
 
Join Date: Aug 2007
Location: Kitchener, ON
Posts: 3,136
Default

Personally, if a video card review doesn't use it, I'm going to loose approximately 0.0000001 secs of sleep, especially since my gaming comp is now on XP. But I'm still mostly in favour of including it.

Many of the other arguments all apply to other popular benchmarking software. 3DMark06 doesn't reflect real games either, and DX9.0c wasn't nearly as 'widespread' when it first came out either. Crysis has apparently barely sold 100k copies or so, even if about 500k people seem have opinions on it and want to see the benchmarks.

In favour of Vantage is the fact that it DOES kick the hell out of current top-end graphics cards, especially on the higher settings. When card A is giving you 70fps, and card B is giving you 80fps, things aren't nearly as critical. But when you're down in the 20's, each fps is a little more important.

The only thing that jumps out at me is that sample variance issue, which you'd think Futuremark would have commented on by now. I'd have misgivings about using a ruler that was +/-20% accurate, no matter how many times it was averaged.
__________________
i7 2600K | ASUS Maximus IV GENE-Z | 580GTX | Corsair DDR3-2133
Reply With Quote
  #20 (permalink)  
Old April 30, 2008, 08:30 PM
SKYMTL's Avatar
HardwareCanuck Review Editor
 
Join Date: Feb 2007
Location: Montreal
Posts: 11,404
Default

Quote:
Originally Posted by jdrom17 View Post
To be honest, I think it should be boycotted until it there's a free version that can be run continuously and be able to post on ORB. I believe HWBot is going this way and trying not to support it.
Let's start this with the above quote. I think this is the major issue MANY people will have with this since you will have to pony up the money for the right to run this benchmark more than once. Personally, I land in both camps since there are plenty of benchmarks that we have access to that are not available for free and are very good at comparing one product to another. At the same time I have a bit of an issue with having someone having to "pay to compare".

Quote:
I guess if you do include it (seems like most people want it), make sure you do a few runs and average it out. Also make sure all other benchmarks are done in Vista as well, and that it's clearly stated, as you can't compare Vista benchmarks to XP as far as I'm concerned. Seeing as the Vista adoption rate is rather low, this is probably a key note to mention.
Did we mention that Vantage is Vista only?

Quote:
Oh, as per the 22" screen thing, I agree there. Should definitely start use 1680x1050 as one of the default resolutions as more and more people are using this now.
Please remember that if a game runs well at 1600x1200 it will do so at 1680x1050. That is why we use it: it encompasses more than one set of buyers.
Reply With Quote
Reply


Thread Tools
Display Modes