Go Back   Hardware Canucks > HARDWARE > Video Cards

    
Reply
 
LinkBack Thread Tools Display Modes
  #31 (permalink)  
Old June 17, 2010, 05:26 PM
geokilla's Avatar
Hall Of Fame
F@H
 
Join Date: Aug 2008
Location: Toronto
Posts: 3,622

My System Specs

Default

Quote:
Originally Posted by SugarJ View Post
Not in a time demo or in-game benchmark. In open game play , yes absolutely it can. That's why you do multiple runs, and over a longer time frame, so that your average FPS is closer to what it actually should be. In a short clip or 20 second play through, you could have events that spike or drop your framerate for a couple seconds. That will have more of an impact on the average framerate in a short run, while the effect would tend to be minimized in a longer run.
So unless my understanding of this is completely wrong, time demos are the built in benchmark feature and in-game benchmark are clips of the actual gameplay correct? Then what's stand-alone benchmark as seen here?



In short, I'm basically confused with what XXX benchmark or whatever is associated with when it comes to reviewing games. The actual game sequence where you're on a mission? The built-in benchmark that the game has? A clip recorded using FRAPS or whatever than replayed multiple times using the same and different GPUs? That kinda thing.
__________________
Reply With Quote
  #32 (permalink)  
Old June 17, 2010, 05:37 PM
Arinoth's Avatar
Moderator
F@H
 
Join Date: May 2009
Location: Halifax
Posts: 8,574

My System Specs

Default

Very well done, way to keep us proud
Reply With Quote
  #33 (permalink)  
Old June 17, 2010, 06:21 PM
SKYMTL's Avatar
HardwareCanuck Review Editor
 
Join Date: Feb 2007
Location: Montreal
Posts: 11,605
Default

Quote:
Originally Posted by geokilla View Post
So unless my understanding of this is completely wrong, time demos are the built in benchmark feature and in-game benchmark are clips of the actual gameplay correct? Then what's stand-alone benchmark as seen here?

In short, I'm basically confused with what XXX benchmark or whatever is associated with when it comes to reviewing games. The actual game sequence where you're on a mission? The built-in benchmark that the game has? A clip recorded using FRAPS or whatever than replayed multiple times using the same and different GPUs? That kinda thing.
Here we go:

A Timedemo: This allows you to record a gameplay sequence AS YOU PLAY and then play it back as needed often using the in-game console commands. You can use FRAPS to record a MOVIE but a timedemo is played back IN-GAME using the game engine.

Built-in-Benchmark: These are rolling sequences which are either flythroughs or actual in-game sequences. A flythrough usually means no characters or interaction as you would find in the game. The in-game sequence is like a timedemo but is predefined rather than user-defined. The main issue with this is that GPU manufacturers can sometimes optimize performance for ONLY the benchmark and not the actual game. The same can be said for the next item...

Stand-Alone: A stand-along benchmark is just that: a separate program that installs all of the game files in a separate folder from the game itself and allows you benchmark but no interaction. As stated, they don't get patches so performance will be affected.
__________________
Reply With Quote
  #34 (permalink)  
Old June 17, 2010, 09:15 PM
Rookie
 
Join Date: Nov 2009
Posts: 16
Default

NO Crysis? Any game with an average of 30fps is already playable, any game over 60fps has no noticeable difference, that is why benchmark like Crysis are important, they can make any card fall to their knees and keep them there. Meaning if its possible to buy a card that can game Crysis at 60fps with everything max-out, no any other game will ever bother you with choppy images. Which is also what is lacking here.
Reply With Quote
  #35 (permalink)  
Old June 17, 2010, 09:22 PM
rjbarker's Avatar
Hall Of Fame
 
Join Date: Feb 2008
Location: Courtenay, B.C
Posts: 5,932

My System Specs

Default

Both JC2 and Metro33 are the "new" Crysis.....no need for Crysis anymore.
__________________
Introducing me n my OCD to Watercooling, is like taking an Alcoholic to an "all you can drink" Beach Bar in Mexico

.
Reply With Quote
  #36 (permalink)  
Old June 17, 2010, 09:59 PM
SKYMTL's Avatar
HardwareCanuck Review Editor
 
Join Date: Feb 2007
Location: Montreal
Posts: 11,605
Default

Quote:
Originally Posted by johnjohn View Post
NO Crysis? Any game with an average of 30fps is already playable, any game over 60fps has no noticeable difference, that is why benchmark like Crysis are important, they can make any card fall to their knees and keep them there. Meaning if its possible to buy a card that can game Crysis at 60fps with everything max-out, no any other game will ever bother you with choppy images. Which is also what is lacking here.
Crysis was pointless for this comparison IMO. It is over with the Crysis commentary because it has been trumped by Metro 2033.
__________________
Reply With Quote
  #37 (permalink)  
Old June 18, 2010, 06:33 AM
martin_metal_88's Avatar
Hall Of Fame
F@H
 
Join Date: Apr 2009
Location: Quebec City
Posts: 4,175

My System Specs

Default

Thx alot sky! Probably one of the most usefull article recently done! It's nice to know more about methodology and other thing like that. Carry one with good work!!!
__________________
Ask for more

Martin_metal_88
Colenzo : See system spec!

Odysseus : I7 860 @ 3.6Ghz || GA P55-UD5 || G.skill Ripjaws F3-16000 2000Mhz 2X2GB || Coolermaster ATCS 840 || Silverstone Strider plus 850W || WD 640 black - WD 1TB green - Samsung F3 2TB || MSI 5670 1GB Cyclone ||Coolermaster Hyper 212 + ||

Reply With Quote
  #38 (permalink)  
Old June 18, 2010, 07:47 AM
Rookie
 
Join Date: Jun 2010
Location: Montreal
Posts: 2
Default

Wow, very nice article ! --- I also registered to the forums just because of it

I am somehow not surprised by the results. Of course the best way to benchmark a demo is to run it manually... but I am glad that you compared this to the timedemos, usually we assume the result would be the same (and I'm glad it's similar). The added worst case scenario bench is really a plus!

It's just too bad that the timedemo feature is so rare. I guess it shows the priority of the QA and the team resources.

I am curious, how much time did it took to make all the benchmarks ??
Reply With Quote
  #39 (permalink)  
Old June 18, 2010, 08:43 AM
SKYMTL's Avatar
HardwareCanuck Review Editor
 
Join Date: Feb 2007
Location: Montreal
Posts: 11,605
Default

Quote:
Originally Posted by Appat View Post
I am curious, how much time did it took to make all the benchmarks ??
The research for this article was done slowly over the course of 4 months. THe testing was done over the course of 3 weeks so in total I would say I burned about 30 hours on it.
__________________
Reply With Quote
  #40 (permalink)  
Old June 18, 2010, 10:26 PM
geokilla's Avatar
Hall Of Fame
F@H
 
Join Date: Aug 2008
Location: Toronto
Posts: 3,622

My System Specs

Default

Quote:
Originally Posted by SKYMTL View Post
Here we go:

A Timedemo: This allows you to record a gameplay sequence AS YOU PLAY and then play it back as needed often using the in-game console commands. You can use FRAPS to record a MOVIE but a timedemo is played back IN-GAME using the game engine.

Built-in-Benchmark: These are rolling sequences which are either flythroughs or actual in-game sequences. A flythrough usually means no characters or interaction as you would find in the game. The in-game sequence is like a timedemo but is predefined rather than user-defined. The main issue with this is that GPU manufacturers can sometimes optimize performance for ONLY the benchmark and not the actual game. The same can be said for the next item...

Stand-Alone: A stand-along benchmark is just that: a separate program that installs all of the game files in a separate folder from the game itself and allows you benchmark but no interaction. As stated, they don't get patches so performance will be affected.
Hm I think this answers all my questions. Thanks :)
__________________
Reply With Quote
Reply


Thread Tools
Display Modes

Similar Threads
Thread Thread Starter Forum Replies Last Post
Guide to the Hardware Canucks GPU Benchmarking Process SKYMTL Video Cards 18 April 3, 2014 05:58 PM
Christmas '08: GPU Performance Part II Comment Thread SKYMTL Video Cards 26 January 15, 2009 09:43 AM
The Games of Christmas '08 GPU Performance Comment Thread SKYMTL Video Cards 33 December 12, 2008 06:39 PM