Go Back   Hardware Canucks > HARDWARE > Video Cards

    
Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old September 13, 2011, 01:16 AM
r0bc's Avatar
Allstar
 
Join Date: Apr 2009
Location: AB
Posts: 688

My System Specs

Default A new look at game benchmarking

SKY have any thoughts on this.
I looked around and didn't see it posted anywhere, remove it if it has.

Inside the second: A new look at game benchmarking - The Tech Report - Page 1
__________________

Reply With Quote
  #2 (permalink)  
Old September 13, 2011, 02:11 AM
r0bc's Avatar
Allstar
 
Join Date: Apr 2009
Location: AB
Posts: 688

My System Specs

Default

Could the increased micro shutters in his testing been caused by the 1GB limitation of the lower end cards at that resolution?
__________________

Reply With Quote
  #3 (permalink)  
Old September 13, 2011, 11:37 AM
Hall Of Fame
 
Join Date: Feb 2011
Location: Ontario
Posts: 1,384

My System Specs

Default

Looks interesting, I actually asked myself this at christmas because I noticed this type of thing happening on my crappy laptop, where my fps average would stay above 30 but I would have some stuttering. On the resolution he was testing on 1 gb was more than enough. For example I read in a review that metro 2033 maxed out on 7680x1600 was only using 2.2 gb so really startcraft is not a GPU intensive game, and it wasn't anywhere near that resolution.
Reply With Quote
  #4 (permalink)  
Old September 13, 2011, 10:50 PM
Dzzope's Avatar
Hall Of Fame
 
Join Date: Oct 2010
Location: Irishman in Kiev, wOOoo, I'm an alien...
Posts: 2,826

My System Specs

Default

i know the article says the "stuck" frames are less of an issue than the "jitter", But I would argue that if the frame "sticks" often enough (maybe ever 30 sec to minuite or worse if more) this would deter me from buying a card as I have experienced this on a HD 5770 in my old rig.
And although in no way conclusive or tested near thoroughly enough i noticed it was far more prevalent on the AMD cards rather than the nvidia ones through out the graphs.


However, I'm glad to see this brought up here, as I always think minimum FPS is far more important than max / average. Stuttering or "jitter" is equally important and IMO "stuck" frames are a high priority too.
Some sort of testing along the lines of the graphss (not the percentile charts as they don't show the stuck frames well IMO and as mentioned in the article have certain problems) would, if not tooo much extra work, be a welcome addition.
That said it's very conditional to the segment that is chosen to show the data from and a full graph for a full test would be huge (mebby linked to rather than embedded, or a scroll box).

Anyway.. good article, worth a read and some thought.
Reply With Quote
  #5 (permalink)  
Old September 18, 2011, 07:05 AM
Bond007's Avatar
Hall Of Fame
 
Join Date: Jun 2009
Location: Nova Scotia
Posts: 3,009

My System Specs

Default

Nice read. Not saying its the perfect answer, but it would be nice to see "microstutter/jitter" somewhat accounted for in reviews so that people could get a better real world picture of performance and playablilty.
__________________
Desktop: Antec Three Hundred, Z77-D3H, 3570K undervolt @4.1ghz with Zalman Optima Cooler, Corsair Vengeance 2x4gb@1866mhz, HD 6950, Seasonic S12II 500W, WD Black 1TB, Intel 530 120GB
Away from home light Gamer: Acer Aspire 15.6", A8-3500m undervolt and OC to 2Ghz, 2x4gb Corsair Vengeance, Crucial M500 240gb
Reply With Quote
  #6 (permalink)  
Old September 18, 2011, 11:38 AM
SKYMTL's Avatar
HardwareCanuck Review Editor
 
Join Date: Feb 2007
Location: Montreal
Posts: 11,711
Default

I read through the whole thing twice the other day. Basically, I agree with some of what he says and don't agree with other parts. I won't go into too much detail but there are some very important point that everyone should take away from the article:

- Microstuttering is only APPARENT to the viewer in very few situations. Scott mentions this a few times and I agree.

- There is no way to conclusively test for microstutter. The polling rate in FRAPS and even PlayClaw isn't frequent enough to accurately test for it.

- Due to the two issues above, there is absolutely no way to really test for user-viewable microstutter. Some people may experience it while others won't.

- Like it or not, a playable game experience is still defined by the 30-60FPS "rule". Below 30FPS for most games and below 60fps for fast paced games and the player will experience begin to suffer. So the standard type of benchmark is still perfectly valid.

- Frametime as he discussed (first example on the first page) can be completely invalidated by items that don't include the GPU's ability to render a scene. He showed frametime increasing exponentially for a single frame and deemed that unplayable if it happened often. I disagree. A good example of this is the Witcher 2. In order to provide a fluid area to area experience, the game preloads upcoming areas which will cause frametime to spike and cause a "stutter" every few minutes. This is a FEATURE within the game (rather than a GPU issue) that is being used more and more (Dues Ex: HR, Dragon Age 2, etc) to provide a seamless gaming experience without load screens.


One thing I didn't see is what software he used. Without that, no one can repeat and validate his results...and THAT is concerning. Methinks it is FRAPS but PlayClaw can achieve the same....and with slightly different results.

In short, the article was interesting but regardless of the hyperbole, we won't be including additional testing until there is a tool available that can poll framerates and sync issues at very quick intervals. Hence his "inside a second" title.

On the flip side of the coin, I have been planning to add frametimes for some time now. It adds an interesting dynamic to the testing. Basically, instead of using AA and non AA tests, I'll likely be only using AA and adding frametimes. But we will see if that actually happens since the tools available don't measure up IMO.
__________________

Last edited by SKYMTL; September 18, 2011 at 12:02 PM.
Reply With Quote
  #7 (permalink)  
Old September 18, 2011, 04:20 PM
r0bc's Avatar
Allstar
 
Join Date: Apr 2009
Location: AB
Posts: 688

My System Specs

Default

Quote:
Originally Posted by SKYMTL View Post
This is a FEATURE within the game (rather than a GPU issue) that is being used more and more (Dues Ex: HR, Dragon Age 2, etc) to provide a seamless gaming experience without load screens.
I actually noticed this, wandered what it was.
Thanks for your reply, was curious what you thought about the article.
__________________

Reply With Quote
Reply


Thread Tools
Display Modes

Similar Threads
Thread Thread Starter Forum Replies Last Post
Benchmarking Suite? Reyzac Overclocking, Tweaking and Benchmarking 3 October 11, 2010 08:51 PM
Fermi benchmarking controversy MrDeodorant Video Cards 9 March 4, 2010 04:08 PM
Benchmarking without an OS Cptn Vortex Overclocking, Tweaking and Benchmarking 6 February 5, 2008 09:34 PM