A new look at game benchmarking
View Single Post
September 18, 2011, 11:38 AM
HardwareCanuck Review Editor
Join Date: Feb 2007
I read through the whole thing twice the other day. Basically, I agree with some of what he says and don't agree with other parts. I won't go into too much detail but there are some very important point that everyone should take away from the article:
- Microstuttering is only APPARENT to the viewer in very few situations. Scott mentions this a few times and I agree.
- There is no way to conclusively test for microstutter. The polling rate in FRAPS and even PlayClaw isn't frequent enough to accurately test for it.
- Due to the two issues above, there is absolutely no way to really test for user-viewable microstutter. Some people may experience it while others won't.
- Like it or not, a playable game experience is still defined by the 30-60FPS "rule". Below 30FPS for most games and below 60fps for fast paced games and the player will experience begin to suffer. So the standard type of benchmark is still perfectly valid.
- Frametime as he discussed (first example on the first page) can be completely invalidated by items that don't include the GPU's ability to render a scene. He showed frametime increasing exponentially for a single frame and deemed that unplayable if it happened often. I disagree. A good example of this is the Witcher 2. In order to provide a fluid area to area experience, the game preloads upcoming areas which will cause frametime to spike and cause a "stutter" every few minutes. This is a FEATURE within the game (rather than a GPU issue) that is being used more and more (Dues Ex: HR, Dragon Age 2, etc) to provide a seamless gaming experience without load screens.
One thing I didn't see is what software he used. Without that, no one can repeat and validate his results...and THAT is concerning. Methinks it is FRAPS but PlayClaw can achieve the same....and with slightly different results.
In short, the article was interesting but regardless of the hyperbole, we won't be including additional testing until there is a tool available that can poll framerates and sync issues at very quick intervals. Hence his "inside a second" title.
On the flip side of the coin, I have been planning to add frametimes for some time now. It adds an interesting dynamic to the testing. Basically, instead of using AA and non AA tests, I'll likely be only using AA and adding frametimes. But we will see if that actually happens since the tools available don't measure up IMO.
Last edited by SKYMTL; September 18, 2011 at
My System Specs
View Public Profile
Send a private message to SKYMTL
Find all posts by SKYMTL