Review bottleneck testing.

[CMetaphor]

Hey all,

I was just wondering if in future reviews (especially for the latest and greatest hardware) if it would be possible to have a "bottleneck" section. Example: When the 4870x2 comes out, how fast of a processor do you need to prevent a bottleneck? Does your motherboard need PCIe 2.0 or not? Etc etc. This example pertains to me even, since I'm considering getting a 4870x2 when It comes out, and I want to make sure that my motherboard (Crosshair with PCIe 1.0) and processor (6000+) will not bottleneck the new card (meaning I'm not wasting my money on a newer, faster card). This section could be fairly easily added if the reviewer had on hand an extra processor or two that were slower (Ex: Regular tests are done with a Q6600, but will a E2180 run the same video card without a large performance difference?).

What does everyone else think?

[FiXT]

It sounds like an interesting concept, the only issue is that with a CPU bottleneck there really is no clear cut and defined line as to where it starts and/or ends. Everyone's interpretation of a bottleneck is going to be different. Frames are going to increase at a rather rapid pace initially, slowing down as the CPU gets faster, however there will always be an incline. The best way to show this would be via a curved graph showing the steady progress as the CPU gets faster and allow readers to determine what they would consider a bottleneck. That however, it really doesn't guarantee any answers, it merely lays out information that could have been approximated.

I'm going to ponder over this and see if there would be a way for it to be useful while still remaining clear and informative - Not requiring people to think or guesstimate the results, but actually show tangible numbers.

[SKYMTL]

With the review timeline, adding this to the GPU reviews is impossible. Forget about us wading into the PCI-E 2.0 / 1.1 bottleneck fiasco as well. That is set in stone.

Maybe in a seperate article we will look at CPU bottlenecking but I would have to look at the viabiliy of testing methodologies. If they do not produce repeatable results Hardware Canucks will NOT broach the subject.

[CMetaphor]

I could volunteer my system as a PCIe 1.0 benchmark system and underclock my cpu slower and slower to give a comparison? Not that Im trying to be greedy or anything, but just if it's a necessity. I think the good people of the forum should know whether or not their first gen PCie-based system (of which probably 50-60% or more of the forum has) can take advantage of a new video card or if a "lesser" newer card would offer the same performance without being as expensive. Either way... I'll help if i can (Im buying the 4870x2 when it comes out either way, lol).

[sswilson]

Originally Posted by CMetaphor

I could volunteer my system as a PCIe 1.0 benchmark system and underclock my cpu slower and slower to give a comparison? Not that Im trying to be greedy or anything, but just if it's a necessity. I think the good people of the forum should know whether or not their first gen PCie-based system (of which probably 50-60% or more of the forum has) can take advantage of a new video card or if a "lesser" newer card would offer the same performance without being as expensive. Either way... I'll help if i can (Im buying the 4870x2 when it comes out either way, lol).

Doing your own benchmarking once you get the card would be a great help to folks looking for that kind of information, but getting it done on a regular basis would be quite an undertaking for any hardware site to tackle.

Imagine the amount of hardware a reviewer would need to keep on hand in order to satisfy all of the different questions folks would start to expect answered... Where would a person draw the line? C2D vrs AMD? PCIe 1.0a vrs PCIe 2.0? DDR2 vrs DDR3? 8X vrs 16X? The list goes on and on, and even if one person had all of that hardware available, the time constraints on doing a normal review/benchmark would be quadrupled.

Keep in mind that a standard review of even the simplest item takes 30+ hours to put together.

[magictorch]

I think with a test of different cpu speeds with 4870X2 in PCIE1.0 or 2.0 you would need to control for confounding variables like differences among chipsets or the bios. That in itself will make a really accurate comparison very difficult.

The best case scenario is simply to test a range of PCIE 1 boards vs PCIE 2 boards at different cpu speeds and see where the results take you. Thats a lot of work though.

[CMetaphor]

I agree it's alot of work, but it answers questions that lots of people ask. The most important of which is "when is the right time to upgrade?" or "what do i buy for my system?" etc etc. It was just an idea tho.... If it ges voted down I'll stay silent and just tell you guys my FPS in games when i get my 4870x2.

[magictorch]

Originally Posted by CMetaphor

I agree it's alot of work, but it answers questions that lots of people ask. The most important of which is "when is the right time to upgrade?" or "what do i buy for my system?" etc etc. It was just an idea tho.... If it ges voted down I'll stay silent and just tell you guys my FPS in games when i get my 4870x2.

Then again who has the time or the hardware to cover all bases?

Your right, the consumer needs to know where they fit into the big picture. After all without the consumer where would these companies be? These are not cheap toys we buy.

The ever growing number of chipsets or GPUs and their partners plus the cpus and bios support make for confusing times for those looking to work out when or how to build a strong gaming rig.

Add to the mix the game developers and you have a mess at the cutting edge.

Makes you feel more like a testtube in an experiment with 30 variables and no degrees of freedom. Its about time things became more objective for the cutting edge perhaps in terms of reviews. This site is definitely a step in the right direction.

[FiXT]

Originally Posted by CMetaphor

I agree it's alot of work, but it answers questions that lots of people ask. The most important of which is "when is the right time to upgrade?" or "what do i buy for my system?" etc etc. It was just an idea tho.... If it ges voted down I'll stay silent and just tell you guys my FPS in games when i get my 4870x2.

Always speak up man!
Whether or not it gets done now, slated for the future or not done at all, we ALWAYS need idea for content. If we aren't able to show the way products get bottlenecked, maybe this idea will spawn a spin off that is a huge hit, but the only way to get that is through feedback and suggestions like yours.
People are going to "pick apart" or change the original idea and it morphs into something awesome. Don't be discouraged, keep those ideas coming

[ChuckleBrother]

Maybe it would be possible to run just a few of the tests on bottlenecked hardware? Take a few of the tests with the lowest score and test them with slower CPU(maybe 1 or 2 of the slowest results), just to get an idea of where the bottleneck affects our realworld performance. Other than that the only thing that could possibly work is running the reviews on a machine that resembles one you would find in the house of one of the forums members.

I was so happy when you did that review with the Q6600, has a lot more relevance to me than a review based around a machine that makes mine look like a kids toy.