WIP [PROJECT]: I ain't gonna work on DT's farm no more

[Dead Things]

I suppose now is as good a time as any to end the speculation regarding Quicksilver, now that the cat is pretty well out of the bag, so without further ado...

Build: Quicksilver

Quicksilver is what the cool kids call mercury. Mercury truly is a fascinating material. It is the only metal whose natural form is liquid in standard conditions for temperature and pressure. But at the same time, it has one of the narrowest temperature ranges for maintaining a liquid state of any metal.

Why does any of this matter? Well, the chemistry buffs among you might recall that mercury has an atomic weight of 80. Another interesting factoid about mercury is that the largest atmospheric deposit of mercury in North American history dating back hundreds of years occurred with the eruption of Mount St. Helens in - you guessed it - 1980. So mercury has a bit of a connection to the number 80 it appears, which makes it an apt basis for the naming of this build...

Motherboard: Supermicro X8QBE-F
CPU: 4 x Xeon Q4GL
CPU Heatsink: 4 x Supermicro SNK-P0045P
RAM: 16x2GB Kingston DDR3-1333 Registered w/Parity ECC RAM
GPU: Integrated
Case: The cardboard box it comes in!
PSU: OCZ ZX-Series 1000W 80+ Gold
HDD: WD 640GB Black
Optical: Pioneer DVD-RW

New stuff in red, as always. The Xeon Q4GL is an A1 revision of the Intel Xeon E7-8860, a 10-core / 20-thread processor at 2.26GHz, although with Turbo enabled, its functional frequency will be 2.4GHz when fully loaded. The X8QBE-F - indeed one of the only non-OEM boards available for socket LGA1567 - is a four socket board, hence the machine will have a total of 40 cores / 80 threads. And there's that number 80. Hence the name, Quicksilver.

I really am defenseless to the inevitable question, "WTF?" I have no legitimate use for this machine outside of folding. But I got such a good deal on the CPUs, I literally couldn't turn down the opportunity. I ended up getting all four for a little over half the retail cost of a single E7-8860.

But the motherboard... whoa the motherboard. It is one expensive sonofabitch! Funny story about the motherboard actually: I ordered it from NCIX (pm'd of course) a week or so ago, and just this evening I got a call from Chris who says something along the lines of, "Hey so Supermicro called me and asked 'Is this guy sure he really wants this board' so... are you sure you really want this board?" I guess it's not a terribly popular platform! But kudos to NCIX for giving me an out like that. Anytime you spend $2000 on a motherboard, there's a reasonable chance that buyer's remorse will set in. But no, I convinced Chris that I was of sound (enough) mind to go through with it.

What else is there to say? Oh yeah - well, there are some decent-looking Dynatron heatsinks (the Dynatron Q3 to be precise) that I almost ordered but realized the protrusion of the heatpipes means that it wouldn't fit on the Supermicro board considering how tightly the sockets are packed together. Kinda retarded, if you ask me, considering that Supermicro is the ONLY non-OEM making boards for LGA1567. You'd think somebody making LGA1567 heatsinks would make sure they fit on the only boards available. So I got stuck ordering Supermicro's own heatsinks which unfortunately only come in a passive flavour. And since the board won't be in a server case with 18 tiny 8000RPM fans pointing every which way, I'm going to have to come up with a creative way to get good airflow over those heatsinks. My plan right now is to use 3600RPM Silverstone FM83 80mm fans for the two outermost sockets and a single 3000RPM Scythe Ultra Kaze for the two middle sockets. Provided my estimated measurements of the board are correct, it should work - but we'll just have to wait and see.

So yeah, the only thing left to report is that we have some more parts trickling in. I'll begin with the power supply. It's an OCZ, but hear me out. This is the ZX-series made by Great Wall, not the Z-series from a couple years ago made by Sirtec. It's a strong design and earned a 9.5/10 from jonnyGURU. I've been kinda wanting to give one a test drive since April, and now my opportunity has come! I've never owned any Ultras or had any other fully modular PSUs before, so it's kinda weird seeing a PSU that is completely cable-less (also weird to see it shrink-wrapped).



Another arrival of note is two of the 16 sticks of RAM for Quicksilver. I ordered them from Buy.com Canada, and originally I thought "Geez, what's the point?" But now I'm happy they sent me two sticks ahead of time instead of waiting to send me the whole order at once because it means I'll be able to test the board when it comes and make sure everything is copacetic. Also in the picture is a G3/8" to G1/4" threaded adapter for Suite300.



So that's all there is in terms of updates for today. Unfortunately I remembered that I ran out of thermal paste and need more before I can really get anywhere with Shoebox - but I made a rookie mistake and ordered it with the Supermicro board, which means it will be delayed. So I doubt I'll have any real updates to report until which time as Quicksilver's board arrives.

Thanks for reading everybody and I hope some of you are finding this at least mildly interesting. And stoanee - your question about eating and sleeping is actually a very good one. I've been working so much over the past couple of months on both real work and the farm rebuild, that there really isn't time to make the kind of progress I need to while maintaining my usual sleeping patterns. But I feel like I've discovered a sort of superpower. I find that if I take a 30-minute nap after putting the kids to bed, I'm able to keep going until 3 or 4 in the morning. I then sleep until 8 or 9 and waltz into the office by around 10-ish (luckily I have a pretty relaxed work environment in this regard) and amazingly, I'm good to go. It all hinges on that nap though. If I try to stay up without the nap, then I'm an absolute puddle of incoherent splooge the next day.

[Dead Things]

This is what the Supermicro X8QBE-F looks like, by the way:

[lowfat]

What kind of numbers do you expect to pull from that thing?

I just looked up the price of that motherboard. That is some serious funds right there.

[3.0charlie]

Motherfolder. Good Lord.

But I must raise the question about those 80 threads: I remember reading a thread over at [H] that some guy had access to a quad Beckton setup, and he never managed to fold properly with 64 threads. Mind you, his replies did show his folding noobness...

edit: here's the thread - need help with setting up offline folding - need smp! got 64 x7560 cores sat idle :-) - [H]ard|Forum.

[stoanee]

Originally Posted by Dead Things

Thanks for reading everybody and I hope some of you are finding this at least mildly interesting. And stoanee - your question about eating and sleeping is actually a very good one. I've been working so much over the past couple of months on both real work and the farm rebuild, that there really isn't time to make the kind of progress I need to while maintaining my usual sleeping patterns. But I feel like I've discovered a sort of superpower. I find that if I take a 30-minute nap after putting the kids to bed, I'm able to keep going until 3 or 4 in the morning. I then sleep until 8 or 9 and waltz into the office by around 10-ish (luckily I have a pretty relaxed work environment in this regard) and amazingly, I'm good to go. It all hinges on that nap though. If I try to stay up without the nap, then I'm an absolute puddle of incoherent splooge the next day.

Wow, a dancing banana for you sir!

[Shadowmeph]

what kind of cpu coolers do you use on the X8QBE-F I sure wish I had the same access as you to all of this parts man oh man

[PerryC]

Originally Posted by Shadowmeph

I sure wish I had the same access as you to all of this parts man oh man

+1^1000 to what he said.

Is that 4 x 8 pin cpu connectors on that board?

[Dead Things]

Originally Posted by lowfat

What kind of numbers do you expect to pull from that thing?

I just looked up the price of that motherboard. That is some serious funds right there.

Honestly, I have no idea. It all depends on how the machine answers 3.0charlie's question below. Best case scenario, though, I'm hoping for something around 600-700,000 PPD on P6903/P6904.

But yeah - that mobo was hella expensive. Crazy how limited the options are for LGA1567. It's like the socket that was... but wasn't.

Originally Posted by 3.0charlie

Motherfolder. Good Lord.

But I must raise the question about those 80 threads: I remember reading a thread over at [H] that some guy had access to a quad Beckton setup, and he never managed to fold properly with 64 threads. Mind you, his replies did show his folding noobness...

edit: here's the thread - need help with setting up offline folding - need smp! got 64 x7560 cores sat idle :-) - [H]ard|Forum.

Didn't read much of the thread, but there are two reasons for me to be optimistic that I should be able to circumvent the problems he was having: it will run on Linux and it will run on Linux. I explain...

1. AFAIK, Windows hadsa problem with >64 threads and scales poorly at the higher end of that range anyway. I remember reading that MS provided a bandaid solution to that problem with Server 2008, but without knowing anything about it, I doubt it would work well with a progressive/symmetric process like fah. Linux has no such threadcount limitation. I do not know how it scales past 64 threads (or even 48 threads), but I guess we're about to find out.

2. The Windows fah client uses 32-bit addressing, which means that even if it could scale well past 64 processing threads it would suffer from memory bandwidth issues that begin to affect anything over probably around 32 threads, really. By the time you get to 64, you're probably stuck running lots of empty cycles because the process will really get choked off by a lack of memory addresses. In addition, there's a chance you could completely run out of addresses over the course of a WU, ditching it near completion. And when that happens, your PPD is zero. The Linux client, meanwhile, uses native 64-bit addressing, so every SMP thread will be well-fed.

But this is all a grand experiment. Theoretically, I think it should work well. In the real world, who knows? The worst-case scenario I'm envisioning is having to run a pair of 40-thread clients, which oughtn't be such a bad thing either.

Originally Posted by stoanee

Wow, a dancing banana for you sir!

Thanks! This guy for you!

Originally Posted by Shadowmeph

what kind of cpu coolers do you use on the X8QBE-F I sure wish I had the same access as you to all of this parts man oh man

Passive Supermicro 2U heatsinks. They look like this:



Certainly not ideal, but I'm hopeful that putting some buzzsaw fans on them will keep the 130W TDP processors in check. If not, I'm going to have to get creative.

Any doofus with a credit card has access to the same parts I do. The key part, though, is being enough of a doofus to do it.

Originally Posted by PerryC

+1^1000 to what he said.

Is that 4 x 8 pin cpu connectors on that board?

Yeah - 4 x 8pin CPU connectors. Clearly a board designed for redundant power supplies. Clearly not the solution I will be attempting. Therefore it was important to select a PSU with a single 12v rail so that I could confidently use adapters knowing the power is coming from the same place regardless. So I'll be using the two modular EPS connectors the PSU comes with in addition to a pair of 6pin PCIe to 8pin EPS adapters. The PSU boasts 83a on the 12v rail, which means my load will likely be somewhere around the 60% mark for optimal efficiency on the 80+ Gold rated OCZ.


...that's if it works.

[lowfat]

Are those 45nm Nehalem CPU's?

[Dead Things]

32nm Westmere-EX