WIP [PROJECT]: I ain't gonna work on DT's farm no more

[Dead Things]

The title of this thread is ironic.

I have been planning to overhaul my farm for a couple months now and am at the point now where I am ready to start turning plans into action. So I figured I would post a worklog of my efforts in case anybody was interested. Part of the overhaul will include putting four machines under water, so hopefully there will be some pretty pictures along the way. Anyway, this is just sort of an introductory post considering I just now decided to do this, so I don't really have much more to say right now. I suppose I'll spend the next few posts detailing the specs of the various rigs, but please feel free to let me know if there's anything in particular you'd like to see or discuss in greater detail. For now, though, I will leave you with this:

‪Bob Dylan - Maggie's Farm (Live at Farm Aid 1985)‬‏ - YouTube

[Dead Things]

Build: Leviathan

Many of you are probably already familiar with Leviathan. Here is my Leviathan build log, for those of you interested in catching up. Anyway, Leviathan is a nice easy place to start since the only thing going on here is a CPU upgrade. I'll be moving from quad Opteron 8356's to quad Opteron 8439SE's. So that means going from 16 to 24 cores, and from 2.3GHz per core to 2.8GHz per core. There are also other efficiencies associated with the 45nm manufacturing process of Instanbul processors compared to 65nm Barcelona's, so all-in-all, it should be a pretty worthwhile upgrade.

This is something that I have been planning for a long time, but the price of Istanbul processors remained high, preventing me from moving forward with it. Thanks to Magny Cours, and everybody's bated breath regarding Bulldozer, though, Istanbul finally started plunging in price over the past couple of months. I wound up paying just under $800 for all four 8439SE's, whereas they would have cost $800 each as recently as March on the second-hand market.

Anyway, the new processors are in transit, so I will update this thread with pictures and results once I get them in. Leviathan is a pure folding box. When I first put it together, I toyed with the idea of using it for my web server, but decided against that early on.

For now, with the 8356's, Leviathan posts the following PPD numbers:

P6904 = 95,000 PPD
P6903 = 85,000 PPD
P6900 = 55,000 PPD
P2684 = 45,000 PPD

Can't wait to see how much those numbers improve with the new goodies.

Anyway, to cap off the introductory post to Leviathan, here are the planned system specs. New/upgraded items are indicated in red.

Motherboard: Tyan S4985
CPU: 4 x Opteron 8439SE
CPU Heatsink: 4 x Coolermaster TX-3
RAM: 8x2GB Micron DDR2-667 Registered ECC RAM
GPU: Integrated
Case: Coolermaster HAF932
PSU: Rosewill RBR1000-M 1000w 80+ Bronze
HDD: Seagate 7200.10 320GB
Optical: LG DVD-RW

[Dead Things]

Build: Ziz

Ziz is Leviathan's little brother. Almost the exact same computer, just with lesser processors. Originally, Leviathan had Opteron 8356's to Ziz's 8354's. After the upgrade, Leviathan will have 8439SE's to Ziz's 8393SE's. Opteron 8393SE's are Shanghai quad core processors on a 45nm manufacturing process. So while Ziz will remain a 16-core machine, its frequency per core will improve from 2.2GHz to 3.1GHz plus additional processing efficiencies related to the die shrink.

I'm cheating a bit here because, well, this upgrade has actually already been done. (Anybody want to buy some Opteron 8354's? ) I'm going to be putting on same fan grills over the next couple of days since I have a tendency to do this to myself:



So when I open it up to put on the grills, I'll be sure to snap a few photos.

The significance of Ziz was that I took a bit of a risk with the processors. Technically, S4985 only supports 65nm Barcelona and the low-end 45nm Shanghai processors with HT1. But I had read all about how the S4985 could be upgraded to the more expensive S4985-SI with a simple BIOS flash, making it compatible with 45nm Shanghai and Islanbul processors with HT3. I had an opportunity to scoop up some 8393SE's cheap locally (paid $400 for all four), so took the plunge. I'm happy to report that the rumours are true and my 8393SE's have been working wonderfully in the vanilla S4985. How wonderfully you ask? How about world record 16x wPrime 32M and world record 16x wPrime 1024M wonderfully? With 16 core Bulldozers due out soon, those records won't stand for very long - but hey I'm enjoying the infamy while I still can.

Like Leviathan, Ziz is a pure folding box. So while wPrime numbers are fun and stuff, all that really matters is how it folds, well, here are the PPD results so far:

P6904 = 155,000 PPD
P6903 = 145,000 PPD
P6900 = 95,000 PPD
P2684 = 85,000 PPD

Obviously, if you contrast those numbers with Leviathan's above and take into consideration that the old Opteron 8354's are 100MHz per core slower than Leviathan's 8356's - then that represents a significant improvement.

So, I will end off here with a run-down of Ziz's post-makeover specs. Again, red denotes the new stuff...

Motherboard: Tyan S4985
CPU: 4 x Opteron 8393SE
CPU Heatsink: 4 x Coolermaster TX-3
RAM: 8x2GB Samsung DDR2-667 Registered ECC RAM
GPU: Integrated
Case: Coolermaster HAF932
PSU: Seasonic SS-850HT 850w 80+ Silver
HDD: Seagate 7200.10 320GB
Optical: LG DVD-RW

[Dead Things]

Build: BOSServer

BOSServer is my VM server. I use it as a media server, file server, encoding box, pfSense box, and occasionally for dev sandboxing when I feel like goofing around with my latest whim. And, of course, folding. This build is still largely undecided. I have flip-flopped at least three or four times on what I want to put into it. Basically, the hardware features I want this box to have include:

- ECC RAM
- compatible with LSI 9260-8i
- potentially a CUDA-enabled GPU, but I'm undecided on whether or not I want this

Recently, I helped a friend of mine test out 18 Xeon engineering samples that he acquired in bulk from - well, I don't really know where. Just to see which ones were working, etc. So to do this, I bought Jody's butchered RIIIE that he used for benchmarking:



As compensation for performing this task, I took my payment in the form of 3 x Q3QS processors, which are essentially the same thing as a Xeon E5649, except with a 95w TDP instead of 80w like the retail version. So I was thinking - hey, I could use the RIIIE with one of my Q3QS's and fill it with 12GB of ECC unbuffered DDR3-1333 RAM. I already knew the RIIIE was compatible with the LSI 9260-8i, so the only question left was - does it support ECC RAM?

Well, turns out it doesn't really. Oh, it will use ECC unbuffered RAM just fine - but there is no ECC functionality in the RIIIE's BIOS. I can get a $150 AM3 mobo from ASUS with full ECC support, but the RIIIE is sorely lacking in this regard, I'm afraid. Since the RIIIE so wonderfully met my other desires for this build, I must admit that I am still considering proceeding with it as planned despite the lack of ECC functionality.

You know that $150 AM3 mobo from ASUS will full ECC support I mentioned earlier? Well, that's one of the other options I'm considering. The idea would be to pair an ASUS M5A99X Evo with a Phenom II X4 905E for now, then potentially upgrade to whatever high-efficiency version of Bulldozer is available later on. This option would provide ECC functionality, which I like. It would limit me to 8GB of RAM, though, which is less than I would want for my VM server, unless I were to buy an additional 4x4GB of RAM. But since I already have 6x2GB of ECC RAM sitting around, I don't really want to do that. Lastly, support for the LSI 9260-8i remains a mystery, although I am relatively confident it would work given ASUS's track record with 3ware and LSI controllers.

My last option is the one I am leaning towards at this time. It would involve pulling my 2 x E5530's out of retirement and pairing them with an ASUS Z8NA-D6 board. I already have one such board, but it is earmarked for another build (see Twofour below, once I get around to it). So I could buy the cheaper Z8NA-D6C for Twofour and use the Z8NA-D6 for BOSServer. I already have the case, the RAM, the processors, and everything else - so from a cost perspective, it would be roughly the same price to do this as it would be to go the AM3 route. The RIIIE option - although "free" since it would mean incurring no additional costs - is less desirable than these two options owing to the lack of ECC support.

So - what do you guys think? Is ECC as useful a feature as I think it is for a VM server that will be running a file server using the XFS file system? If so - should I opt for the AM3 option or the dual-LGA1366 option given that cost is a wash?

Oh, and one more thing: BOSServer will be going under water no matter what!

edit - Okay, I've made an executive decision, I am going with the Z8NA-D6C(MIO) option. So here are the planned system specs...

Motherboard: ASUS Z8NA-D6
CPU: 2 x Xeon E5530
CPU Heatsink: Custom water loop*
RAM: 6x2GB Kingston DDR3-1333 ECC Unbuffered RAM
GPU: Integrated OR GTX 260 OR 8800GTS (Undecided)
Case: Coolermaster Cosmos S
PSU: Rosewill Lightning 800w 80+ Gold
HDD: OCZ Vertex 60GB and LSI 9260-8i RAID5 array with 7 x Hitachi 7K1000.C
Optical: Pioneer DVD-RW

*Watercooling loop:
Pump: Jingway DP1200
CPU block: 2 x Swiftech Apogee GT
GPU block: Depends
Reservoir: Danger Den Single Bay reservoir
Radiator: 1 x HWLabs Black Ice GT Stealth 360
Fittings: Bitspower silver compressions of various types
Tubing: Feser acid green 3/8" ID 1/2" OD

[Dead Things]

Build: Suite 300

Suite 300 is my main rig. I use it for - well, general usage kinda stuff. That means surfing the web, checking email, watching pr0n, playing video games, and one other thing - can't remember what that is - oh yeah, folding.

So since this is my main rig, and since I am a vain mofo, I like it to look pretty. With this in mind, the key feature for Suite 300 will be its aesthetics. I don't want to pour too much processing power into it since it will be folding less than most of my other rigs. Also, since I use it for games I will be going all-in with the GPUs. So if I plan on using it for GPU folding (which I do), then it behooves me not to put a processor capable of bigadv folding into it so as not to cannibalize my own production.

This rig is essentially finalized in terms of planning. I still have to buy watercooling fittings for it, but I'm waiting for FrozenQ to start shipping his new dual-bay reservoirs with the integrated D5 pump tops so that I can just do one order for this build from dazmode. I have always been a fan of FrozenQ's work, but since I am addicted to bay reservoirs, I've never had a chance to use his stuff before. Until now. Look at this beauty:



I can't wait! The colour theme is going to be red and black, even though the mobo (an ASUS P7P55 WS) is predominantly black and blue. Meh, I'm not crazy picky. Besides, I'll be loading that mobo up with enough components that most of the blue will be concealed.

The one question that remains about this build is: will my planned mobo support my planned HBA? I am hoping to stick an LSI 9211-4i on an ASUS P7P55 WS. I don't have the answer yet, but I will soon!

So, here are the planned specs for Suite 300. New stuff in red.

Motherboard: ASUS P7P55 WS
CPU: Core i7 875k
CPU Heatsink: Custom water loop*
RAM: 2x4GB GSkill DDR3-1600 CL8
GPU: 2 x 9800GX2 in quad-SLI plus GTX260 for PhysX
Case: Coolermaster ATCS840 with side window panel
PSU: Silverstone Strider 1000 Plus 1000w 80+ Silver
HDD: 2 x Corsair Force 3 120GB in RAID0 and 2 x Samsung F1 750GB in RAID0 on an LSI 9211-4i
Optical: Samsung DVD-RW with... wait for it... Lightscribe!

*Watercooling loop:
Pump: Swiftech MCP-655
CPU block: EK Supreme HF
GPU block: 2 x Koolance VID398-GX2
Reservoir: FrozenQ Dual Bay reservoir with integrate D5 pump top
Radiator: 2 x Thermochill PA120.2 (one with 15mm fan spacing, the other with old school 25mm fan spacing, both with G3/8" fittings)
Fittings: Bitspower matte black compressions of various types
Tubing: Primoflex red 1/2" ID 3/4" OD

[Dead Things]

Build: Twofour

Now that the planned specs for BOSServer have been finalized, so too have the specs for Twofour. Twofour will make use of the new ASUS Z8NA-D6C so that my existing Z8NA-D6 (no C) can be used in BOSServer. What's the difference between the -C and non-C variants? Not much really. The non-C variant has 8 onboard SAS ports which are activated by a controller in the ASUS PIKE slot. I don't know if PIKE is a proprietary connection, but I've not seen it on anything except ASUS boards. Here's the kicker about PIKE, though... the only hardware RAID option for the PIKE slot is the ASUS PIKE 1078E - but there is no BBU available for that unit. The 1068E is software RAID only and does not do parity RAID. For software-based parity RAID, your only option is the ASUS PIKE 6480, which is EOL. I have one and must admit that I do not recommend it. The 6480 is based on a Marvell 88SE6480 controller and suffers from both poor performance and poor reliability.

But that's neither here nor there when it comes to Twofour. All the RAID-relevant stuff will be in BOSServer; Twofour is just a folding box now. In terms of the CPUs for Twofour, I have a pair of Intel Xeon E5645's and a pair of Q3QS's (basically Xeon E5649's) that I can use. I'm not sure which ones will go in Twofour yet. It sort of depends on if my SR-2 (in Shoebox below) prefers the Q3QS's or the E5645's.

Cooling will not be a problem here. Even though there will be a total of 12 processing cores (and 24 threads) jammed into an ATX form factor, the Z8NA-D6C does not offer any overclocking functionality at all. Not even software overclocks using utilities such as SetFSB or CPU-Tweaker exist for this platform. Therefore, at bone stock, these chips won't be kicking up too much heat. Nevertheless, I want to keep it nice and cool, so I will be using my Corsair H70's in this rig. Combined with the high-airflow HAF922, I don't anticipate any heat-related issues at all, which is good news for 24/7 folding in the dog days of summer.

Anyway, without further delay, here are the planned specs for Twofour (keeping in mind the CPU choice is yet to be finalized)...

Motherboard: ASUS Z8NA-D6C
CPU: 2 x Intel Xeon E5645
CPU Heatsink: 2 x Corsair H70
RAM: 6x2GB Mushkin Silverline DDR3-1333 CL9 RAM
GPU: Integrated
Case: Coolermaster HAF922
PSU: Antec TPQ 1000w 80+ Bronze
HDD: WD 640GB Black
Optical: Pioneer DVD-RW

[Dead Things]

Build: Shoebox

Shoebox is my SR-2 rig. Like the title of this thread, it is an ironic name due to the case I am planning to use for this rig. Shoebox is one of the planned watercooling projects. The defining characteristic of Shoebox is its motherboard, as it will be an EVGA SR-2. When I first got my SR-2, I had so much fun playing with it that it was literally months before it folded its first work unit. Instead, I spent the better part of the winter benchmarking with it (and my Xeon X5677's) and earned quite a number of boints and cups in the process.

But that was just a phase, and I've grown out of it. The SR-2's glory days as a benchmarking diva are behind it, and now it's destiny is to crunch. But just because its a cruncher doesn't mean it can't look pretty, and so I am planning an orange, white and black-themed watercooling loop for Shoebox. Besides, DP and MP watercooling builds are totally radical.

So the plans for Shoebox at this point are as follows. Shoebox may end up swapping CPUs with Twofour above, and other aspects of the build are subject to change as well considering the case is so highly allocated that it's difficult to find anywhere without a $45 shipping charge (I'm lookin' at you, Newegg ).

Motherboard: EVGA SR-2
CPU: 2 x Xeon Q3QS
CPU Heatsink: Custom water loop*
RAM: 6x2GB GSkill DDR3-2000 RAM
GPU: EVGA 8600GT
Case: Xigmatek Elysium
PSU: Silverstone Strider Plus 1000w 80+ Silver
HDD: WD 500GB Blue
Optical: Pioneer DVD-RW

*Watercooling loop:
Pump: Danger Den CPX-Pro
CPU block: 2 x Swiftech Apogee GTZ
GPU block: None
Reservoir: Danger Den Dual Bay reservoir
Radiator: 1 x Thermochill PA120.3 (25mm fan spacing, G3/8" fittings)
Fittings: Bitspower matte black compressions of various types
Tubing: Feser orange 1/2" ID 3/4" OD

[Dead Things]

Build: Rexy

Well, I guess that leaves enough hardware for one extra build. We'll call this one Rexy, just like Caillou's favourite pet dinosaur. What, you don't all watch Caillou every morning at 7:30am? What's wrong with you? Here's a taste of what you're missing:

‪CAILLOU LOST IN THE JUNGLE‬‏ - YouTube

Anyway, remember that RIIIE that I used to test the engineering samples and briefly considered using in BOSServer? Well, it remains unaccounted for, so my options are to try to flip it or use it. And since I already have all the hardware necessary to use it, that's just what I'll do. I have one extra Q3QS, so it will go in there. The big question here is how to cool it. I have a Coolermaster Hyper 212 and a Zalman CNPS10X Flex I could use - and both are very adept heatsinks. But, both are a bit boring. The RIIIE is a neat board insomuch as it has both LGA1366 mounts and LGA775 mounts for the CPU heatsink, and it is the latter that has got me thinking...

I have an old Swiftech Apogee Drive. Remember those? Before the current all-in-one LCS craze, the Apogee Drive was a trailblazer, integrating the pump (MCP-350) and CPU block into one unit. It's a bit of a dinosaur now, especially since I only have the LGA775 hardware for it. But since Rexy a) has LGA775 mounts and b) is a dinosaur, it seems like a match made in heaven! So I'm going to try to keep this loop as simple as possible - especially since I will be attempting to cram it into an Antec 900, notoriously one of the least watercooling-friendly cases of all time. Wish me luck!

Planned specs:

Motherboard: ASUS Rampage III Extreme
CPU: Xeon Q3QS
CPU Heatsink: Custom water loop*
RAM: 3x2GB Patriot DDR3-1333 CL9 RAM
GPU: XFX 8800GS
Case: Antec Nine Hundred
PSU: Tagan BZ 800w
HDD: Maxtor 250GB
Optical: Samsung DVD-RW

*Watercooling loop:
Pump: Swiftech Apogee Drive
CPU block: Swiftech Apogee Drive
GPU block: None
Reservoir: Probably just going to use a T-line
Radiator: 1 x HWLabs Black Ice GT Stealth 240
Fittings: Bitspower silver barbs of various types
Tubing: Primoflex green 3/8" ID 5/8" OD

[ZZLEE]

You Need your own Water fall and generator. :)

lots of fun planned.

[Zygote]

Hey DT, for your BOSServer, what thermal paste/adhesive did you use to stick those ram sinks on your vrms on your ASUS Z8NA-D6?