Render Farm Build for Twilight and Kerkythea

[bradleyabbott82]

architectural animations/ walkthroughs ?
8 threads will not do it for you. You will need to look into that dual xeon workstation at bare minimum. Better, you will need to look at something like what Bradley did here:
http://twilightrender.com/phpBB3/viewto ... 068#p23068
I'm sure he'd be glad to tell you all about it.

for still shots, I've created probably more than 400 professional level still renderings in 2 years on a machine that was under $1000. (i7920, middle-of-the-road graphics card).

If you are on a tight budget, the 2600k overclocked to 4.something will serve you well - do much research before overclocking as it's quite technical. You can buy machines from a professional fully overclocked for you. It may be worth it. Or perhaps you have a smart geek/friend.

Twilight Render is not responsible for anyone overclocking their machine. Do so at your own risk.

I am Currently building (should be done today and operational Tomorrow) a 5 Machine Net Render Farm. Specs:

-Intel Core i7-2600K Overclocked to 4.5Ghz x 8threads (180Ghz Total Processing Power)
-8Gb PC1600 DDR3 Dual Channel Ram (more than enough for rendering)
-Win7 64 Bit
-KT x64 Boost
-Water Cooled Processor with Custom Radiator/Fan Setup (during Testing I'm getting ~60c Temps at full render load)

All 5 Machines are Controlled and Contribute to Biased Frame Renders as well as Unbiased Net Rendering. I will post performance specs and some examples of rendering this weekend/next week after construction and they're all up in a 43U Server Rack.

This system will be setup on A Gigabit Cat6 Network, Each machine has a custom application for remote Launching/Saving so that I can literally upload my scene with a standard naming convention to a shared Network Folder (on the Host), Launch the App via the Network and each machine will automatically begin rendering/contributing with no direct human interaction on the individual machines. This will simplify the process to the extent that I basically have an automated Chain of processors acting as if it is all 1 computer. I'll post Specs/Pics/Performance soon.

I will likely be setting up an Automated Render Service where Forum users can Upload to a folder (via a web interface), have the scene automatically rendered, and emailed to them with no interaction on my part. I will be offering this as a service, and if it grows, I will double or triple the number of Servers in the Farm. TL Forum Members will be offered a discount. Imagine that I'm currently using a ~24Ghz machine that takes 12 hours to clear 150 passes (on a given model), theoretically I should be able to clear the same passes in about 1-2 hours. If this is a service the Forum Members are interested in, please contact me or post here with your input - so that I can gauge the level of interest.

Bradley

[bradleyabbott82]

All Machines Built... Going in the 42u Rack tomorrow... more to follow...

Bradley

[bradleyabbott82]

Getting Everything into the rack... Should be done tonight... Loading OS's and getting the automation apps/system perfected this weekend... Should be ready to render next week

28" LCD Display on 5in x 1out HDMI switch. All Machines output 1080p to display.

Comments Welcome...

Hoping to be stable around 4.8-5Ghz per thread on 40threads.... (200Ghz total render power)

Bradley

[Fletch]

Looking very cool, Bradley! We will certainly help promote any service you can provide for Twilight Render users.

[alvydas]

It seems very promising

[bradleyabbott82]

Okay Folks... The farm is up and running... here is a test on a model that took 60hours to get "relatively clear" on my 8thread corei-7 3.2Ghz machine a few weeks ago... The first image is 60hours on 1 box (from a few weeks ago), the second image is a conglomeration of 6 machines (3.8Ghz - 8 threads x 5 machines - 152Ghz total power) over a 40 minute render. I have not yet REALLY overclocked the machines (they will eventually be running 4.5Ghz each = 180Ghz), so this is a pretty good indication of how FAST this thing will be... The third image is the output of ONE of the clients (so you can see the SUBSTANTIAL clearing effected by a forced stop/commit @ 40minutes in the second image).

I'm having a bit of an issue with the KT net-render though - If I have the Network Log Window open during rendering, at the first attempted commit, the host CRASHES, gives the standard "KT crashed we saved your work" message, but the clients keep running. I just restarted the host, restarted the hosted render and forced a commit by stopping the clients and the contributed data was sent through (obviously since the second image posted here only ran for 60 seconds and is OBVIOUSLY a conglomerated contributed render from the clients)... Thoughts anyone? It seems if I never open the Network Log window (which means I cant see my commits), that I don't have a problem, but if I open it (or have it open), KT crashes during a commit... The COMMITS ARE HAPPENING - I JUST CAN'T OPEN THE LOG TO SEE OR KT CRASHES...

[Fletch]

That is a unique observation about getting commits when the log is not open, but opening to check and see causes a crash. You may be onto something.

KT's net render is unfortunately a bit quirky, and I can not provide much more info than that.

Testing clarity of an unbiased render vs. another unbiased render is also not very scientific as far as I can understand. You can only compare sizes of files when saved as .png or .jpg, the more the noise, the greater the file size. This will be more pronounced, I imagine, with .png files.

If not opening the log allows the commits to happen, I would somehow make a periodic saving mechanism for the net render, and avoid opening the log. Obviously this has many negative sides, but I do not have any tech. information for the KT net render feature.

[bradleyabbott82]

Yes, I'm aware of the "file size evaluation trick" - and I did test it here - the file size of the "net rendered conglomeration image" was "x" when saved after stopping all clients but 1, when I force stopped the render of the last client, then resaved the image (to see if the clients were submitting contributions), the file size reduced. So obviously the clients are contributing their data during a "forced stop"... however it would be nice to see the number of commits etc.... to evaluate (and ensure that) all clients are contributing as much as possible during the render process before a forced unbiased stop.

I was hoping to get (since right now I'm manually starting the host, then manually starting the client on each machine) a command reference for the KST file and a syntax/example of a script file to do the following (please provide a format/syntax suggestion for the .kst file to do the following):

line 1 open SCENENAME.XML
line 2 start render SCENENAME.XML with DEFAULT/SAVED settings (since I am saving the "client" settings in a seperate file from the host)
line 3 quit kerkythea (since I'm using a custom #pass setting, the render will finish, commit its render, then quit the app).

Anyone "got help"???? or "got milk"... since I now have cookies but no milk so to speak lol...

Bradley

[bradleyabbott82]

Here is a comparison for you...

The first image was rendered on a Core i7 3.2Ghz 8thread (25Ghz) for 63 HOURS - NO POST PRODUCTION

The Second Image was rendered on my new farm for 2.5hrs - I did LITERALLY 5 MINUTES OF POST-PRODUCTION (JUST A BASIC DENOIZE)... As you can see, 2.5hrs is BETTER (even in a LARGER IMAGE) than the first which rendered for 30x LONGER!

I AM NOW NEARLY 30X FASTER.... Once I overClock Each box to 4.5Ghz, I will be 25% faster than THIS!

Damn... It hurts a little bit

Bradley

[bradleyabbott82]

For those that are interested - this is the raw output at 2.5hrs of the render farm vs. a single box @ 60+hours..

Bradley