Multi-threaded compression on multiple boxes

Kalium · Post by **Kalium** » Sun Apr 23, 2006 2:06 pm

The total job looks like this:

Job A: Partition the input. Due to the difficulty in determining scene changes, this is a non-trivial task, but not impossible. It is serial in nature, or at least appears to be and probably would not parallelize well.

Job B: Encoding each chunk. This job is very serial. However, there are many chunks, and they can be encoded in parallel. Note that the number of possible simultaneous encodes is limited by the number of chunks.

Basically, Job A is preprocessing for Job B. Both are serial, there are just a number of Job Bs that can be parallel processed. Assuming that all chunks are completely independant, I don't see why this wouldn't be possible. It just strikes me as overkill.

DJ_Izumi · Post by **DJ_Izumi** » Sun Apr 23, 2006 2:08 pm

I've spent 2hrs encoding a 3pass h264 encode of a 2min and 30 second video... (Athlon 2500 XP+) I can see where having a few extra computers on one's local network help out would be appreciated.

Kalium · Post by **Kalium** » Sun Apr 23, 2006 2:10 pm

I just don't think the required investment would pay off well. Video encoding is very intensive on both CPU and RAM, so a good (read: expensive) computer is called for.

Plus, you'd have to deal with all the problems introduced by the network, and you'd have to come up with software to do this with.