Advanced Video Compression - Part 1
Various Codecs, Formats and their Pros and Cons

Now that you know the basic concepts and terminology behind video compression, we can talk about various formats and codecs, and which one is right for you. First off, we need to point out what the difference between a Codec and a Format is. Codec stands for Compressor/Decompressor, meaning it is something which can both encode and decode something, in this case video. When we talk about video format in this context, we're talking about how that information is stored on disks. Formats include things like AVI and Quicktime. Codecs include things like MJPEG, Sorenson, Cinepak, and DivX. Some Codecs are also formats (or have specific formats attached to them), such as MPEG and ASF.

A format does not necessarily mean anything about the video quality, it only dictates the underlying structure of a file. I always scratch my head when people say things like "I like AVIs better than MPEGs because they're smaller," which is a statement that makes absolutely no sense. I've had an AVI file that's 26GB large, orders of magnitude larger than most MPEG files. The fact is that formats like AVI and Quicktime have little to nothing to do with the underlying Codec, except that the limitations of a format must be adhered to in the codec. For instance, AVI does not support bidirectional-predicate frames, therefore the MPEG4 implementation that Microsoft made for AVI/ASF is incomplete as it does not implement this essential feature of the MPEG specification.

Please note that due to issues I have with both RealMedia as a format and RealNetworks as a company, I will not be covering information on their proprietary format. Their player is intrusive and obnoxious enough that I refuse to install it. Also, despite the fact that RMs nowadays can indeed have good video quality, their reputation of being small files with awful quality will make the average user avoid downloading files of that kind. Therefore, we do not recommend encoding files in RM regardless of how good they'll look since you'll be fighting an uphill battle to even get people to install the player software.

Also note that most of this information will be useless to those of you who are ripping DVDs and editing with those. This section is for codecs to use if you are capturing via analog or Firewire to obtain your footage. Now on to the various codecs that we'll be talking about. We'll be splitting the various codecs into two sections - one for codecs which are good when used for editing, and one for codecs that should be used for distribution purposes.

I. Common Editable Codecs

1) MJPEG - Motion Joint-Picture-Experts-Group Compression

Origins: All of you know what JPEG is, I'm sure. If you've surfed the web, then you've seen a JPEG image. JPEG was a compression scheme developed by the Joint Picture Experts Group for the compression of images. Now imagine instead of compressing single images as JPEGs, compressing 24 to 30 images a second and storing them in sequence. That's essentially what MJPEG is - it's the JPEG compression algorithm applied to frames of video.

How it Works: MJPEG is a lossy codec. JPEG uses what's called a psycho-optical algorithm, or a visual-perception algorithm. Basically, what this means is that JPEG exploits flaws and short comings in the human perceptual system. The human eye has a much harder time differentiating between slight differences in color than differences in brightness. Also, JPEG plays other various tricks, but what you wind up with is an image which looks about the same to the human eye.

Benefits: There's already lots of MJPEG hardware already out there, including very cheap pieces of hardware like the Pinnacle DC10, the Matrox Marvel series, and lots and LOTS of other pieces of hardware (scaling into the tens of thousands of dollars for the old high-end Matrox DigiSuite hardware). MJPEG can achieve decent compression rates and good image quality for most natural images like live video.

Disadvantages: Here's where MJPEG starts looking much less attractive. First off, JPEG does not compress anime well. Due to the way that JPEG works, abrupt changes in color and brightness (for instance, thin black lines in between different blocks of color) are not handled well by JPEG, which means MJPEG suffers from the same problem. JPEG has a problem with sharp edges which is unavoidable, partly because the standard was meant to compress natural images like scanned photographs.
There's also the problem of recompression. While the human eye can be fooled by JPEG's compression, computers aren't. Thus, when you compress from MJPEG into another format to use for distribution, you will receive a lower-quality picture than if you had a lossless source because there is data missing which, while it's not important to the visual cortex, is very important to a mathematical analysis of things such as color data.

Recommendations: MJPEG is an old standard. JPEG itself is very outdated - newer image compression formats are vastly superior (like PNG or the upcoming JPEG2000 standard which uses wavelet compression). There are much more refined and better Discrete Cosine Transform-based compression algorithms out there (like DV) that I would recommend over MJPEG. If you've got hardware that supports MJPEG and that's all, then by all means use it, but if you haven't bought hardware yet, I recommend going with DV equipment. The only time you should use MJPEG otherwise is for fast-editing temporary files, but DON'T use them as your final source.
2)

DV - The Best Consumer Digital Format Available Today

Origins: In 1997 and 1998, a consortium of 10 electronics companies (Matsushita Electric Industrial Corp (Panasonic), Sony, Victor Corporation of Japan (JVC), Philips Electronics, N.V., Sanyo Electric, Hitachi, Sharp, Thompson Multimedia (RCA), Mitsubishi Electric Corporation, and Toshiba) came together to form a standard for use in both consumer and professional digital video compression. DV was finalized in 1998, and first appeared in camcorders in 1999.
DV is often associated with IEEE-1394, more commonly known as FireWire or iLink. These two standards are closely intertwined but are not the same. DV is the standard that defines how video is compressed/decompressed and how it is stored on standard DV tape. IEEE-1394 defines a standard for a serial bus topology used in computer components. However, the two standards were codeveloped (and the way data is stored on DV tape is the same as the packet structure used across an IEEE-1394 bus), so much so that the first DV device was also the first IEEE-1394 device.

How it Works: DV is a lossy codec. I've heard lots and lots of misinformation floating around about how "DV is lossless," which it definitely is not. DV is a different approach towards a Discrete Cosine Transform algorithm than what was taken with MJPEG, partly because DV is a newer standard. I won't bore you with all the gory details, but lets just say that in every respect except one, the codec is technically superior. That one issue is that DV is fixed to 720x480 in 29.97 FPS (or 720x576 in 25 FPS) at 25MBit/second. That means very little flexibility if you want to do low-quality previews of your video or you're pressed for disk space (25MBit/second is 3.4MB/sec, or roughly 5 minutes per GigaByte!). Like MJPEG, DV is a good editing codec because it doesn't use any inter-frame compression, thus every frame is a "keyframe."

Benefits: DV looks very good. VERY good. Better than MJPEG in every respect. DV is good enough that it's used by both consumers and many professionals as a compression standard. Variants of the algorithm exist such as DVCAM and DVCPRO, which are essentially slight modifications that make them more durable when stored on tape, but the actual compression algorithm is identical. The only reason more pros don't use it is because it stores color in 4:1:1 sample rate (meaning there's 1/4 the chrominance samples as there are luminance samples). To make up for this, formats like Digital-S from JVC and DVCPRO50 from Panasonic use 2 DV codecs in parallel at 50MBit/second to store 4:2:2 color data. But none of that will really matter to you very much if only do a recompression once or twice.
DV is also cross-platform, meaning it works the same on PCs, Macs, AVID video editing workstations, you name it. You can also create a DV project, and then pipe it out through FireWire to DV tape and get a complete, lossless digital copy of your master as a backup.

Disadvantages: As stated above, it's rather restrictive in terms of framerate, resolution, and datarate. There are also issues editing DV in Adobe Premiere 5.1 - due to the way DV data is stored, depending on whether your FireWire device captures DV in "Type 1" or "Type 2" AVI files, you may be unable to open them in Premiere. Premiere 6.0 doesn't have this issue, and if you have a card like the DV500 which is meant to work with Premiere, its files will work fine.

Recommendations: Unless you have the disk space/enormous processing power/hard drive throughput to edit in Uncompressed or Huffyuv, this is the way to go.
3)

Huffyuv - Lossless Video Compression, for a price.

Origins: A guy named Ben Rudiak-Gould wrote this wonderful little codec which is a lossless compression codec for both YUV and RGB video data.

How it Works: Huffyuv is a lossless codec. The name stands for Huffman-compressed YUV. Huffman was a guy who came up with something called "Huffman entropy encoding" which is basically what all lossless compression is originally based off of (Huffman coding is used in things like ZIP, RAR, etc). So you could say that basically Huffyuv simply ZIPs every frame for its compression.

Benefits: Obviously, the biggest benefit is that you have a perfect recreation of the original video data (unless you do colorspace conversions). That means no matter how many times you recompress the video in Huffyuv, you'll still have the same video data that you had in the beginning. This is a very nice thing.

Disadvantages: Unfortunately, while compressing video in Huffyuv is pretty fast, decompressing it is not. Also, due to the very large space requirements (Huffyuv takes a LOT of disk space, sometimes 4x more than even DV), disk throughput becomes a very large factor. Playback at full resolution and full framerate is a daunting task for even the fastest computers.

Recommendations: I only use Huffyuv when I want to store something, only to open it in another application like VirtualDub so I can do some filtering/etc on it. If you have a capture card that allows you to choose your codec while capturing and Huffyuv is an option, by all means use it but it does take quite a lot of disk space.

Addendum: There are newer lossless codecs that compress better than huffyuv. These are discussed in the clip creation guide. However, Huffyuv is a tried and tested method and is very reliable.

ErMaC and AbsoluteDestiny - May 2004