ffdshow, in it's full version (available at the ffdshow-tryouts project page), has a VFW interface. What this means is that, aside from using ffdshow to decode video in VDub/VDubMod (and avoid that 'no VFW codec found; DirectShow codecs cannot be used' warning, whatever the actual wording of that error is), you can use it to compress video also.
Now, most of the formats it offers are available or are better encoded elsewhere or just good for more experimental encodes. These include things like MPEG-4 variants, WMV formats, DV, and other formats like Snow, FLV1, and Theora, so only those that like to tinker would find a use for them. However, it does have the ability to encode to two formats which are very useful - MJPEG and a YV12-mode version of HuffYUV.
The particular advantage of encoding these two formats through ffdshow is that, for MJPEG, there is no watermarking or [obviously] the need to pay to use the encoder like one would have to do with PICVideo, and for HuffYUV, you can keep your video in YV12 colorspace. Encoders like TMPGEnc, that sometimes have problems with Lagarith (which is really the only other well-used lossless YV12 compressor that gets toted around here because of relative speed), usually don't have problems with HuffYUV's YV12 mode, at least in my experience. HuffYUV-YV12's output is still larger than Lagarith's YV12 output, but is much closer to it than the output of standard YUY2 HuffYUV, provided the Adaptive Huffman tables option is selected.
The downside, which only really applies to HuffYUV, as in this case I assume one already uses ffdshow to decode MJPEG, both in the standard decoder and through VFW, is that using HuffYUV-YV12 (or using the Adaptive Huffman tables option with YUY2-mode, which ffdshow can also encode to), requires the user to use ffdshow to decode HuffYUV at all times, as neither YV12 or the tables are compatible with the normal HuffYUV decoder. There is a plus side here, though, and that is that using ffdshow to decode HuffYUV-YV12 is fast. Even on my 6-year-old computer, HuffYUV-YV12 often does achieve or closely approaches full-speed playback at SD resolutions and sane framerates, as long as there aren't huge, constant amounts of motion.
Anyway, onto the actual explanation. As usual, I'm assuming VirtualDub is set on Fast Recompress because it preserves colorspace, and that therefore, all manipulations are being done via AviSynth.
Go to the Video menu and select Compression. You should see an option for ffdshow Video Codec. Select it and then click Configure.
First, set up the Decoder. You should be on that tab, but if not, select it and check to see if MJPEG and HuffYUV are enabled (the box next to them should say 'libavcodec' - if they don't, click on the box and select it yourself).
Pop back over to the 'Encoder' tab. You'll need to use the drop-down menu and select HuffYUV or MJPEG from the list there. We'll start with HuffYUV:
Now, the screen presented to you (it should be the 'Generic' options screen) is the only one you'll have to worry about. It should also already be set on YV12, and Plane predictor type. However, you'll probably need to check the Adaptive Huffman tables option yourself, and maybe change the FourCC to HFYU, as this screenshot shows:
Click Apply to set the changes, then OK to go back to the Compression dialog. Click OK again and continue saving your video.
Now onto MJPEG. Select MJPEG from the drop-down menu shown previously. You should presented with this screen:
You can select different modes to determine quality, bitrate, quantizer, etc. Generally I find this virtually useless to mess with because these aren't going to be preserved for quality renders - it's just to speed up the process. So if it isn't set on 'one pass - constant bitrate' set it to that now. The only thing about this is to set the bitrate. I find 900kbps to be adequate for small resolution video (for instance, 320x240 - I use that to seek through and find clips, and write down the frame numbers for a better-quality export of those same clips later with HuffYUV - in RGB-mode, since I use Premiere). If you want higher-resolution video, you can adjust accordingly so it doesn't look like complete crap. You can change the bitrate either by typing it in through the dialog field box or by using the slider located to the box's right.
And again, you can now click Apply to set the changes, then OK to go back to the Compression dialog. Click OK again and continue saving your video.
