AVI, MKV, OGM... they're all just containers. It's what's in them that changes so much, and makes it a pain to edit with.
As for compatibility mode, I wouldn't really recommend it. There shouldn't be a need.
As for why it crashes as soon as you hit save... hmm. If it's having trouble seeking in DSS mode... it could be a decoder conflict. As in, two decoders trying to decode the exact same thing. Computers don't like it when that happens.
Here, try this.
http://cccp.mayoko.net/CCCP-Insurgent-2006-01-01.exe
Run that program. Do a test render on the file you're trying to feed into VirtualDub. It'll output a bunch of information, like what decoders are trying to turn on. Plus, the program lists all of the decoders installed on your computer, so it should be easier to see what's there.