It's H.264, and H.264 in MKV can come in two different flavors - VFW (i.e. AVI mode = bad bad bad), and Native mode (the proper storage method, also used in MP4). While some VFW-H.264 encodes were released early on - and a few still do - the other 99% do the right thing and use Native mode.
VirtualDub (and VDubMod) can only use VFW-based solutions, end of story. VirtualDub does have import plugin support since like, 1.7.6 or something, but MKV is a minefield to support that way, and I've not played around too much with that stuff to say it.
So AviSynth is your only option. It simply acts as a proxy, telling VirtualDub that it's opening an AVI when AviSynth is controlling the rest of the process. And AviSynth has proper support for DirectShow, DGAVCDec, and FFmpeg based input of H.264 - the last two also being frame-accurate, with FFmpegSource supporting Native-mode MKV directly.