im sure im going to get yelled at for this but:

[Zarxrax]

Ok, now exactly what steps did you take which resulted in it going from "before" to "after"?

Is the "before" before any avisynth filtering, or after avisynth filtering? If its before avisynth filtering, then you don't seem to need any filtering at ALL because its already very clean looking. And I certainly don't see any rainbows.

Are you compressing the video to something crappy in virtualdub?
You shouldn't be compressing the video in virtualdub.

[AlexanderVonCross]

before is just using the keypad on my computer to search through the scenes, the before is also after i have encoded it with the x264, the after is what happens when i play it in vdubmod. when i try to compress it, it comes out looking like the after. I did no other filtering. But the video is from Sony Vegas after it has been rendered.

[Zarxrax]

What you say is very confusing. Are you trying to compress your video several times? That will definitely make it look bad.

[AlexanderVonCross]

yeah im sorry, i was in a rush. all i was trying to do was compress the video but keep the quality.

[Zarxrax]

Well, you are only supposed to compress to x264 at THE END.

[AlexanderVonCross]

so how do i do that without the file being large and choppy and looking like crap? that's what i dont know who to do. i have heard ppl say to use vdubmod for compression and they keep the quality looking great. Do I do the scripts when I am cutting my clips? This is where I am really confussed. IM srry if I am confussing you again.

[Zarxrax]

Ok, you have already finished editing your video right?

What you do, is
1. you export it from vegas with a lossless codec.
2. if necessary, apply any avs filters
3. encode to x264 with zarx264gui

Then that's it.

[AlexanderVonCross]

what scripts, if any, do you suggest and what codec? also, when i encode it with the zarx264 it splits the video and gives me an aviscript. I think that's where im getting into trouble. When i do export it, should I keep the audio? i know, im probably making this out to be much harder than it really is. Sorry.

[Zarxrax]

If your video quality looks fine before encoding, then you don't need to do ANYTHING in the avisynth script.
Export from vegas with any lossless codec such as huffyuv, lagarith, or UTvideo. If you are unsure, you can't go wrong with UNCOMPRESSED.
You can keep the audio when you export, just make sure its uncompressed.

Load it into zarx264gui (make sure you have the latest one: http://amvhell.com/stuff/zarx264gui/) and press the encode button.

[AlexanderVonCross]

i will do that. if i screw it up, ill be back.