quality issues, what am i doing wrong?:(

[Overture]

k so ive read a & e's guide and do everything the guide says. i rip my dvds, dgindex, filter, huffyuv, edit and render in huffyuv, then once the file comes out of my vegas in huffyuv avi( i stuck to huffyuv the whole way) its all still nice and clean. so heres the problem... ill input my amv into megui, and try to configure it and whatnot, then when it comes out OH MY GOOOOOD it looks HORRIBLE!!! mostly because of macroblocking and a little sharpness issues. everytime:( i convert it to x264 mp4 and everything and it seems just all at once i lose all my quality from that. i kno as long as u compress it the quality will get worse but it looks nothing like the perfectly encoded vids(funky-kuns just for a day in my opinion is amazingly encoded) ive seen.

can anyone please help:) how do i encode my amv to look like funky-kuns or anyone elses? i follow the guide but is that not enough? heres what a typical script from me looks like if this will help, though keep in mind i lose my quality from compressing to h264, NOT in vdubmod:
Avisource(blahblahblah)
converttoyv12()
LanczosResize(848,480)
deen("a2d",4)
temporalcleaner()
mftoon()
fastlinedarken()

ive even tried adding in a temporalcleaner and mftoon(strength=128) in my megui script. doesnt help. please help:) thanx in advance

[Hareoic]

In this order:

Cropping
Cleaning
Sharpening
Resizing
Colorspace conversion

I think that's the order you're supposed to do it in.

Also, try reading the new version of that guide (http://www.animemusicvideos.org/guides/avtechbeta/).

[Overture]

alrite, but the colorspace in the beginning is for my filter compatibility. the one at the end is if i want it to be rgb rite? but ill try resizing towards the end too, thanx:). so how about my problem with compressing? seems i lose WAY more quality then i could actually keep. thats my MAIN concern:P

[The Origonal Head Hunter]

never used MEGUI, does it have any settings for bitrate or quantizer or anything?

[Overture]

yes it does but i dont touch it since i dont kno too much about it. the bitrate is at 1000 and the quantizer says it at 0.6 and the ratio is (I:P/P:B) 1.4 / 1.3
hope thats makes sense. maybe i should try using zarxgui instead? do u happen to kno a guide or something to learn to use quantizers and such?

[Qyot27]

MeGUI is probably the most transparent of the Windows x264 apps, meaning it will allow you to configure just about all of the options.

A) Make sure MeGUI itself and its tools are all updated. If you have to, change the update servers from Stable to Development.

A2) When selecting the profile, choose x264 *scratchpad* and then use the Config dialog to specify your own settings. Once you have them customized to your liking, you can save it under a unique name for easy selection the next time you encode something.

B) Read the tooltips that appear when your mouse hovers over an option. They'll help you tweak the settings to your advantage.

C) Use Constant Quality mode rather than ABR. A value of 18 is roughly equivalent to XviD's Quantizer 2 setting. Besides that, 1000kbps may not be enough to really handle the content in the video. If there's a lot of motion and complex imagery, it won't be enough. H.264 may have amazing compression abilities, but it can't work miracles.


And as for more of an AviSynth thing, please, please make sure that the source is horrible enough to justify using strong smoothing settings, or mftoon at all. Setting those filters so strongly (and using harsh filters like the -toon family) do more harm to the source than good, and are/were meant for only certain scenarios, not as a panacea for anything you'd be working with. If the source is good, you don't have to do practically anything.

See this article (yeah, it comes from a fansubbing perspective but it's amazingly relevant here):
http://www.mod16.org/hurfdurf/?p=13

[Qyot27]

Also, you can save on bitrate by not resizing to 848x480. Leave the video at 720x480 and use a 16:9 aspect flag. This can be done when you mux the video and audio together after-the-fact, and can be done when you load the video into YAMB/MP4Box or MP4Creator or MKVMerge or whatever.

That is, unless you're encoding for some device that doesn't support 16:9 flags. But seeing as how even iTunes' TV Show purchases use 16:9 flags, I think that number is probably rather small.

[Overture]

i actually didnt kno toon would actually harm my footage so bad. ive seen a little harm but i didnt kno it was that bad. thanks for the valuable info. but i would try using powerful smoothers thinking they might help reduce some of the excessive macroblocking and im guessing alot of it is excessive grain too. any suggestions in filters to prevent grain and macroblocking? thats definitely the biggy in the quality drop of my encodes. that would be much appreciated:)

[Qyot27]

Smoothing can indeed help on macroblocking and grain, but at the expense of other elements in the video. Macroblocking won't completely be gone, and if it does completely go away, then the picture will look watery. Grain is a personal issue - some can look aesthetically appealing, that's why it's often there in Films. Mosquito noise, on the other hand, could be referred to as 'grain', and often shows up around sharp edges.

Generally, for DVD rips I'll still use deen, but use w3d mode with lighter settings. Typically, this means deen("w3d",2,8,8,8,8) - that enables both spatial and temporal smoothing, and is usually what I use for sources that are already generally clean, as it's only really powerful enough to catch some of the glaring errors rather than get really intensive. If it's bad, I might use 3,6,8 instead of 2,8,8. If it's even worse, I take it on a more custom basis. The worst I dealt with was probably X/1999, where I went all the way up to 4,10,12 - although that was a few years ago, when other parts of my filtering chain were different. Today I'd probably be more apt to use 3,8,10 or 3,10,12 on it.

[Overture]

i see:) so scripting is much MUCH more important and indepth then a & e's guide leads me to believe. i guess ill try out tweaking all my script settings and seeing what does wat i guess. would you happen to kno a place with a good guide to learn more indepth scripting, tweaking them out, or even more about quantizers and vbr settings? i only kno the gist of it but not enough to do any real changes on my own. oh and THANX SO MUCH for the advice. it really did help ALOT. my macro and grain is already down a little:)