x264 encoding resulting in more obvious banding?

If you have questions about compression/encoding/converting look here.
Locked
User avatar
BasharOfTheAges
Just zis guy, you know?
Joined: Tue Sep 14, 2004 11:32 pm
Status: Breathing
Location: Merrimack, NH
Org Profile

x264 encoding resulting in more obvious banding?

Post by BasharOfTheAges » Thu Feb 10, 2011 8:31 am

I don't have any screen shots at the moment (I'm not at my desk), but while encoding something last night with zarxgui I noticed one scene in my video that had pretty severe and ugly banding issues at crf 20 and even crf 10. The footage itself, after closer inspecting, did have some banding in it both in Premiere and after export (so nothing screwed up in that step) but it was far more subtle, and I practically had to stand above my monitor to notice it at a bad viewing angle. I tried adding gradfunkmirror(x) {x=1,2,3} to the avs created for the import into zarxgui and brought the crf all the way down to 5 but only noticed a slight improvement. Interestingly, the banding was not visible to me in vdub with the gradfunkmirror(3) avs script, but was still there in the encoded version. What other options do I have to get rid of this banding? Are there other flags that might be able to deal with this issue better? It's only a problem in maybe 5 seconds of the total video length.

EDIT - the scene is a dark one with an obvious gradient of color (deep blues tones) if that makes any difference or explains why it might be showing up only there and nowhere else in my footage.
Anime Boston Fan Creations Coordinator (2019-2023)
Anime Boston Fan Creations Staff (2016-2018)
Another Anime Convention AMV Contest Coordinator 2008-2016
| | |

User avatar
mirkosp
The Absolute Mudman
Joined: Mon Apr 24, 2006 6:24 am
Status: (」・ワ・)」(⊃・ワ・)⊃
Location: Gallarate (VA), Italy
Contact:
Org Profile

Re: x264 encoding resulting in more obvious banding?

Post by mirkosp » Fri Feb 11, 2011 9:01 am

Don't use gradfunkmirror over 1.2 (or well, maybe 1.5 might still be fine), because going beyond will easily kill detail (I'd say in your case doing 1.01 would suffice). Anyway, I guess the issue is a quantization issue... try adding a slight grain to your video (just addgrain() in your avs) too, and then increase aq-strength (default aq in preset animation is 0.6, which is meant for "paint bucket filled" colours) to something like 0.8 and beyond if it still bands.
Image

User avatar
Cannonaire
Joined: Wed May 05, 2010 5:59 pm
Status: OVERLOAD
Location: Oregon
Org Profile

Re: x264 encoding resulting in more obvious banding?

Post by Cannonaire » Fri Feb 11, 2011 10:48 am

If you're getting the kind of blocky banding I think you are with dark gradients, on the first page in zarxgui try using Tune: grain and on the advanced page deblocking alpha: 2 and beta -2. You might also try using a quantizer of about 17-19, which I've found works wonders in conjunction with tune grain. All of this will raise your filesize slightly, but it shouldn't raise it nearly as much as just raising the quantizer.

I had to use settings like this for a specific scene in Cowboy Bebop which had really dark gradients over most of the picture.
Image Think millionaire, but with cannons. || Resident Maaya Sakamoto fan.

User avatar
BasharOfTheAges
Just zis guy, you know?
Joined: Tue Sep 14, 2004 11:32 pm
Status: Breathing
Location: Merrimack, NH
Org Profile

Re: x264 encoding resulting in more obvious banding?

Post by BasharOfTheAges » Fri Feb 11, 2011 2:31 pm

I'll try a few test encodes tonight with some of these things in mind.

mirko: how high should I go with aq?

If it makes any difference, the scenes i'm working with are the new CGI scenes from the BRD re-release of Ghost in the Shell. I believe they are pretty high-res and high detail, especially compared with the rest of the movie. I'd rather not lose that kinda detail if I can help it, so i'm looking for both a sane approach to use for a distro copy and a ultra HQ approach to send as a con copy.

EDIT - Ok, seems like after actually turning the aq-mode flag on and setting the strength to 0.85, it looks a hell of a lot better. Trying a crf 5, aq-strength 1.1 run as well to see if there's any noticeable difference.
Anime Boston Fan Creations Coordinator (2019-2023)
Anime Boston Fan Creations Staff (2016-2018)
Another Anime Convention AMV Contest Coordinator 2008-2016
| | |

Mister Hatt
Joined: Tue Dec 25, 2007 8:26 am
Status: better than you
Contact:
Org Profile

Re: x264 encoding resulting in more obvious banding?

Post by Mister Hatt » Sat Feb 12, 2011 5:47 am

Funnily enough, your debanding is causing your banding in the first place I imagine. You won't get rid of it unless you use 10bit or crf4 or so. The correct thing to do is add grain, not dither. One of the smoothers in your filterchain might be exaggerating it too.

Seeing as I missed the part about it being the GITS BD (which needs AQ 1.2 or so, possibly 1.5 if it's SAC which has some weird moires and shit on the side). If this is the GITS 2.0 BD, your CRF doesn't need to be below 18, or even 20. Just keep your quantizer settings in check and be careful of trellis and rdo.

User avatar
BasharOfTheAges
Just zis guy, you know?
Joined: Tue Sep 14, 2004 11:32 pm
Status: Breathing
Location: Merrimack, NH
Org Profile

Re: x264 encoding resulting in more obvious banding?

Post by BasharOfTheAges » Sat Feb 12, 2011 12:24 pm

There is no filter chain actually, unless you want to count gradfunkmirror() by itself from an export a filter chain. I have no idea what I used to preprocess this (I ripped and cleaned this over a year ago on another system) but the banding wasn't noticeable in premiere going in. There was an aq bug in zarxGUI; I already spoke to zarx about it, and did my last exports via command line - that was the cause of my problems. The new exports look a hell of a lot better and aren't HUEG anymore. I have an unrelated reason I want to export again, so I'm going to try one last series of aq settings.
Anime Boston Fan Creations Coordinator (2019-2023)
Anime Boston Fan Creations Staff (2016-2018)
Another Anime Convention AMV Contest Coordinator 2008-2016
| | |

Locked

Return to “Conversion / Encoding Help”