Sane aq settings for h264 encoding

This forum is for questions and discussion of all the aspects of handling your footage. If you have questions about capturing/ripping footage, AviSynth, or compression/encoding/converting, look here.

Sane aq settings for h264 encoding

Postby BasharOfTheAges » Wed Mar 07, 2012 11:53 am

So, I'm removing some minor banding, made more obvious with an upscale from 480p to 720p (not in the mood to debate the need of it). I've followed advice from a previous encode where I had horribly strong banding, and added grain and whatnot, but when it came to the second part of the advice, adjusting the aq setting for the encode, I found I didn't really have a grasp on the scale that was being used and where the benefit/drawback tipping points were... or even what the drawbacks to high values might be. Anyone have insight?
Another Anime Convention AMV Contest Coordinator 2008-2014 & Head of the AAC Fan-works Theater - follow us on Twitter: https://twitter.com/#!/AACFanTheater
:sorcerer: :sorcerer: |RD: "Oh, Action!" (side-by-side) | |
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Re: Sane aq settings for h264 encoding

Postby mirkosp » Wed Mar 07, 2012 12:29 pm

Values lower than 1 help compression, but if too low introduce obvious banding. Values greater than one will preserve fine detail, at the cost of even huge bitrate increases. We're talking more or less of actual bitrate doubling from 0.7 to 1.4 (which makes sense, I guess). Generally a 0.8 value is very safe, won't inflate the filesize and overall the banding should be absent. The default 0.6 for the animation preset is more meant for western animation, which has much less gradients and details than eastern animation, as it uses more paint bucket-like fillings.
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Sane aq settings for h264 encoding

Postby BasharOfTheAges » Wed Mar 07, 2012 12:38 pm

Exactly what I was looking for, thanks.
Another Anime Convention AMV Contest Coordinator 2008-2014 & Head of the AAC Fan-works Theater - follow us on Twitter: https://twitter.com/#!/AACFanTheater
:sorcerer: :sorcerer: |RD: "Oh, Action!" (side-by-side) | |
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Re: Sane aq settings for h264 encoding

Postby Mister Hatt » Thu Mar 08, 2012 5:41 am

If you have heavy handing (FMA:B, Cobra, etc) then AQ of up to about 1.2-1.3 is necessary. Anything more than that and you want to look at dithering and 10bit encoding tbqh. The general way it works is as mirko said, where bits from flat areas are relocated to areas of higher detail. The strength is a scale of 0-3 on how aggressive that is. Note that you can use zones in x264 to apply different strengths to different sections, IIRC anyway. Once you get to high AQ strengths however, you tend to make flat areas blocky, and at lower strengths you'll have the opposite effect of detailed areas getting squandered for useless chroma gradients.
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Sane aq settings for h264 encoding

Postby mirkosp » Thu Mar 08, 2012 6:12 am

Mister Hatt wrote: Note that you can use zones in x264 to apply different strengths to different sections, IIRC anyway.

No, you can't. As D_S stated himself, the way aq works is incompatible --zones and would require its own --aq-zones command if anything.
Related: http://forum.doom9.org/showthread.php?t=162268
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Sane aq settings for h264 encoding

Postby Mister Hatt » Thu Mar 08, 2012 7:57 pm

Oh right. When I wrote that I was like "now AQ either works with zones or is the one thing that doesn't… well nobody would zone crf, must be aq" and then I remembered a few hours later w. Also I totally wrote IIRC, first "I" of which is "if" :3
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Sane aq settings for h264 encoding

Postby mirkosp » Thu Mar 08, 2012 8:11 pm

Heh. Also you CAN zone crf anyway... >_>
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Sane aq settings for h264 encoding

Postby Mister Hatt » Thu Mar 08, 2012 10:10 pm

Yes I know. What I meant was the main reason for zoning was either CRF or AQ but I forgot which.
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you


Return to Footage Help

Who is online

Users browsing this forum: No registered users and 1 guest