tweaking the x264 settings to prevent banding

[Mister Hatt]

D_S is not wrong in that link, but it is 5 years old and x264 changes frequently. RDO doesn't bias as hard now (unless you tell it to) and there are better things to do for rate optimisation and bit allocation than RDO now. FGO is a good one (if you use the patch) but only in HD anime. Otherwise, the qcomp, aq-strength, mbtree, and ip/pb settings generally do better. Deadzoning is worth looking at too in the event of grain (even dither) issues but that stuff is difficult to tweak unless you really know what's going on. The main reason I don't give hard numbers is I think people should read up on things instead of just waiting for an exact answer which is probably wrong.

[l33tmeatwad]

D_S is not wrong in that link, but it is 5 years old and x264 changes frequently. RDO doesn't bias as hard now (unless you tell it to) and there are better things to do for rate optimisation and bit allocation than RDO now.

The way I've always understood it was that the subpixel refinement level controlled how much the RDO influenced the encode. I don't recall seeing any documentation about any kind of change to this, if you wouldn't mind linking me to your sources I would like to read up on this so I can stay informed.

FGO is a good one (if you use the patch) but only in HD anime. Otherwise, the qcomp, aq-strength, mbtree, and ip/pb settings generally do better. Deadzoning is worth looking at too in the event of grain (even dither) issues but that stuff is difficult to tweak unless you really know what's going on. The main reason I don't give hard numbers is I think people should read up on things instead of just waiting for an exact answer which is probably wrong.

I can't disagree there, tweaking those can help, although as I said before, you don't want to get too crazy with lowering the qcomp so much that it negatively impacts the rest of the video.

The main reason I don't give hard numbers is I think people should read up on things instead of just waiting for an exact answer which is probably wrong.

It's not about giving hard numbers for them to use, it's about giving them help by saying "here is a good place to start, and you go up from here" etc. Anything short of that is not actually helpful and just serves to frustrate people by trying to work with incomplete answers.

[Mister Hatt]

I can't think of any big changes to RDO, just small things over time like how CRF differs between revisions. I suppose you could read over the commit logs if you really wanted to. The main thing is, it's more neutral than blur-biased like it used to be. I think that's a good thing considering sources have gotten better over the years and micro-detail is more often than not grain-based rather than dither-based.