tweaking the x264 settings to prevent banding

User avatar
x265
Joined: Tue Dec 25, 2012 1:52 pm
Org Profile

tweaking the x264 settings to prevent banding

Post by x265 » Wed Apr 24, 2013 7:28 pm

Which of the x264 settings should i change in order to prevent banding?
“The only true wisdom is knowing that you know nothing.”

User avatar
l33tmeatwad
Joined: Wed Feb 16, 2005 3:22 pm
Location: Christiansburg, VA
Contact:
Org Profile

Re: tweaking the x264 settings to prevent banding

Post by l33tmeatwad » Wed Apr 24, 2013 7:45 pm

x265 wrote:Which of the x264 settings should i change in order to prevent banding?
You cannot prevent banding, it's a side effect of 8-bit YUV encoding. You can reduce it from happening quite so bad through tweaking your footage through filters and adjusting the colors, but there's no way to prevent it via encoder settings.
Software & Guides: AMVpack | AMV 101 | AviSynth 101 | VapourSynth 101
PixelBlended Studios: Website | Twitter | YouTube

User avatar
x265
Joined: Tue Dec 25, 2012 1:52 pm
Org Profile

Re: tweaking the x264 settings to prevent banding

Post by x265 » Wed Apr 24, 2013 11:39 pm

I'm using the 10 bit version of x264.
“The only true wisdom is knowing that you know nothing.”

User avatar
l33tmeatwad
Joined: Wed Feb 16, 2005 3:22 pm
Location: Christiansburg, VA
Contact:
Org Profile

Re: tweaking the x264 settings to prevent banding

Post by l33tmeatwad » Wed Apr 24, 2013 11:59 pm

x265 wrote:I'm using the 10 bit version of x264.
10 bit won't remove all banding, it simply reduces the amount.
Software & Guides: AMVpack | AMV 101 | AviSynth 101 | VapourSynth 101
PixelBlended Studios: Website | Twitter | YouTube

User avatar
BasharOfTheAges
Just zis guy, you know?
Joined: Tue Sep 14, 2004 11:32 pm
Status: Breathing
Location: Merrimack, NH
Org Profile

Re: tweaking the x264 settings to prevent banding

Post by BasharOfTheAges » Thu Apr 25, 2013 6:05 am

If It's not noticeable before encoding but is after, upping the VAQ could help.
Anime Boston Fan Creations Coordinator (2019-2023)
Anime Boston Fan Creations Staff (2016-2018)
Another Anime Convention AMV Contest Coordinator 2008-2016
| | |

User avatar
x265
Joined: Tue Dec 25, 2012 1:52 pm
Org Profile

Re: tweaking the x264 settings to prevent banding

Post by x265 » Thu Apr 25, 2013 8:01 am

is it the aq-strength option?
“The only true wisdom is knowing that you know nothing.”

User avatar
l33tmeatwad
Joined: Wed Feb 16, 2005 3:22 pm
Location: Christiansburg, VA
Contact:
Org Profile

Re: tweaking the x264 settings to prevent banding

Post by l33tmeatwad » Thu Apr 25, 2013 8:03 am

x265 wrote:is it the aq-strength option?
If you provided more information we might be able to help you tweak your settings and maybe suggesting some filters for your footage to reduce the banding, but we do need more information to be able to do that. You existing settings and screenshots of before & after for the current problem would be very helpful.
Software & Guides: AMVpack | AMV 101 | AviSynth 101 | VapourSynth 101
PixelBlended Studios: Website | Twitter | YouTube

User avatar
x265
Joined: Tue Dec 25, 2012 1:52 pm
Org Profile

Re: tweaking the x264 settings to prevent banding

Post by x265 » Thu Apr 25, 2013 9:25 am

Source : Death Note R2J
DGSource("C:\Users\Abhijith Nair\Desktop\location\Episode.dgi")
Interp=NNEDI3(nns=4, qual=2, pscrn=4)
deint = tdeint(edeint=Interp, emask=tmm())
tfm(pp=6, micmatching=2, slow=2, clip2=deint)
smartfade(dgm=true)
tdecimate(mode=1)
Episode=trim(1918, 30688)
Preview=trim(32368, 32727)
Opening=avisource("opening.avs")
Ending=avisource("ending.avs")
AlignedSplice(Opening, Episode, Ending, Preview)
#Dehalo_Alpha_mt(ss=3.0)
Dither_Convert_8_to_16()
Dither_crop16(2,4,-2,0)
#F3kdb(dither_algo=2, input_depth=16, grainY=25, grainC=25, output_depth=10)
GradFun3(thr=0.4, smode=2, lsb_in=true, lsb=true)
#Dither_Quantize(bitdepth=10, reducerange=true, mode=0)
#Dither_Out()

Screenshots : (Without filtering)

Image
Image

x264 settings i'm planning to once i'm done correcting the avs script :

--qcomp 0.7 --aq-strength 0.8 --psy-rd 0.75:0.2 --no-dct-decimate --no-fast-pskip
“The only true wisdom is knowing that you know nothing.”

User avatar
l33tmeatwad
Joined: Wed Feb 16, 2005 3:22 pm
Location: Christiansburg, VA
Contact:
Org Profile

Re: tweaking the x264 settings to prevent banding

Post by l33tmeatwad » Thu Apr 25, 2013 11:35 am

First off let me say that Death Note is a tough source considering the color scheme and will be tough to keep the banding down. Increasing your aq-strength closer to 1 would help maintain detail in low detail areas and prevent more banding from occuring, but it's also going to require more bitrate if you want to maintain the same quality and avoid noise around the edges. I'd probably recommend raising psy-rd a little as well.
Software & Guides: AMVpack | AMV 101 | AviSynth 101 | VapourSynth 101
PixelBlended Studios: Website | Twitter | YouTube

User avatar
x265
Joined: Tue Dec 25, 2012 1:52 pm
Org Profile

Re: tweaking the x264 settings to prevent banding

Post by x265 » Thu Apr 25, 2013 12:56 pm

Should i raise psy-rd to 0.9?
“The only true wisdom is knowing that you know nothing.”

Locked

Return to “Video & Audio Help”