Problem with resize: Jagged Lines

This forum is for questions and discussion of all the aspects of handling your footage. If you have questions about capturing/ripping footage, AviSynth, or compression/encoding/converting, look here.

Problem with resize: Jagged Lines

Postby wilder » Mon Oct 12, 2009 5:42 am

hello..
i have use lanczosresize before with no problem, but this time..
it produce jagged lines..
i resize it from 480p to 368p..

why is this happening? never before it happen..
i don't use any other filter except FFVideoSource and LanczosResize..

here is the snapshot
original : http://i575.photobucket.com/albums/ss19 ... iginal.jpg
jagged: http://i575.photobucket.com/albums/ss19 ... jagged.jpg

if someone know, please tell me OK?
thank you
wilder
 
Joined: 12 Oct 2009

Re: Problem with resize: Jagged Lines

Postby Scintilla » Mon Oct 12, 2009 6:09 am

1) I'm not seeing the problem.
2) Both of those shots are 1024x578, not 480p or 368p.

Maybe you have the wrong URLs?
ImageImage
:pizza: :pizza: Image :pizza: :pizza:
User avatar
Scintilla
(for EXTREME)
 
Joined: 31 Mar 2003
Location: New Jersey
Status: Quo

Re: Problem with resize: Jagged Lines

Postby wilder » Mon Oct 12, 2009 7:30 am

i took the snapshots in full screen while playing in MPC..
the different is not that obvious but in other parts of the video, it looks ugly..

this is another example..
look at her mouth..
original: http://i575.photobucket.com/albums/ss19 ... inal-2.jpg
jagged: http://i575.photobucket.com/albums/ss19 ... gged-2.jpg

also, i noticed that there are black border at left and right of the jagged picture..
i don't remember adding a border..
is this common when using FFVideoSource?

thanks
wilder
 
Joined: 12 Oct 2009

Re: Problem with resize: Jagged Lines

Postby Qyot27 » Mon Oct 12, 2009 10:34 am

It looks like edge noise and/or slight banding.

What decoder are you using in MPC? CoreAVC, or ffdshow? If CoreAVC, then it's applying additional post-processing before showing it. The same *could* also be true with ffdshow, but it's less likely by default. FFVideoSource will not show the video with additional post-processing (well, it has the ability to perform pp, but those are function calls that advanced users would be inclined toward). What you're seeing is the video how it really is.

Hit it with a temporal/spatial smoother and a debanding filter and it should be improved. In my case that would be something like deen("w3d",3,6,8,8,8).gradfun2db(2.0) - but there's other smoothers out there to do the work if you don't want to use deen.
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog
User avatar
Qyot27
Surreptitious fluffy bunny
 
Joined: 30 Aug 2002
Location: St. Pete, FL
Status: Creepin' between the bullfrogs

Re: Problem with resize: Jagged Lines

Postby wilder » Mon Oct 12, 2009 1:04 pm

In MPC i use ffdshow..

to see the difference better, here videos i uploaded:
original:

jagged:


actually, i convert 720p to 480p first. then 480p to 368p. i think this way the file can compress better..
and i'm using x264 with deblock settings -2,-1. does this has something to do with the problem?

sorry if I break forum rules.
thank you
wilder
 
Joined: 12 Oct 2009

Re: Problem with resize: Jagged Lines

Postby Zarxrax » Mon Oct 12, 2009 1:17 pm

First off don't test the quality of your videos in MPC. There are too many variables to account for. Look at it in AvsP or virtualdub.
Next, don't resize twice, its unnecessary.
Finally, decreasing the deblock settings in x264 will make blocks and other artifacts more noticable, which *may* be contributing to the problem. I would highly recommend leaving them at 0 or even setting them to positive values.
User avatar
Zarxrax
 
Joined: 01 Apr 2001
Location: Concord, NC

Re: Problem with resize: Jagged Lines

Postby wilder » Mon Oct 12, 2009 9:25 pm

thanks..
i will use no-deblock, it's safer this way..
wilder
 
Joined: 12 Oct 2009

Re: Problem with resize: Jagged Lines

Postby wilder » Tue Oct 13, 2009 12:22 am

I think the problem is not with the deblock..
I add FFT3DGPU with sigma=20 and the jagged lines still there..
maybe a problem with renderer? i open it in virtualdub too but also jagged..
the jagged lines only appear/clearer in 368p while in 480p it's not that clear..
someone know about this?
thanks
wilder
 
Joined: 12 Oct 2009

Re: Problem with resize: Jagged Lines

Postby Qyot27 » Tue Oct 13, 2009 12:43 am

Now that I took a look at the samples in motion, I'm convinced that what you're seeing is - considering you said you were using lanczos as your resizer - ringing artifacts. It's the Lanczos algorithm that creates it. The possible solutions are to pick a different resizer (you could try the Spline family of resizers, for instance), or to look for a specific anti-ringing plugin. I don't know of any offhand, though.

As a general tip when asking for help with a source, don't use JPEG for screenshots. Use PNG, because then there won't be any additional artifacting produced from the compression.

And if you're looking to make for easy compression of an end product, Lanczos is not going to help the situation - it's designed to retain sharpness, which adds to the compression cost (meaning it'll need a higher bitrate to look good). You could try Bicubic or, if you prefer a softer-looking image, Bilinear resizing.
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog
User avatar
Qyot27
Surreptitious fluffy bunny
 
Joined: 30 Aug 2002
Location: St. Pete, FL
Status: Creepin' between the bullfrogs

Re: Problem with resize: Jagged Lines

Postby Qyot27 » Tue Oct 13, 2009 12:50 am

Gah, double-post.

In addition to the ringing artifacts produced by Lanczos, I also suspect it's getting exacerbated by the settings you're using in x264. Post your compression settings and the x264 revision number.
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog
User avatar
Qyot27
Surreptitious fluffy bunny
 
Joined: 30 Aug 2002
Location: St. Pete, FL
Status: Creepin' between the bullfrogs

Re: Problem with resize: Jagged Lines

Postby wilder » Tue Oct 13, 2009 1:15 am

Thanks for replying..
i don't think lanczos is the problem because i try with bilinearresize and it also jagged..
maybe i exaggerate too much..(noob) :sweat:

Writing Library : x264 core 76 r1271 496d79d

for now, this is the most original (720p) video settings:
Spoiler :
cabac=1 / ref=16 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=9 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=6 / nr=0 / decimate=0 / mbaff=0 / bframes=8 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=1803 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00


this is the second video (480p) settings:
Spoiler :
cabac=1 / ref=10 / deblock=1:-2:-1 / analyse=0x3:0x113 / me=umh / subme=9 / psy=1 / psy_rd=0.4:0.0 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=3 / nr=0 / decimate=1 / mbaff=0 / bframes=5 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=16.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=2:0.60

avisynth:
Spoiler :
LanczosResize(848,480)


and this is the video with jagged lines (368p) settings:
Spoiler :
cabac=1 / ref=16 / deblock=1:-2:-1 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=0.4:0.0 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=3 / nr=0 / decimate=1 / mbaff=0 / bframes=5 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc_lookahead=60 / rc=2pass / mbtree=1 / bitrate=306 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=2:0.60

avisynth:
Spoiler :
Toon(0.3)
FFT3DGPU(sigma=3, precision=2)
LanczosResize(640,368)


thanks
wilder
 
Joined: 12 Oct 2009

Re: Problem with resize: Jagged Lines

Postby Qyot27 » Tue Oct 13, 2009 11:37 am

wilder wrote:and this is the video with jagged lines (368p) settings:
cabac=1 / ref=16 / deblock=1:-2:-1 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=0.4:0.0 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=3 / nr=0 / decimate=1 / mbaff=0 / bframes=5 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc_lookahead=60 / rc=2pass / mbtree=1 / bitrate=306 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=2:0.60

Why such a low bitrate? That's definitely causing noise and artifacting problems, especially if you're recompressing from that CRF 16 480p copy. You need to raise the bitrate, and you also could try to raise scenecut to 100 and see if that helps. With the resizing, try with Spline64Resize and see if you can still see the problem in the script.

And you might want to update x264 - a slew of bugs were found and fixed over the past couple days and I'm not sure if they were just restricted to using GCC 4.4.x as the compiler (which wouldn't apply to x264.nl's builds, they use GCC 3.4.5) or if some were intrinsic in the code itself. They're up into the 1290s now.
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog
User avatar
Qyot27
Surreptitious fluffy bunny
 
Joined: 30 Aug 2002
Location: St. Pete, FL
Status: Creepin' between the bullfrogs

Re: Problem with resize: Jagged Lines

Postby wilder » Wed Oct 14, 2009 4:31 am

this problem made me crazy..
i try to watch it (368p) in VLC and the jagged line is not as clear as in MPC..
maybe i have to try CCCP instead of K-Lite..
thanks
wilder
 
Joined: 12 Oct 2009

Re: Problem with resize: Jagged Lines

Postby Kariudo » Wed Oct 14, 2009 9:45 am

That shouldn't make any difference, but go ahead and do it anyway.

K-lite is a cancer that must be killed with fire. In fact...don't stop at killing it once, kill it as many times as you can (and then kill it one more time to make sure it doesn't turn into a zombie.)
ImageImage
Image
User avatar
Kariudo
Twilight prince
 
Joined: 15 Jul 2005
Location: Los taquitos unidos
Status: 1924 bots banned and counting!

Re: Problem with resize: Jagged Lines

Postby wilder » Thu Oct 15, 2009 7:25 am

yeah, you are right..
no different at all..
now i know what is the real problem..
all video with low resolution(400p,368p...), if open in full screen will result in jagged/pixelated/blockiness lines/edges..
maybe a problem with decoder or my graphic card..
in the end avisynth has nothing to do with it..
tq
wilder
 
Joined: 12 Oct 2009


Return to Footage Help

Who is online

Users browsing this forum: No registered users and 2 guests