Beginning to experiment with Avisynth - No clue what to do

[Phantasmagoriat]

True enough.
I guess when I nuke, I nuke HARD!

You know, I think I'll make a new thread for range() instead of hijacking this one
http://www.animemusicvideos.org/forum/v ... 5&t=101949

[bloppyblue]

Thanks for the responses guys, but there are just some things I want to get clear...
I'm more confused about what filters I should be using than how to separate what gets filtered... And almost all the answers were regarding trimming the clips and such... Sorry for not making that clear

So I tried to do ttempsmooth and I'm confused as to what I should put in for "pfclip" When I put in "null" an error window opens. When I close that error window, AvsP starts going haywire and more error windows pop up.
Error window says:

Code: Select all

Traceback (most recent call last):
  File "AvsP.pyo", line 7123, in OnPaintVideoWindow
  File "AvsP.pyo", line 9467, in PaintAVIFrame
  File "pyavs.pyo", line 322, in DrawFrame
  File "pyavs.pyo", line 306, in _GetFrame
  File "avisynth.pyo", line 309, in GetPitch
ValueError: NULL pointer access

But for a thousand times or so.

Then I tried dfttest and it won't locate the libfftw3f-3.dll in the System32 folder (with PATH set... I read that in another thread) So I gave up on that.

So I started to mess around with it...
The original:


With
Deblock(quant=25, aOffset=0, bOffset=0, mmx=true, isse=true)
gradfun2db(7)


And then with
Deen("w3d",3,3,5)


But it STILL looks crappy. Holy crap, this is driving me insane!

I know somebody said NEVER to use Deen, but I don't know what alternative to go to!

[bloppyblue]

Oh yeah, and is there anyway to avoid having things just flat out destroyed? That's gonna have to be dealt with range/trim/whatever, right?

Regard the black dress.
Before:

After:

Voila, it's gone.

Argh, I'm going insane!!

[Phantasmagoriat]

I'm more confused about what filters I should be using than how to separate what gets filtered... And almost all the answers were regarding trimming the clips and such... Sorry for not making that clear

don't mention it, we were probably getting OT anyways, that's why I made a new thread

So I tried to do ttempsmooth and I'm confused as to what I should put in for "pfclip" When I put in "null" an error window opens. When I close that error window, AvsP starts going haywire and more error windows pop up.
Error window says:

Code: Select all

Traceback (most recent call last):
  File "AvsP.pyo", line 7123, in OnPaintVideoWindow
  File "AvsP.pyo", line 9467, in PaintAVIFrame
  File "pyavs.pyo", line 322, in DrawFrame
  File "pyavs.pyo", line 306, in _GetFrame
  File "avisynth.pyo", line 309, in GetPitch
ValueError: NULL pointer access

But for a thousand times or so.

did you try ttempsmooth() just by itself without any arguments? Do you even have TTempSmooth.dll in your avisynth plugins folder?

Then I tried dfttest and it won't locate the libfftw3f-3.dll in the System32 folder (with PATH set... I read that in another thread) So I gave up on that.

Did you check if you actually have libfftw3f-3.dll in your C:\Windows\System32 folder?

So I started to mess around with it...
[...]
But it STILL looks crappy. Holy crap, this is driving me insane!

It's not that that bad, some of it may even go away in the final stage when you compress your video... but still it looks like you'll need to experiment with temporal cleaning filters [like fft3dgpu(), ttempsmooth(), dfttest(), fluxsmoothst() etc...] and a gradient-fixer [gradfun2db(), gradfun2dbmod(), gradfunkmirror() etc...] I can't recommend much for temporal filters because they are... temporal. Thus require a sequence of frames to process [I suppose if you are really desperate, you could post a lossless video clip... ] But anyways, you'll definitely need a gradient-fixer. Higher values with gradfun2db show some good improvements at the expense of some haloing around edges, but if your temporal filters do a good job you won't need as high values for gradient fixing.

I know somebody said NEVER to use Deen, but I don't know what alternative to go to!

Yeah, Deen is generally not a good idea, but if it gives you the results you want, nobody is stopping you from using it. I would just combine it with something like range() so it doesn't get applied to your whole video... otherwise most of your video will look overfiltered. If you want alternatives, check out my last post in this thread:
http://www.animemusicvideos.org/forum/v ... 5&t=101111

Oh yeah, and is there anyway to avoid having things just flat out destroyed? That's gonna have to be dealt with range/trim/whatever, right?

yup. Combine the filters that work the best with range(), and only certain frames of your video will be filtered. But I guess, the long and sort answer is you'll have to experiment.

[Qyot27]

To be completely honest here, monitor settings have a lot to do with how noticeable this is. I threw the before and after shots next to each other and jumped back and forth in Firefox, and I couldn't see the problem with the black dress. I had to crank my brightness values way up just to see it (and yeah, that's pretty nasty, but I'm not seeing why the smoother - any smoother - was even necessary in the first place).

K-ON! is a really, really recent source. There should be practically no reason to smooth it at all, aggressively or otherwise (especially depending on other factors like the resolution you'll be distributing at, as some resizing methods will mask imperfections when you downscale). And without the smoothing being there, the need for gradfun to clean up the banding the smoother may cause/exaggerate is even less necessary.

The key to AviSynth filtering is essentially less is more. If you can't identify the exact problem you want to fix, and then locate an adequate filter to do that job, just don't filter it. 'Filtering' in this case meaning things intended to do enhancement - smoothing, sharpening, Tweak or ColorYUV, etc.; frame manipulations like inverse telecine (IVTC) and resizing are a different story. Again, however, IVTC and resizing operations are used on an as-needed basis, not simply because it's possible to run them on any video you may come across. And if the filter seems to be doing too much, lower its strength. If you get to a point where even the lowest strength attempt is too much, find another filter and test its results.

So my primary suggestion here is to bone up on what, if any, actual issues to fix are there first, not to basically throw paint at a canvas and hope you get the The Disintegration of the Persistence of Memory.

[bloppyblue]

did you try ttempsmooth() just by itself without any arguments? Do you even have TTempSmooth.dll in your avisynth plugins folder?

I just used it's default... So I used
TTempSmooth(maxr=3, lthresh=4, cthresh=5, lmdiff=2, cmdiff=3, strength=2, scthresh=12.0, fp=true, vis_blur=0, debug=false, interlaced=false, pfclip=?)

Except in place of ? I had put null (because it told me to...)

Did you check if you actually have libfftw3f-3.dll in your C:\Windows\System32 folder?

Yes, I do. I have libfftw3f-3.def plus pairs of two others that are very similar (came with a pack).

It's not that that bad, some of it may even go away in the final stage when you compress your video... but still it looks like you'll need to experiment with temporal cleaning filters [like fft3dgpu(), ttempsmooth(), dfttest(), fluxsmoothst() etc...] and a gradient-fixer [gradfun2db(), gradfun2dbmod(), gradfunkmirror() etc...] I can't recommend much for temporal filters because they are... temporal. Thus require a sequence of frames to process [I suppose if you are really desperate, you could post a lossless video clip... ] But anyways, you'll definitely need a gradient-fixer. Higher values with gradfun2db show some good improvements at the expense of some haloing around edges, but if your temporal filters do a good job you won't need as high values for gradient fixing.

Thank you for listing those. I'll definitely check them all out and write it down. And no, I'm not gonna post a lossless clip I'm sure this just takes a lot of trial and error to get right.

Thanks for all that help Phantasmagoriat. Your username is also very fun to type.

To be completely honest here, monitor settings have a lot to do with how noticeable this is. I threw the before and after shots next to each other and jumped back and forth in Firefox, and I couldn't see the problem with the black dress. I had to crank my brightness values way up just to see it (and yeah, that's pretty nasty, but I'm not seeing why the smoother - any smoother - was even necessary in the first place).

Hm... Alright. I'll make sure in the future I don't rely on Avisynth for everything and filter every source I have. Although on my monitor, the noise and random blocks in the backgrounds were very noticeable... Thanks for bringing that to my attention.

[Cannonaire]

If you haven't tried it already, MCTemporalDenoise can do an amazing job without destroying your source in many cases. It requires a lot of other filters to be installed in order to work, but once it's working it's really easy to use as far as parameters go. The main drawback though is it's glacial speed. On low settings you can expect it to take between 2-5 hours to work on a 23~ minute episode depending on your system, source resolution, and other factors. Regardless, I find it to be much better than many alternatives; I just set Virtualdub to render a few scripts with job control while I'm asleep or at work.

Here's the avisynth page for it: http://avisynth.org/mediawiki/MCTemporalDenoise

I believe it also adds grain by default with all of the presets, which should keep your footage from banding when you encode with x264 or w/e your codec of choice is, but I could be wrong on this point.

[Mister Hatt]

I just noticed you're using 1920x1080. K-ON S1 was actually upscaled on the bluray. How does it look if you bicubicresize(1280x720,0.333,0.333) and don't filter at all? I might also suggest using the actual BDMV seeing as I can tell from the quality that you're using some torrented raws and not the bluray itself. It doesn't look bad at all and requires no filtering, just a resize and IIRC some slight cropping.

@Cannonaire: I've seen the source material, MCTD is not required. Expecting an amver to even use it is somewhat ridiculous given how slow it is. 1920x1080p24 on a single 24 minute episode on a core2duo takes around 6 DAYS. I can't see anyone doing that, and it's not like everyone here has powerful machines. Quadcores make little difference due to it not being threaded very well. I think I turned it into 5.5 days on a Q9450, or just short of 5 days on a dual X5580. It is insanely slow, and that was only on settings="medium". Test material was the Tayutama bluray if anyone cares.

[Phantasmagoriat]

I just used it's default... So I used
TTempSmooth(maxr=3, lthresh=4, cthresh=5, lmdiff=2, cmdiff=3, strength=2, scthresh=12.0, fp=true, vis_blur=0, debug=false, interlaced=false, pfclip=?)

Except in place of ? I had put null (because it told me to...)

Oh, I see what error you are getting... just type:

Code: Select all

TTempSmooth()

That's it. All the defaults will load automatically. Alternatively, you could substitute the ? with the word: last
That way, you are at least offering TTempSmooth a clip to use for comparison.

Did you check if you actually have libfftw3f-3.dll in your C:\Windows\System32 folder?

Yes, I do. I have libfftw3f-3.def plus pairs of two others that are very similar (came with a pack).

hmm... are you running a 64-bit OS? You might need to copy it into your windows SysWOW64 folder [whoever named that directory is awesome ]

Thank you for listing those. I'll definitely check them all out and write it down. And no, I'm not gonna post a lossless clip I'm sure this just takes a lot of trial and error to get right.

Yeah, you seem to get the idea And if you want to install/experiment with new filters [deblock_qed() is another one...], just follow the directions in that link I posted earlier, but google the documentation in case of special instructions.

Thanks for all that help Phantasmagoriat. Your username is also very fun to type.

No problem some people call me phan 'cuz it's easier... or some variation thereof... it's an old username I think I'm outgrowing... but really I don't care as long as I know you're talking to me



Also, I'll echo what Hatt was saying about filtering. Try to err on the side of underfiltering. That way, all the details gets preserved.

[mirkosp]

http://www.lagom.nl/lcd-test/

Before going on with the tests, make sure your monitor is properly calibrated... it might be showing more issues than the ones that are really there.