Fixing grain and deinterlacing? (Code Geass)

This forum is for questions and discussion of all the aspects of handling your footage. If you have questions about capturing/ripping footage, AviSynth, or compression/encoding/converting, look here.

Re: Fixing grain and deinterlacing? (Code Geass)

Postby Dext3r » Wed Jul 24, 2013 4:10 am

mirkosp wrote:
Also how do I get the HQDering filter to work? I've found the .dll's for SmoothD2 and NLMeansCL (given they're right in those threads); but for HQDering all there is is that wiki page with the source code?


HQDering is a script, so you make an .avs file, paste the code you can find in that page inside, save the .avs, rename the extension to .avsi and place the .avsi in the plugins folder of avisynth.

Also you don't have to wait the whole preview (if you mean File > Preview)... that's just slow. You can directly do File > Save Project. That just indexes as fast as your cpu allows, so it won't take too long. Take the screenshot after it's done.


Ok, I setup the HQDering script now; and I'm just waiting for my comp. to index the file.
Dext3r
 
Joined: 18 May 2011

Re: Fixing grain and deinterlacing? (Code Geass)

Postby l33tmeatwad » Wed Jul 24, 2013 4:17 am

Dext3r wrote:
mirkosp wrote:OT, but it might be useful in future: if you hold Alt while pressing Stamp, it'll only save the highlighted panel (so you could make a screenshot of just the information panel).

Anyway, that information panel looks indeed like a soft pulldown source. Which would mean force film should be the correct choice.
However you made the screenshot before it finished indexing the whole thing, so it would be better if you took a screenshot of after the whole source has been indexed. There might be some stray 60i bit amidst the soft pulldown footage, so that might be the cause.
If not, then I'd assume the green issue is either a mastering issue, or a MPEG-2 fuckup.


Oh christ, now you're just making me feel bad, I'm supposed to wait for that whole preview to finish? XD

I've NEVER waited for the DGIndex preview to finish, I only ever made sure I wasn't looking at the opening or ending (for obvious reasons since the opening and ending of the show can be running at different settings, like in Code Geass the opening is interlace).
Just to add on (I know the conversation moved a bit past this already), you definitely should wait until you preview the entire video before selecting Force Film because you never know when there could be mixed soft and hard telecine material. If you are coming across issues, handling all the interlacing with AviSynth is the safer choice. Also, as mentioned by mirkosp, there is always the possibility that it's a mastering or encoding problem (which is quite common for a lot of anime).
Guides, Scripts, & Software: AMV 101 | AMVpack | AviSynth 101 | Anime Upscale
PixelBlended Studios: Website | Facebook | Tumblr | Twitter | YouTube
User avatar
l33tmeatwad
 
Joined: 16 Feb 2005
Location: Charleston, SC
Status: Trolling

Re: Fixing grain and deinterlacing? (Code Geass)

Postby Dext3r » Wed Jul 24, 2013 4:32 am

Ok so a few things.

Here's the settings after its entirely indexed (Forced Film)

Image

Another slight problem I've noticed is apparently cropping or something strange going on in the edges of my source (particularly on the top, and whatever all that is going on the right side as well).

Image

Are these problems I'm causing myself with my other unnecessary filters you already pointed out?
Dext3r
 
Joined: 18 May 2011

Re: Fixing grain and deinterlacing? (Code Geass)

Postby Dext3r » Wed Jul 24, 2013 4:34 am

Also, for that second one, make sure you pull up the image in a different window so you can see what's going on the right side.
Dext3r
 
Joined: 18 May 2011

Re: Fixing grain and deinterlacing? (Code Geass)

Postby l33tmeatwad » Wed Jul 24, 2013 4:48 am

That is normal for most videos, they often have "blank space" on some sides that you will need to crop off. Just keep in mind, when you crop, crop evenly to avoid distorting the image when you resize it. The amount cropped from the top & bottom added together should equal that of what is cropped from the left & right added together. The parts of the video you are seeing are usually outside of the "TV safe" area so you won't see it on most TV sets as it naturally cut it off.
Guides, Scripts, & Software: AMV 101 | AMVpack | AviSynth 101 | Anime Upscale
PixelBlended Studios: Website | Facebook | Tumblr | Twitter | YouTube
User avatar
l33tmeatwad
 
Joined: 16 Feb 2005
Location: Charleston, SC
Status: Trolling

Re: Fixing grain and deinterlacing? (Code Geass)

Postby Dext3r » Wed Jul 24, 2013 4:55 am

ok, I'll play around with that a bit to get rid of those parts.

Still waiting to hear back from @mirkosp about the others though.
Dext3r
 
Joined: 18 May 2011

Re: Fixing grain and deinterlacing? (Code Geass)

Postby mirkosp » Wed Jul 24, 2013 7:07 am

The information box you need to show us is the one of honor pulldown, as that is the reliable one.
That said, even with that forced film information, you can see it's not 100% film. 99.85% is very high though so, if at the end of the video you have something like staff credits for this particular dvd release or some studio logo, that could be what's offsetting it. Otherwise, perhaps one or two scenes might not be film, in which case I'll have to ask you whether the chroma issue you pointed out in the beginning happens all over the place or just in that specific scene.
I have to admit, though, that I find it likely that it is some sort of encoding problem either on master or on the DVD. If it's just there you might just find it easier to deal with by opening photoshop and manually fixing the frames, in case you need to use that scene in your AMV.

Now onto the other issue. As I explained, nlmeanscl uses opencl to run faster than it would on the cpu. Assuming you placed the cloo and avsfilternet .dll files in the plugins folder as well, then I'd guess that perhaps your GPU doesn't have OpenCL support (which is quite likely if you have just on-board graphics instead of a dedicated card). You could run the filter on the cpu but at that point it'd be quite slow. Since blocking is the only issue at hand, I think smoothd2 should be enough. I'd suggest properly setting the qtype value of smoothd2, btw, as that tremendously helps out the resulting efficiency of the filter.
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Fixing grain and deinterlacing? (Code Geass)

Postby Dext3r » Wed Jul 24, 2013 3:02 pm

mirkosp wrote:The information box you need to show us is the one of honor pulldown, as that is the reliable one.
That said, even with that forced film information, you can see it's not 100% film. 99.85% is very high though so, if at the end of the video you have something like staff credits for this particular dvd release or some studio logo, that could be what's offsetting it. Otherwise, perhaps one or two scenes might not be film, in which case I'll have to ask you whether the chroma issue you pointed out in the beginning happens all over the place or just in that specific scene.
I have to admit, though, that I find it likely that it is some sort of encoding problem either on master or on the DVD. If it's just there you might just find it easier to deal with by opening photoshop and manually fixing the frames, in case you need to use that scene in your AMV.

Now onto the other issue. As I explained, nlmeanscl uses opencl to run faster than it would on the cpu. Assuming you placed the cloo and avsfilternet .dll files in the plugins folder as well, then I'd guess that perhaps your GPU doesn't have OpenCL support (which is quite likely if you have just on-board graphics instead of a dedicated card). You could run the filter on the cpu but at that point it'd be quite slow. Since blocking is the only issue at hand, I think smoothd2 should be enough. I'd suggest properly setting the qtype value of smoothd2, btw, as that tremendously helps out the resulting efficiency of the filter.


Oh, I didn't think about how since I use honor pulldown flags for the preview, I obviously need to use it to get the accurate results in the end too.

I have an Nvidia 480 GTX card, so I should definitely have a card that has that kind of support. So if I just use SmoothD2 and HQDering; how do I use SmoothD2 specifically then for that given it has so many parameters? Is it just something like SmoothD2(10)?

Or do I have to account for some of the others too?
Dext3r
 
Joined: 18 May 2011

Re: Fixing grain and deinterlacing? (Code Geass)

Postby mirkosp » Wed Jul 24, 2013 3:54 pm

Yeah your card should definitely be supported... I'll just blame it on opencl in avisynth being buggy. It's not too important anyway.
As for smoothd2, the documentation is pretty good.
As I said, the qtype parameter's pretty useful.
Qtype (1, 2, 3, 4): Quant Type. Selects the type of quant-dequant processing that is used in the filtering. They are ordered by their deblocking strength from least strong to strongest. h263 intra and h263 inter, Qtype’s 2 and 4, ignore the matrix argument and use their own built in matrix.

1. mpeg intra
2. h263 intra
3. mpeg inter
4. h263 inter
The intra in Qtypes 1 and 2 referes to the encoding done for I-Frames. The inter in Qtypes 3 and 4 refers to the encoding done for P-frames and B-Frames. See http://www.fh-friedberg.de/fachbereiche ... ginnzi.htm for a good introduction to mpeg coding. Different matrices are usually used when encoding I-Frames then when encoding P-Frames and B-Frames. All the information about how a particular frame was originally encoded, I-Frame, P-Frame, B-Frame, and Matrix value, has been lost by the time SmoothD2 is passed a frame to process, SmoothD2 does not do automatic intra-inter Qtype or matrix switching.

Generally you are supposed to manually set the qtype for the frame, but if you don't feel like spending all that effort, in the case of mpeg-2 DVDs, set it to 3 (which is mpeg inter).
That way it'll be much more effective in removing the blocking and mosquito noise at lower quant values, which is its filtering strength parameter. The default of 3 might not be enough for some of the most action intense scenes, but I really would suggest you to at least filter only those scenes at that point, since filtering clean scenes will just ruin them.
As for the other parameters, feel free to read the documentation, but I personally haven't found to be nearly as necessary and useful as these two, though I guess in some cases the matrix parameter could make wonders.
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Fixing grain and deinterlacing? (Code Geass)

Postby Dext3r » Wed Jul 24, 2013 4:11 pm

mirkosp wrote:Yeah your card should definitely be supported... I'll just blame it on opencl in avisynth being buggy. It's not too important anyway.
As for smoothd2, the documentation is pretty good.
As I said, the qtype parameter's pretty useful.
Qtype (1, 2, 3, 4): Quant Type. Selects the type of quant-dequant processing that is used in the filtering. They are ordered by their deblocking strength from least strong to strongest. h263 intra and h263 inter, Qtype’s 2 and 4, ignore the matrix argument and use their own built in matrix.

1. mpeg intra
2. h263 intra
3. mpeg inter
4. h263 inter
The intra in Qtypes 1 and 2 referes to the encoding done for I-Frames. The inter in Qtypes 3 and 4 refers to the encoding done for P-frames and B-Frames. See http://www.fh-friedberg.de/fachbereiche ... ginnzi.htm for a good introduction to mpeg coding. Different matrices are usually used when encoding I-Frames then when encoding P-Frames and B-Frames. All the information about how a particular frame was originally encoded, I-Frame, P-Frame, B-Frame, and Matrix value, has been lost by the time SmoothD2 is passed a frame to process, SmoothD2 does not do automatic intra-inter Qtype or matrix switching.

Generally you are supposed to manually set the qtype for the frame, but if you don't feel like spending all that effort, in the case of mpeg-2 DVDs, set it to 3 (which is mpeg inter).
That way it'll be much more effective in removing the blocking and mosquito noise at lower quant values, which is its filtering strength parameter. The default of 3 might not be enough for some of the most action intense scenes, but I really would suggest you to at least filter only those scenes at that point, since filtering clean scenes will just ruin them.
As for the other parameters, feel free to read the documentation, but I personally haven't found to be nearly as necessary and useful as these two, though I guess in some cases the matrix parameter could make wonders.


Ok awesome, so how should I put my SmoothD2 command then? Because the first parameter is Clip or ClipZmask, so I can't just skip that one to get to the quantum?
Dext3r
 
Joined: 18 May 2011

Re: Fixing grain and deinterlacing? (Code Geass)

Postby Dext3r » Wed Jul 24, 2013 4:16 pm

Essentially what I'm saying is how do I just use the default value for a parameter?
Dext3r
 
Joined: 18 May 2011

Re: Fixing grain and deinterlacing? (Code Geass)

Postby mirkosp » Wed Jul 24, 2013 4:28 pm

Code: Select all
smoothd2(qtype=3)


By specifying a parameter's name, you can just change that one. The other parameters will retain their default value.
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Fixing grain and deinterlacing? (Code Geass)

Postby Dext3r » Wed Jul 24, 2013 4:54 pm

mirkosp wrote:
Code: Select all
smoothd2(qtype=3)


By specifying a parameter's name, you can just change that one. The other parameters will retain their default value.


Alright, now when I try to run HQDering, even though I made that .avsi file, it says there's no function called "Edgemask".

Am I missing something else for that one?
Dext3r
 
Joined: 18 May 2011

Re: Fixing grain and deinterlacing? (Code Geass)

Postby l33tmeatwad » Wed Jul 24, 2013 6:32 pm

Dext3r wrote:
mirkosp wrote:
Code: Select all
smoothd2(qtype=3)


By specifying a parameter's name, you can just change that one. The other parameters will retain their default value.


Alright, now when I try to run HQDering, even though I made that .avsi file, it says there's no function called "Edgemask".

Am I missing something else for that one?

It's part of masktools: http://avisynth.org/mediawiki/MaskTools
Guides, Scripts, & Software: AMV 101 | AMVpack | AviSynth 101 | Anime Upscale
PixelBlended Studios: Website | Facebook | Tumblr | Twitter | YouTube
User avatar
l33tmeatwad
 
Joined: 16 Feb 2005
Location: Charleston, SC
Status: Trolling

Re: Fixing grain and deinterlacing? (Code Geass)

Postby Dext3r » Wed Jul 24, 2013 7:50 pm

l33tmeatwad wrote:
Dext3r wrote:
mirkosp wrote:
Code: Select all
smoothd2(qtype=3)


By specifying a parameter's name, you can just change that one. The other parameters will retain their default value.


Alright, now when I try to run HQDering, even though I made that .avsi file, it says there's no function called "Edgemask".

Am I missing something else for that one?

It's part of masktools: http://avisynth.org/mediawiki/MaskTools


Yeesh its like one problem causes another! Now its asking about a YV12Layer function, which says its contained within the Masktools, but even though I added the .dll file, that's not enough though apparently.

Do I need to extract the MaskTools folder as well?
Dext3r
 
Joined: 18 May 2011

PreviousNext

Return to Footage Help

Who is online

Users browsing this forum: No registered users and 1 guest