Artifacts in Anime/Toon Footage

This forum is for questions and discussion of all the aspects of handling your footage. If you have questions about capturing/ripping footage, AviSynth, or compression/encoding/converting, look here.

Re: Artifacts in Anime/Toon Footage

Postby Anno-san » Thu Sep 16, 2010 12:23 pm

Haali renderer might be slightly better. I can't really tell.

Some questions.

1. If haali eliminated the chroma error should there be no jagged edges at all? I still see some.

2. What would happen if I encoded the video anyway?

3. Am I only seeing the artifacts on MY screen, so that if others viewed the same video on their computers they WOULDN'T see the problem? Or is it an issue with the source file itself?

4.Renderers only affect how the video looks in MPC, correct? Or does a renderer affect how video will look when it is copied to the computer via DVDDecrypter?
User avatar
Anno-san
 
Joined: 24 Dec 2009

Re: Artifacts in Anime/Toon Footage

Postby Mister Hatt » Thu Sep 16, 2010 12:45 pm

Why do you keep saying you don't know what the problem is and that people are giving you contradictory advice? I've fully explained exactly what it is AND told you what to do about it. Every decent encoder I threw your screenshots to said exactly what I already have. The chroma subsampling error (there is no UPSAMPLING error at all and anyone who says it obviously doesn't know what upsample means) is part of the footage and your splitter choice will not fix it.

Renderers affect how your GFX card outputs each pixel. It has nothing to do with software for the most part. Rather it takes colours, applies display operations to figure out what each should be represented as in output, and then outputs it. It's done mostly on the hardware side by your video driver; however different renderers in your player will send slightly different information to your hardware.
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Artifacts in Anime/Toon Footage

Postby Anno-san » Fri Sep 17, 2010 7:56 am

@Mister Hatt

So the problem can be fixed with either Deblock_QED etc. or Edgecleaner or another masking filter? I'd prefer to solve the issue in as little lines as possible. If your script really was 4000 lines, I clearly can't do that as I have only been using AviSynth for a > a year.
User avatar
Anno-san
 
Joined: 24 Dec 2009

Re: Artifacts in Anime/Toon Footage

Postby Anno-san » Fri Sep 17, 2010 8:54 am

Also, I have Edgecleaner v. 1.03 and when I try to run it it give the error "there is no function called aWarpsharp". Usually this error almost always means that the wrong version of said filter (in this case aWarpsharp) is being used. So what aWarpsharp do I need.
User avatar
Anno-san
 
Joined: 24 Dec 2009

Re: Artifacts in Anime/Toon Footage

Postby Anno-san » Fri Sep 17, 2010 10:18 am

Mister Hatt wrote:The other problem with the red jaggies/blocking on line edges is due to terrible chroma subsampling. The chroma issue can be fixed with EdgeCleaner I guess, although contrasharpening might be a better option.


Edgecleaner did not help much. what is contrasharpening and what filters can do this?

Mister Hatt wrote:The blocking you need to find the golden settings of whatever your favourite deblocker is but I would consider dfttest masking or highly tweaked FFT3D as another option; I ended up using a dfttest mask on Cobra which had similar issues and it turned out pretty sugoi.


What were you saying about finding the chroma's frequency? Is that the sigma? Basically, will FFT3DFilter fix the jagged chroma edges on red objects or will EdgeCleaner? If FFT3D then what parameters?
User avatar
Anno-san
 
Joined: 24 Dec 2009

Re: Artifacts in Anime/Toon Footage

Postby Mister Hatt » Sat Sep 18, 2010 4:51 am

The sigmas in frequency domain filters are the frequency sums, yes. DFTTest is a somewhat global sigma, while FFT3D lets you define smoothing strengths for various sigmas. The default in FFT3D is for larger blocking, about what you'd be throwing Deblock_QED_MT2 at. Sigma2 is for slightly smaller blocks, common in poor quantization. Sigma3 is probably the one you want; it's for stuff that is dirt or larger grain sized. Sigma4 is for mosquito noise. I guess something like FFT3DFilter(sigma=0.6, sigma2=1.2, sigma3=2, sigma4=0.4, bt=4, plane=4) should so it.

Contrasharpening is a method of smoothing detail as little as possible on edges, then masking it and applying a difference mask to the unfiltered video. It basically cleans up blocky edges. You can get it here. EdgeCleaner is probably giving you the error because you don't have awarpsharp loaded. You might also have a version it doesn't like, and tbh if that happened then you get what you deserve for being dumb and auto-loading standard filters which should NEVER be done.

I don't think you need Deblock_QED at all really. I'd just find a lowish sigma and dft nuke it or something. If you really care though, your best bet is probably a two-stage FFT3D followed by some EE or something just to clean it up a bit. I would avoid lsfmod but my reluctance to use that filter is based more on a personal dislike for it than any justifiable problems it has.

I just checked my script, it is 4091 lines long but I hope you don't think I wrote that manually. It was created with YATTA of course; how else would one make black magic so agreeable?
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Artifacts in Anime/Toon Footage

Postby Anno-san » Sat Sep 18, 2010 1:17 pm

Here is a better sample of the footage.
User avatar
Anno-san
 
Joined: 24 Dec 2009

Re: Artifacts in Anime/Toon Footage

Postby Mister Hatt » Sat Sep 18, 2010 1:21 pm

Direct link please. Or someone else like mirko can take a look. This isn't really interesting enough for me to go out of my way and follow crappy-download-provider-x links.
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Artifacts in Anime/Toon Footage

Postby mirkosp » Sat Sep 18, 2010 2:15 pm

I'm downloading it, but it has no extension. Is this an m2v or what? I'll reupload it to voodoo-powered with an extension, so Hatt won't have excuses to not take a look at it. :asd:
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Artifacts in Anime/Toon Footage

Postby Anno-san » Sat Sep 18, 2010 6:46 pm

mirkosp wrote:I'm downloading it, but it has no extension. Is this an m2v or what?


Source VOB.
User avatar
Anno-san
 
Joined: 24 Dec 2009

Re: Artifacts in Anime/Toon Footage

Postby mirkosp » Sat Sep 18, 2010 7:30 pm

Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Artifacts in Anime/Toon Footage

Postby Anno-san » Sat Sep 18, 2010 10:19 pm

Yes. That sample has the scenes that I took the images from. It will be much easier for others to test out the filters that way.
User avatar
Anno-san
 
Joined: 24 Dec 2009

Re: Artifacts in Anime/Toon Footage

Postby Anno-san » Sun Sep 19, 2010 8:41 pm

Has anyone tried a script on the new sample to eliminate the chroma jaggies?
User avatar
Anno-san
 
Joined: 24 Dec 2009

Re: Artifacts in Anime/Toon Footage

Postby mirkosp » Sun Sep 19, 2010 9:01 pm

I actually didn't get any chroma subsampling error with the source... even if I just try to playback the vob it looks fine (well, as far as subsampling goes, duh :P ). I didn't post any earlier because I was hoping for Mister Hatt to have a better insight on what might causing the issue on your end, but he's having some issues with some of his software atm from what I understood. As for me I don't really know what else to say. There probably is something wrong with your playback setup but I can't quite put my finger on what since it's not about the renderer itself, apparently.
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Artifacts in Anime/Toon Footage

Postby Anno-san » Sun Sep 19, 2010 9:20 pm

mirkosp wrote:I actually didn't get any chroma subsampling error with the source... even if I just try to playback the vob it looks fine (well, as far as subsampling goes, duh :P ). I didn't post any earlier because I was hoping for Mister Hatt to have a better insight on what might causing the issue on your end, but he's having some issues with some of his software atm from what I understood. As for me I don't really know what else to say. There probably is something wrong with your playback setup but I can't quite put my finger on what since it's not about the renderer itself, apparently.


Does it matter if the processor is 64 bit but I am using the 32 bit version of AviSynth and all the filters? Is it some problem with the graphics card? It's an ATI Radeon 3200 HD.
User avatar
Anno-san
 
Joined: 24 Dec 2009

PreviousNext

Return to Footage Help

Who is online

Users browsing this forum: No registered users and 2 guests