Looking for suggestions on End of Evangelion improvement

This forum is for questions and discussion of all the aspects of handling your footage. If you have questions about capturing/ripping footage, AviSynth, or compression/encoding/converting, look here.

Looking for suggestions on End of Evangelion improvement

Postby wightwizard8 » Sun Jun 20, 2010 11:05 pm

I'm working on filtering and improving my rip of End of Evangelion (which is unfortunately the R1 version) in AviSynth, and I've got a fairly good-looking version by my standards, but my only problem is that the final result looks a little too "cel-shaded-ish", for lack of a proper adjective.

This is the script I'm using:
Code: Select all
Import("C:\Program Files\AviSynth 2.5\plugins\FastLineDarken.avs")
mpeg2source("X:\Source\VTS_01_1.d2v",cpu=4)
AMVDeint(mode=3)
SmoothD(quant=1, num_shift=4, adaptive_shift=0, zero_weight=1)
DeRainbow(10)
aWarpSharp()
LimitedSharpenFaster(strength=200)
FastLineDarken(strength=50,luma_cap=225,thinning=0)
Crop(8,0,-8,0).Spline36Resize(640,480).Crop(8,16,-14,-16).Spline36Resize(640,480)
ConvertToRGB32()

This is using filters that are found either in the AMVApp or in Scintilla's guide to postprocessing here: http://www.aquilinestudios.org/avsfilters/index.html.

Here's an example:
Original - http://yfrog.com/0kendofeva64627originalfrp
Edited - http://yfrog.com/msendofeva64627remasteredp

I'm concerned about the reduction in detail, especially in the trees, but I'm not sure how to keep the detail and still smooth out all the ugly.
If anyone has any suggestions on how to make it look better, please let me know. Thanks for the input!
wightwizard8
 
Joined: 10 Aug 2009

Re: Looking for suggestions on End of Evangelion improvement

Postby Mister Hatt » Sun Jun 20, 2010 11:57 pm

Is it even interlaced as such, or is it telecined? I forget.

To be brutally honest: your original looks FAR better than your filtered. The filtered one has MASSIVE chroma blurring, is oversmoothed, straight lines are deformed hideously, and it's oversharpened. I have no idea why you double crop+resized on it either.

If I was filtering, ignoring interlacing status, I would start with some really minor derainbowing, although at this time I'm not sure what I would use to do so. I would then removegrain(1) it and possible run some dirt removal, otherwise I would do an fft3d contrasharpen followed by EE(). If it really needs it I guess some freq domain smoothing is in order but I doubt it. The aliasing you have should go away with proper treatment of the fields coding depending on how it's done. It might be better to derainbow post-noise filtering though, depends on which filter you use however.
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Looking for suggestions on End of Evangelion improvement

Postby wightwizard8 » Mon Jun 21, 2010 12:49 am

Yes, it's interlaced, at least the version I have is, and there's also a lot of black space around the video that I didn't want to have in the final product. So instead of doing the math, I just cropped on top of the first crop that was recommended in Zarxrax's "Guide to all things audio and video" (8 pixels from left and right and resize to 4:3).

Thanks for the critique, but your suggested solution has a lot of methods that I can't find, don't seem to do anything, or can't get to work. Could you point me to where I could find out how to get the FFT3D filters working, or tell me what specific filters you'd recommend, as far as dirt removal? And what is EE()?

This is my first time doing anything like this, so thanks for the help.
wightwizard8
 
Joined: 10 Aug 2009

Re: Looking for suggestions on End of Evangelion improvement

Postby Mister Hatt » Mon Jun 21, 2010 1:06 am

EE() is a filter that is part of Kintaro's Useless Filterscripts, you can google for it or I can upload it later. FFT3D filters are called FFT3DFilter or FFT3DGPU depending on whether you have access to DirectX9 or not. RemoveDirt is a pretty decent filter but you could try undot or using FFT3D for it, or even removegrain is capable of doing similar. ContraSharpening is kind of annoying and I don't think I could explain it here well enough.

And again, are you sure it's interlaced and not telecined? Telecine video is encoded interlaced but it isn't same-field matching like regular interlacing. I have never heard of End of Eva being interlaced.
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Looking for suggestions on End of Evangelion improvement

Postby mirkosp » Mon Jun 21, 2010 6:27 am

Mister Hatt wrote:EE() is a filter that is part of Kintaro's Useless Filterscripts, you can google for it or I can upload it later.

http://kaze-tenshu.pastebin.com/f91ndpGM
function EE(clip clip, int "str", string "cspace"){

str = default(str,80)
cspace = default(cspace,"useless")

yv12 = clip.isYV12() ? clip : clip.converttoyv12()
aa_clip = yv12.LanczosResize(width(clip)*2,height(clip)*2). \
UnsharpMask(strength=str,radius=2,threshold=2). \
LanczosResize(width(clip),height(clip))
maske = yv12.EdgeMask(7,7,5,5,"sobel").inflate()
semi = MaskedMerge(yv12,aa_clip,maske)
semi = overlay(yv12,semi,mode="darken")

aa_clip2 = semi.LanczosResize(width(clip)*2,height(clip)*2). \
TurnLeft().SangNom().TurnRight().SangNom(). \
LanczosResize(width(clip),height(clip))
maske2 = semi.EdgeMask(7,7,5,5,"sobel").inflate()
new = MaskedMerge(semi,aa_clip2,maske2)
new = overlay(semi,new,mode="darken").MergeChroma(yv12)

orig_cspace = clip.isRGB32() ? new.converttoRGB32() :
\ clip.isRGB24() ? new.converttoRGB24() :
\ clip.isYUY2() ? new.converttoYUY2() :
\ new

final = (cspace == "YV12") ? new :
\ (cspace == "YUY2") ? new.converttoYUY2() :
\ (cspace == "RGB24") ? new.converttoRGB24() :
\ (cspace == "RGB32") ? new.converttoRGB32() :
\ orig_cspace

return final }

It's still available on Kaze-Tenshu's pastebin! :rofl:

As for FFT3D: http://forum.doom9.org/showthread.php?t=89941 get the exe of the version 0.8.2 and follow the easy installation steps. It's FFT3DGPU, 'cause I don't feel like writing up how to "install" fft3dfilter right now and 'cause if you have a proper gpu it's faster... :uhoh:
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Looking for suggestions on End of Evangelion improvement

Postby Mister Hatt » Mon Jun 21, 2010 9:06 am

Incidentally I pastebinned it to Tenshu like a year ago on IRC, lol :3

Also incidentally, if you use a better compiled version of FFTW but the same lib version as what is packaged, you can run FFT3DFilter significantly faster. I might poke some people for actual specs. Compiling is good for you.
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Looking for suggestions on End of Evangelion improvement

Postby wightwizard8 » Mon Jun 21, 2010 11:14 am

Thanks for all the help!

But unfortunately, I've still got problems.

Here's my code now:
Code: Select all
Import("C:\Program Files\AviSynth 2.5\plugins\EE.avs")
mpeg2source("X:\Source\VTS_01_1.d2v",cpu=4)
Checkmate()
AMVDeint(mode=3)
EE()
DeRainbow()
FFT3DGPU()
Crop(8,0,-8,0).Spline36Resize(640,480).Crop(8,16,-14,-16).Spline36Resize(640,480)
ConvertToRGB32()


And here's what it gets me:
Original - http://img713.imageshack.us/img713/4885 ... iginal.png
Filtered - http://img706.imageshack.us/img706/6484 ... ltered.png

Now the main problems I've got are the fact Unit 02's red color seeps into everything around it, like you can see along her arms, around her head and shoulders, and around the handle of the other blade. There's also still a lot of noise in the flat spaces and around the edges, is there some sort of smoother that won't over-blur it? Is there some parameter in one of the filters I'm using that I should change? And what exactly does EE() do? I can tell it's doing something, but it's such a tiny change that I can't tell what it's doing.
wightwizard8
 
Joined: 10 Aug 2009

Re: Looking for suggestions on End of Evangelion improvement

Postby wightwizard8 » Mon Jun 21, 2010 11:16 am

Okay, sorry, just figured out how to stop the bleeding - the default Rainbow() settings are too strong. But the rest of the problems still apply.
wightwizard8
 
Joined: 10 Aug 2009

Re: Looking for suggestions on End of Evangelion improvement

Postby mirkosp » Mon Jun 21, 2010 11:35 am

Don't call fft3dgpu like that. Try to call it like so, instead:
Code: Select all
strength = 0.5
fft3dfilter(bw=6, bh=6, ow=3, oh=3, plane=0, bt=1, sigma=strength)
fft3dfilter(bw=216, bh=216, ow=108, oh=108, plane=0, bt=1, sigma=strength/8, sigma2=strength/4, sigma3=strength/2, sigma4=strength)

Adjust the strength as needed. The default would be 2, but since apparently 0.8 is enough for rather noisy anime, I think 0.5 should be a good starting point for eva.
Also, EE should go after the denoising and derainbowing.
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Looking for suggestions on End of Evangelion improvement

Postby Mister Hatt » Tue Jun 22, 2010 7:45 am

Crop/resize before filtering. You don't need checkmate(). Put EE() after everything else. The last line should be the ConvertToRGB one but tell it what matrix to use. Are you sure you need RGB32 and not RGB24? It will make avisynth use 30% more RAM than necessary, and you don't have an alpha channel to begin with anyway. You are having SEVERE chroma bleeding issues, find out why. You don't need AMVDeint, do it properly. You still haven't confirmed whether this is telecine or pure interlacing. Do so.
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Looking for suggestions on End of Evangelion improvement

Postby mirkosp » Tue Jun 22, 2010 10:17 am

Mister Hatt wrote:You are having SEVERE chroma bleeding issues, find out why.

It was derainbow apparently.
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Looking for suggestions on End of Evangelion improvement

Postby Mister Hatt » Tue Jun 22, 2010 10:58 am

Actually I don't think it was. I wouldn't put it past blurring and then interpolating the blur, which is what at a guess checkmate().amvdeint() is doing.
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Looking for suggestions on End of Evangelion improvement

Postby mirkosp » Tue Jun 22, 2010 11:02 am

amvdeint(mode=3) is TDeint(mode=2, mtnmode=3, blim=100) so it should be fine...?
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Looking for suggestions on End of Evangelion improvement

Postby Scintilla » Tue Jun 22, 2010 5:04 pm

Mister Hatt wrote:Are you sure you need RGB32 and not RGB24? It will make avisynth use 30% more RAM than necessary, and you don't have an alpha channel to begin with anyway.

From the AVISynth documentation:

The documenters wrote:RGB24 data is often much slower to process than RGB32 data, so if your source is RGB24, you may get a speed gain by converting to RGB32. There are no known advantages of using RGB24 except that TMPGEnc and VFApi requires RGB24 input).

Probably because each pixel is one 32-bit word instead of having to be split...?
ImageImage
:pizza: :pizza: Image :pizza: :pizza:
User avatar
Scintilla
(for EXTREME)
 
Joined: 31 Mar 2003
Location: New Jersey
Status: Quo

Re: Looking for suggestions on End of Evangelion improvement

Postby wightwizard8 » Mon Jul 05, 2010 2:13 pm

Well, thanks for all your help everyone. I finished the AMV I used that footage in, if you're interested in watching it - you can find it here: http://www.animemusicvideos.org/members/members_videoinfo.php?v=177150.

Again, thanks for all the help and suggestions.
wightwizard8
 
Joined: 10 Aug 2009


Return to Footage Help

Who is online

Users browsing this forum: No registered users and 2 guests