Cleaning up Outlaw Star

This forum is for questions and discussion of all the aspects of handling your footage. If you have questions about capturing/ripping footage, AviSynth, or compression/encoding/converting, look here.

Cleaning up Outlaw Star

Postby Cannonaire » Sun May 09, 2010 7:47 pm

Hey everyone. For my second AMV project I'm making an experiment vid using Outlaw Star (not to be released most likely) to play around with things like lip sync and effects, but afterward I'm going to make a real video using the same ripped footage. I've gotten my clips to look pretty good, but there is some really annoying dot crawl still, most noticeable around lips, which is a big problem as learning lip syncing is one of the goals. If anyone has experience cleaning up this anime, your help/opinions would be appreciated.

I only just learned to use AviSynth a few days ago, but it helped tremendously for cleaning up the source material for my first AMV (Final Fantasy 9, not released quite yet). I've tried using Checkmate(), Tcomb(), and Dedot() according to the post-production guide here: http://www.animemusicvideos.org/guides/ ... l#dotcrawl
None of these really worked for the dot crawl I'm seeing unless you cranked them way up (like using 999 on Checkmate), but that obviously causes more problems than it solves.
If anyone else has ripped Outlaw Star footage, you can see the dot crawl specifically on ***first episode spoiler*** Hilda's lips when she's talking to Jim on the phone in disguise. ***end spoiler***. It's kind of slight in this example, but it persists throughout the series, better or worse in different scenes.

Here is my script for the moment, and it does a pretty good job for everything else, but I'm not opposed to changing it, even completely. I'm new, so all good advice is welcome. Thanks in advance.

mpeg2source("J:\video_rip\outlawstar\ep1a.d2v",cpu=6)
#checkmate(24)
AMVIVTC(mode=1)
Crop(8, 0, -8, -0)
LanczosResize(624, 480)
DeRainbow(10)
Toon()
Image Think millionaire, but with cannons. || Resident Maaya Sakamoto fan.
User avatar
Cannonaire
 
Joined: 05 May 2010
Location: Oregon
Status: OVERLOAD

Re: Cleaning up Outlaw Star

Postby Scintilla » Sun May 09, 2010 8:05 pm

You could give my own solution, LUTDeCrawl, a try (though if TComb, Checkmate, AND DeDot couldn't fix it, I'm not optimistic that mine will either).

Aside from that, I wouldn't know what more specifically to suggest without seeing a clip of your source (for most things a few screenshots will do, but as you know, dot crawl has to be examined in the context of full motion).
ImageImage
:pizza: :pizza: Image :pizza: :pizza:
User avatar
Scintilla
(for EXTREME)
 
Joined: 31 Mar 2003
Location: New Jersey
Status: Quo

Re: Cleaning up Outlaw Star

Postby Cannonaire » Sun May 09, 2010 9:01 pm

Thanks for the response. A bit later I'll find some web hosting and take a few screencaps, and try out your solution.
Image Think millionaire, but with cannons. || Resident Maaya Sakamoto fan.
User avatar
Cannonaire
 
Joined: 05 May 2010
Location: Oregon
Status: OVERLOAD

Re: Cleaning up Outlaw Star

Postby Kariudo » Sun May 09, 2010 9:39 pm

What were your settings for Tcomb?
I was able to clean up some pretty epic dotcrawl using it
ImageImage
Image
User avatar
Kariudo
Twilight prince
 
Joined: 15 Jul 2005
Location: Los taquitos unidos
Status: 1924 bots banned and counting!

Re: Cleaning up Outlaw Star

Postby Cannonaire » Sun May 09, 2010 9:42 pm

That's incredible dot crawl removal. :shock:
To be honest, I spent about a half hour fiddling around with it but couldn't tell the difference between different settings. Not much documentation that I could find, but from the looks of it that filter would really help me out. Would you be able to help me? I'll see if I can get those screens up sooner rather than later.
Image Think millionaire, but with cannons. || Resident Maaya Sakamoto fan.
User avatar
Cannonaire
 
Joined: 05 May 2010
Location: Oregon
Status: OVERLOAD

Re: Cleaning up Outlaw Star

Postby Kariudo » Sun May 09, 2010 11:57 pm

It's a relatively simple filter to tweak once you know what to do. The readme.txt file that comes with tcomb is pretty good at explaining it.
BASIC SETUP/USAGE:


Setting up TComb is pretty simple. The only values that would ever really need adjusting
are fthreshL/fthreshC, othreshL/othreshC, and mode.

Set mode to 0 if you want to do dot crawl removal only, set it to 1 if you want to
do rainbow removal only, or set it to 2 to do both.

Dot Crawl Removal Tweaking (fthreshL/othreshL):

To find good values for fthreshL/othreshL, start with the following line:

tcomb(mode=0,fthreshL=255,othreshL=255)

Now, keep othreshL at 255 but set fthreshL down to 1. Keep increasing fthreshL
in steps of 1 to 2 until you find the point at which all dot crawl is removed.
Remember that value. Next, set fthreshL back to 255, and set othreshL to 1.
Now, increase othreshL in steps of 1 or 2 until you find the point at which all
dot crawl is removed. You've now got values for fthreshL/othreshL.

Rainbowing Removal Tweaking (fthreshC/othreshC):

To find good values for fthreshC/othreshC, start with the following line:

tcomb(mode=1,fthreshC=255,othreshC=255)

Now, keep othreshC at 255 but set fthreshC down to 1. Keep increasing fthreshC
in steps of 1 to 2 until you find the point at which all (or most) rainbowing is
removed. Remember that value. Next, set fthreshC back to 255, and set othreshC
to 1. Now, increase othreshC in steps of 1 or 2 until you find the point at which
all (or most) rainbowing is removed. You've now got values for fthreshC/othreshC.

Once you've got values for mode, fthreshL/fthreshC, and othreshL/othreshC, add the
necessary tcomb() line into your script and run through part of it. If you see any
artifacts try lowering your fthresh/othresh values.


I ended up using fthreshl=12, othreshl=19 (defaults are 4 and 5 respectively) for fumoffu. I highly doubt that you'll need to go that high though. The documentation says a good range is between 4 - 7 and 4 - 8. The main thing you have to watch out for is really nasty looking artifacts (you'll know it if and when you see it), but if you stay around 12 or less it shouldn't be much of a problem.
ImageImage
Image
User avatar
Kariudo
Twilight prince
 
Joined: 15 Jul 2005
Location: Los taquitos unidos
Status: 1924 bots banned and counting!

Re: Cleaning up Outlaw Star

Postby Cannonaire » Mon May 10, 2010 12:23 am

Good instructions. I tried it out, but no good results. I can get a single frame at a time to look good, but in general I've only succeeded in adding worse artifacts with TComb. :(
Image Think millionaire, but with cannons. || Resident Maaya Sakamoto fan.
User avatar
Cannonaire
 
Joined: 05 May 2010
Location: Oregon
Status: OVERLOAD

Re: Cleaning up Outlaw Star

Postby Cannonaire » Mon May 10, 2010 12:32 am

Here's an example of the dot crawl I'm seeing. It's fairly minor at first glance in this image, except it gets really noticable in motion. It is also worse in other scenes, and to make matters worse one of the entire points for using this footage was to use it for lip syncing. Ripped from the Outlaw Star Perfect Collection Box Set.

Image

Uploaded with ImageShack.us

Zoom 200%(different frame):
Image

Uploaded with ImageShack.us
Image Think millionaire, but with cannons. || Resident Maaya Sakamoto fan.
User avatar
Cannonaire
 
Joined: 05 May 2010
Location: Oregon
Status: OVERLOAD

Re: Cleaning up Outlaw Star

Postby Kariudo » Mon May 10, 2010 12:44 am

hmm, are you using mode=0?
mode=2 would also work, but that also includes de-rainbowing (which it doesn't look like you need).
if you're using mode=1 it won't do too much for you.

Also, TComb works on interlaced footage, so make sure you put it before Telecide(), TFM(), animeIVTC() or whatever else you may use to make your footage progressive.

There is a chance that the dot crawl was there before the footage was telecined (in which case there isn't much that can be done) but I haven't heard of a case where this happened (yet).
ImageImage
Image
User avatar
Kariudo
Twilight prince
 
Joined: 15 Jul 2005
Location: Los taquitos unidos
Status: 1924 bots banned and counting!

Re: Cleaning up Outlaw Star

Postby Cannonaire » Mon May 10, 2010 12:52 am

I've tried mode 0 and 2. Also, I do have it before the interlacing, but it's not really helping. Maybe this is the first case you've seen where it's in the source before it was interlaced? That would really suck. Thanks for all the help though, I'll see what i can do with it. Here's my script as it is currently:

Code: Select all
mpeg2source("J:\video_rip\outlawstar\ep1a.d2v",cpu=6)
#checkmate(24)
Tcomb(mode=0,fthreshl=3, othreshl=8)
AMVIVTC(mode=1)
Crop(8, 0, -8, -0)
LanczosResize(624, 480)
#DeRainbow(10)
#Toon()


It's a shame, cause with the other stuff (commented out in the code) the video looks great aside from the dot crawl.
Image Think millionaire, but with cannons. || Resident Maaya Sakamoto fan.
User avatar
Cannonaire
 
Joined: 05 May 2010
Location: Oregon
Status: OVERLOAD

Re: Cleaning up Outlaw Star

Postby Mister Hatt » Mon May 10, 2010 2:17 am

There was a huge discussion on dotcrawl removal last night in Darkhold. The eventual decision was actually useless in your case, but one of the really good suggestions was unboxb(), although I'm not sure where to grab that from. TComb is just really bad and should be avoided, while Checkmate and LUTDeCrawl blur the shit out of things. DeDot is useless. Unboxb is a targetted dotcrawl remover which shouldn't blur but I might advise running EE() or LimitedSharpen() (not LSFMod or LSF, just plain LS) over the output anyway.
Additionally, get rid of Toon(), DeRainbow(), and switch Lanczos to Spline36. Just because your video is sterile as hell doesn't mean it looks good. If there is actual rainbowing, see how sangnom flipping it goes, otherwise bifrost might be a good idea. I have no idea what you are using Toon() for so I can't suggest an alternative.
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Cleaning up Outlaw Star

Postby mirkosp » Mon May 10, 2010 2:31 am

Mister Hatt wrote:but one of the really good suggestions was unboxb(), although I'm not sure where to grab that from.

http://forum.doom9.org/showpost.php?p=1303129&postcount=1
Duh. :roll:
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: Cleaning up Outlaw Star

Postby Mister Hatt » Mon May 10, 2010 2:32 am

>Implying I care enough to google
(☞゚∀゚)☞ O U

Most of us are on doom10, or even doom11 now. Keep up.
Mister Hatt
 
Joined: 25 Dec 2007
Status: better than you

Re: Cleaning up Outlaw Star

Postby Cannonaire » Mon May 10, 2010 3:03 am

I like to clean up footage so it's sterile, then once that's achieved I add more lines to make it look good, or to make it 'pop' as some say. The screenshots were taken with Toon and Derainbow off. The Derainbow actually does a great job of removing rainbow wherever I've seen it in Outlaw Star, I just didn't want it interfering in any way with my visual comparisons while tweaking the dot crawl removal. Thanks for the suggestions though, I'll take a look at that other forum.
Image Think millionaire, but with cannons. || Resident Maaya Sakamoto fan.
User avatar
Cannonaire
 
Joined: 05 May 2010
Location: Oregon
Status: OVERLOAD

Re: Cleaning up Outlaw Star

Postby Kariudo » Mon May 10, 2010 3:11 am

Once again Mister Hatt, you fail to explain why <insert filter/program/method> is bad.

I expected more of you by now, but I guess I was too optimistic.

Cannonaire: pay no mind to Mister Hatt there
ImageImage
Image
User avatar
Kariudo
Twilight prince
 
Joined: 15 Jul 2005
Location: Los taquitos unidos
Status: 1924 bots banned and counting!

Next

Return to Footage Help

Who is online

Users browsing this forum: No registered users and 1 guest