I started considering doing something like this a few weeks after Into The Labyrinth premiered, but have always been too lazy to do so. I figured that questions about how to use Twixtor would not last for very long, but I was wrong. So instead of continuing to answer every question from 0 again, here is the text I will link to everytime someone asks me again
. I originally planned to transform this to a fully fledged tutorial with pictures and examples and such nice things at one point, but I guess my limited time and my laziness will postpone such events into a distant future.
Note that this is my procedure for using animated ("cartoony") footage only, using regular live-action footage or other stuff that is more true to it's playback framerate is way easier to handle and doesn't require most of the stuff I detail here.
Let me warn you that motion interpolation using Twixtor limits you really strongly in your scene selection. There are only certain scenes that can be interpolated well, and moving hair usually indicates that it at least will be pretty difficult. I always look for scenes that don't change much between 2 different "redrawn" frames. That's the case when the scene already uses slow-motion or slow movement in general. Fast action scenes or badly animated scenes usually tend to look really bad when interpolated.
For Twixtor I developed a kind of motion-interpolation-workflow in After Effects that looks like this:
Let Comp A be the comp where you want to do your "usual" editing in. I import the raw footage into there, kind of put it in place, and then pre-comp it (ctrl+shift+c). I name it something with "Twixtor Output" and check "Leave all attributes...". Then I open this newly created comp, it should contain your raw footage and have the exact same framerate and resolution as your clip. This is where I apply Twixtor (or Twixtor Pro for difficult scenes), and set the "input framerate" to the exact framerate of the clip (and comp). This is also where the masking happens in case I use Twixtor pro, after I completed the following steps.
Update: In case your clip is animated in a constant framerate for the whole duration, instead of the following, you can just simply set the "twixtor input framerate" to the appropriate framerate ("23.976/3" (roughly 8 fps) or "23.976/2" (roughly 12 fps)). If it changes framerate, you gotta do the following instead.
Then I pre-comp again (ctrl+shift+c), name it something with "Twixtor Input" and check "leave all attributes" again. I open that comp, should find the raw clip again, and make sure that the scene is time-remapped in a way so that every new frame of that new comp represents an actual "newdrawn" frame in the animation. So for most scenes in anime that means I simply speed the footage up by 300% (stretch the clip to 33.333% length), or 200% (50%). Some more complicated scenes (in Bakemonogatari for example) require me to use manual time-remapping for every frame, because the "newdrawn" frametimes sometimes change in the middle of the scene. After that you can go all back to comp A again and use the "Twixtor Output" comp in any way you want: Slow it down, use "Force Motion Blur", incrcease the framerate of comp A, and all other kinds of useful stuff, the "Twixtor Output" should always display the exact correctly interpolated frame. For example if you use timeremapping so that you request a frame "12.5", it will actually display a interpolated frame exactly in the middle between frame 12 and frame 13. Notice that if you don't do anything with it it should already show some interpolation because it should have changed the "anime-typical stuttering" to actual fluid motion that shows a "newdrawn" frame every single frame rather than every 3rd or 2nd frame.
This is where you rather frequently will notice that the interpolation looks horrible, that's where the fine-tuning with Twixtor Pro comes into play. I will not explain that, it is beautifully explained in the Manual for Twixtor Pro that comes with it. (You set a directory of the manual when installing Twixtor). But keep in mind that everything regarding that should be done in the "Twixtor Output" comp. Many times even Twixtor Pro can't help make the footage look good, that's when you should start thinking about giving up that particular scene and look for something more Twixtor compatible instead.
I hope this can be of some help, I realize that this is not written beautifully, so if you got any more questions: The thread should be open. Sometimes I'm available on Skype, the skype name is pretty easy to guess.
Also, I'm pretty sure there is at least one mistake in this rather bumpy explanation, so ask me if something doesn't work as it should, and I will correct it if it really is an error.