If you're interested in learning how to do "proper" roto in AE, here's a crash course (keeping in mind this is definitely more centered around live-action. I'm sure the same principles could be applied to a lot of anime shots).
1) Start on a fairly neutral frame where most of your character's body is in the shot
2) Create masks around each moving part of the body. So like, one for the head (if the hair is moving independently of the head, do head and hair separately), neck, torso, upper arm, lower arm, etc.
3) When you're making each mask, use as FEW mask points as possible. Use bezier curves whenever possible (don't just draw a shitload of points with no curves)
4) Move forward or back in time to the next "key pose" (for example, if your character is walking, go to the next spot where a step lands, or something like that. If the movement is somewhat eratic, just go like, 8 frames forward or something)
5) Moves your masks to match up with the new positions. Try and move/rotate/scale the masks as a whole instead of just moving individual points. You'll be surprised how little point tweaking you'll have to do if you just treat the mask shape as an individual thing. Naturally you probably will need to tweak them, but the idea here is to do this quickly, so try just moving the whole damn thing first.
6) Repeat steps 4-5 for the entire sequence.
7) Now that you have your key poses (or every 8-16 frames) keyframed, you've got some pretty good in-between animation set up for you. Now its a matter of finding the mid-points between the keyframes to further tweak. The goal of this method is that each time you set a new keyframe, the mask is closer and closer to being correct, so tweaking is less intense each time.
8) Rinse and repeat til you have a full roto'd shot.
The major goal with this method is to use as few keyframes and as few mask points as possible. Let the tweening do more of the work. You'd really be surprised how far you can get with fewer keyframes and points.
So yeah. That in a nutshell is the "professional" method for rotoscoping footage. I put professional in quotes because it does go a lot further than that. For one, most roto-pro's don't use AE. They tend to roto in a tracking package like Mocha to let the tracker do half the work. There's also specialized tools like Silhouette (which granted as fallen somewhat out of vogue in the past couple years) that give you a lot more toys to play with.
Now, the major hurdle for this method as it applies to anime is that in most cases, you don't have fluid animation on your characters each frame, since it's more often than not animated on the 2's or 3's (your footage might be 23.976 fps, but the actual character animation was done at something like 12 fps). So those in-betweens will end up being off half the time because it doesn't realize the character had a doubled frame, if that makes sense. I don't know the absolute best way to compensate for this, but you can easily just copy and paste your mask keyframes for a 2nd or 3rd frame wherever you need to. There might be a good way of taking out those doubled-frames before you start masking, then putting them back in afterwards, but I've never experimented with that.
Ask Brad an After Effects Question! - Forum - Site