How much will mp be of use in AMV's?

giga_d · Post by **giga_d** » Sat Sep 17, 2005 11:50 am

http://www.tv-tokyo.co.jp/wbs/2005/08/2 ... ma/tt.html

This technology is called 'Motion Portrait' and developed by Sony-Kihara Research Center that designed Graphics Synthesizer for PS2. It generates realtime (30fps) 3D facial animation from a single 2D picture, and is possible on an ordinary PC (you don't have to have a massive render farm). As you see in the movie, you can add facial animation to anything including pineapples and anime characters. The researcher says in the movie its application to games is interesting, for example animating your face or an anime character face in a game.

Pretty cool I think.

Zarxrax · Post by **Zarxrax** » Sat Sep 17, 2005 11:53 am

Looks awesome, but I'll believe it when I see the software.

Org Profile · Post by **Otohiko** » Sat Sep 17, 2005 2:07 pm

Zarxrax wrote:Looks awesome, but I'll believe it when I see the software.

Yea, the demos can be a little deceiving.

I can believe it's possible though; just depends on the level of quality.
That said, I've always thought that writing an auto-lip-sync program based on rudimentary phonetics/phonology principles wouldn't be too complicated.

ssj4lonewolf · Post by **ssj4lonewolf** » Sat Sep 17, 2005 2:22 pm

simply amazing however, ur comp would proly have to be pimp shit in order to run it. On my comp I can barley run full time on vegas...

Org Profile · Post by **Otohiko** » Sat Sep 17, 2005 2:29 pm

ssj4lonewolf wrote:simply amazing however, ur comp would proly have to be pimp shit in order to run it. On my comp I can barley run full time on vegas...

I don't think it neccesarily has to be. Look at even Half-Life 2 - they managed to put out some fairly impressive facial animation there, in real time, on very modest hardware requirements.

Naturalistic speech production has been studied in a lot of detail for a while. If I were a little less lazy, I could probably apply to good effect some of the phonetics/phonology I've learned in the last few years into making better lip-sync in AMVs (but eh...).

Personally though... I have suspicions that, even if it works as it works, it may end up in a lot of lazy lip-sync in AMVs. People will naively decide that the program will do everything for them, not really pay attention to nuisances, and as a result we may just end up with much of what we have now, only with 50x more lazyness

(on the bright side, those who use this intelligently may end up with some very neat stuff)

trythil · Post by **trythil** » Sat Sep 17, 2005 2:33 pm

Otohiko wrote: That said, I've always thought that writing an auto-lip-sync program based on rudimentary phonetics/phonology principles wouldn't be too complicated.

It's been done to an extent by Square Pictures, at least; I'd bet that other animation houses do something similar.

Basically, create a library mapping phonemes to meshes, and then given a line you select where each mesh should appear in line with a given phoneme. The mesh is then tweened by the computer.

Castor Troy · Post by **Castor Troy** » Sat Sep 17, 2005 2:38 pm

Think of all the post-con pictures that will be manipulated by this.

Kusoyaro can actually puke on paizuri.

DeinReich · Post by **DeinReich** » Sat Sep 17, 2005 2:39 pm

If it does what it says it does, that would be amazing. Depending on the quality, I wouldn't be supprised if people begin adding CG animations made with these renders to AMVs (i.e. a character acting like he/she/it is the vocalist of the song).

Org Profile · Post by **Otohiko** » Sat Sep 17, 2005 2:56 pm

trythil wrote: Basically, create a library mapping phonemes to meshes, and then given a line you select where each mesh should appear in line with a given phoneme. The mesh is then tweened by the computer.

Well, sure. In industry, I'm fairly sure that'd be fairly standard by now; it'd be pretty ridiculous to do all that stuff just by hand.

Anime is generally way the hell simpler than this. In fact I think it'd feel pretty awkward is Pikachu's mouth suddenly had a full-fledged 30-FPS animation for his um... expressionisms

It's really a BIG exception in conventional, 2-D anime, when a character (unless there's a deliberate close-up on the lips) speaks and you see more than a few actual frames for mouth position (closed vs partially open vs fully open rounded/unrounded). It's rare to have anything beyond that; luckily most of us usually don't pay attention as long as they're flapping in rhythm. That's why I think some of this may well end up as overkill.

SarahtheBoring · Post by **SarahtheBoring** » Sat Sep 17, 2005 6:27 pm

You'll need to stock a lot of Ethers, but...

...oh.

As far as overuse goes, I think the community will learn very fast what sloppy use of a program like that looks like, and opinionate accordingly.

...and as for full mo-cap precisely articulated Pikachu, I'm getting mental images of that what's-it-called animation that they use on Conan. O_o Now that's disturbing. Heh.