Question concerning MPEG 4 feature implementation...

rose4emily · Post by **rose4emily** » Tue Oct 12, 2004 2:31 am

I've been reading a few references on video codec technologies as part of the research I'm doing for writing a video editing application (and a video editing guide for Linux), and I've noticed that the MPEG 4 specification contains some very interesting elements that could be very useful in encoding animation. These are:

Alpha-mapped sprites. Apparently, the MPEG 4 specification states that it should support multiple layers, with alpha channels, that can be translated around the screen in either a prescribed or an interactive manner. In other words, it should be able to perform compositing on playback, using these layers in a manner similar to traditional animation cels or video game sprites.

and

Still-image backgrounds. It also mentions a feature that essentially specifies that a single large still image can be used as a background, and panned behind the rest of the (alpha-mapped, I suppose) picture as necessary. For that matter, it should be possible to encode the slow-panning stills and things like slide shows and the imfamous motionless scenes in Eva as a single image, rather than the tens, or even hundreds of frames that are used when they are treated as normal video.

finally

Interactivity? I haven't yet found anything that discusses this part of the specification in any detail, but I've seen hints at the ability to encode interactive content as part of an MPEG4 stream. I suppose this doesn't have much to do with the efficient encoding of animation, but it sounds like the MPEG4 specification dictates features that should make it possible to put simple 2D games, or at least DVD-style menus in an MPEG4 stream.

---

Now, the question part of this post:

Does anyone know anything about any of these features as they relate to MPEG4, and is anyone aware of a codec implementation that actually supports encoding or playing back video containing any of these features. For that matter, are they buried somewhere in one or more of the popular implementations and just going unused becuase no convenient interface exists for accessing them?

I've seen statements about every popular MPEG4 implementation I can think of stating something to the effect of it having an "incomplete" or "non-complient" featureset, and I'm afraid that these features might be one of those points of non-complience. Still, it'd be very cool to hear about a piece of software that does support these features, or any information on anyone working on an implementation that supports them.

---

And yes, I am aware that none of these features would be terribly meaningful in an AMV-making context, where the original footage has already been composited and cut up into frames. I'm thinking from the perspective of someone who's just getting started with creating original animated footage through traditional processes, and who's building a tool that should facilitate the process of turning a pile of still images into an animated film.

Zarxrax · Post by **Zarxrax** » Tue Oct 12, 2004 9:25 am

These features have not been developed by anyone yet.

Sounds a lot like Flash ;p

post-it · Post by **post-it** » Tue Oct 12, 2004 6:15 pm

rose4emily if I understand what it is you are looking for - it is not MPEG-4:

Alpha Channel lay-overs and animation "on-que" at a desired location within
an AMV are all still possible - but can not be controlled like the IFO's
from DVD's and GAME Disc's in the fashion called "scripts"

To have such control over an avi would require a program
to be running which would "trigger" an Event at a given location/time-zone.

It is possible to have this program ( load itself ) if a container were developed
to "load" the program before starting the avi and ending when
the "show" was over.

I see no practical use for this, outside of giving someone a virus, so "no thanks."

Kalium · Post by **Kalium** » Tue Oct 12, 2004 6:17 pm

Rose is asking if anyone has implemented this segments of the MPEG-4 spec.

Apparantly, the answer is no.

Zarxrax · Post by **Zarxrax** » Tue Oct 12, 2004 6:55 pm

post-it: Of course this is not possible within AVI. This is the reason the MPEG-4 container is developed.

post-it · Post by **post-it** » Tue Oct 12, 2004 7:06 pm

Tab. · Post by **Tab.** » Tue Oct 12, 2004 7:07 pm

Rather, the answer is yes. Mpeg-4's interactive layer is part of Mpeg-4 Systems (ISO 14496-1) and is known as XMT. XMT exists as a scripting language, divided into two parts: the SMIL-esque XMT-O and VMRL-based XMT-A. These can be used to "program" the content of an MP4 file in virtually unlimited ways. See the MP4-faq on Doom9 for tools and further info.

Btw, I'm only helping out cause Alan pointed me here :/
P.S. Kal is a nazi.

post-it · Post by **post-it** » Tue Oct 12, 2004 7:14 pm

TAB - has anyone actually tryed - to make an AMV using XMT yet?

rose4emily · Post by **rose4emily** » Tue Oct 12, 2004 11:16 pm

Thank you, Tab. That MP4 FAQ is exactly what I needed - a pointer to all of these resources I've seen hinted at but not explicitly named or referenced.

It seems I might have been confusing portions of the MPEG4 codec specification with portions of the MPEG4 container specification, much as people often confuse DivX with AVI. Once again, I thank you for helping me figure this one out.

Tab. · Post by **Tab.** » Wed Oct 13, 2004 4:56 pm

Blame the standard -- Mpeg-4 is incredibly large and convoluted. More information is available here.
3.3 states the purpose of Systems, including BIFS.
5.1 describes the visual profiles. Main profile supports transparency, as well as arbitrary shape/sprite coding.
5.3 describes graphics profiles as part of Systems.
10.6 contains further information on BIFS

So, part of what you want is in Video, part in Systems. Unfortunately, no known implementations of the Core or Main Video profiles exist. GPAC, the Systems tool set, is the best you'll be able to do at the moment