Important Information for MP4 Users

User avatar
Zero1
Joined: Fri Jan 02, 2004 12:51 pm
Location: Sheffield, United Kingdom
Contact:
Org Profile

Important Information for MP4 Users

Post by Zero1 » Fri Jun 15, 2007 12:38 pm

The short version
Some quite old builds of MP4Box have been creating MP4 files with incorrect NAL Unit sizes (using 16-bits instead of 32-bits), the result of which is some players do not play the files correctly. I advise replacing all your MP4Box builds with the one inside this zip file:
http://aflux.deltaanime.net/Zero1/MP4/mp4muxtools.zip

Other software such as YAMB or MeGUI may also be affected since they use MP4Box, so check their program directories and replace those builds also.

Old files can be fixed by extracting the streams and remuxing using this build of MP4Box. If you simply use the original MP4 as input rather than extracting the streams to raw format, the problem will then be transposed to the new file. Remuxing will take a few seconds to a few minutes and does not affect quality at all. To check if your old videos are affected or not, use the -info switch in MP4Box and check the H.264 stream and see whether it says the NAL Unit size is 16 or 32-bits. 32-bits is fine, 16-bits is not.

The problem itself isn't that serious (if your target audience is PC playback, the popular players and software handle them fine), but the fact that people send encoding tools to their friends means that those bad builds keep getting spread around so the number of non compliant encodes will only get larger if this isn't bought to their attention.

If you know people who use MP4, please link them to this thread as a precaution.

User avatar
Zero1
Joined: Fri Jan 02, 2004 12:51 pm
Location: Sheffield, United Kingdom
Contact:
Org Profile

Post by Zero1 » Fri Jun 15, 2007 12:39 pm

The long version
Recently, I bought a hardware H.264/MP4 player and it has come to my attention that some older versions of MP4Box were not creating fully compatible (and possibly not specification compliant) files. This thread is to outline the problems, symptoms and what you can do about it, as well as provide a tested and working build of MP4Box.
Parts of this FAQ/guide may require you remux a file, or to find file information by using MP4Box. A correct build of MP4Box and a simple batch file to perform the info operation, can be found here:
http://aflux.deltaanime.net/Zero1/MP4/mp4muxtools.zip
This is an MP4Box version and build I can confirm works, and recommend you use.

What is the problem?
The problem is files created with old versions of MP4Box have limited compatibility due to an incorrect value. Old versions of MP4Box set the NAL Unit size as 16-bits (which are the problem files), and new recent versions of MP4Box use a NAL Unit size of 32-bits.

How does it affect me?
The problem is fairly transparent since a lot of software MP4 parsers either share the same code, or are more lenient than some software or hardware. Files muxed with old builds of MP4Box may not work with some software or hardware, therefore limiting compatibility and interoperability. Some file conversion programs that use their own internal MP4 splitters rather than relying on installed splitters may also have problems or not be able to deal with such files. What this means is that some people may not be able to play your encodes as intended, depending on their playback device or software.

How does the problem affect playback?
The result on the KiSS 1600 (a hardware H.264/MP4 DVD player, among many other formats) is that you get audio and no video on muxes using an old MP4Box with the 16-bit NAL Unit size. Other software or hardware may refuse to play the file, or even return errors such as a corrupt file. It depends a lot on the software and/or hardware. What they all have in common is that affected software or hardware will not play back the file satisfactorily, if at all.

What other software makes use of MP4Box?
There is a lot of software that makes use of MP4Box, or it's code. For instance x264's MP4 output uses it's code, and so old versions of x264 that wrote files directly to MP4 may also have a 16-bit NAL Unit size. With this, the only choice is to update your x264 build and hope that it's MP4 output was also updated (it was probably fixed ages ago), or output to .264 (raw) and mux it with a reliable MP4Box build yourself. Other software like YAMB or MeGUI make use of the MP4Box executable and so early versions of those may use bad builds of MP4Box too. You can of course update these programs, or just download the MP4Box I linked to and overwrite the one that these programs use.

What can I do about it?
What you need to do is replace any existing MP4Box builds with this known good one. This will ensure that any files you encode from now on have the correct NAL Unit size to stop any more non compliant encodes being spread around. As mentioned previously, check the program directories for YAMB or MeGUI if you use those and replace the MP4Box you find there with the MP4Box posted in this FAQ/guide. The encoding and muxing process is completely the same as you have become used to, it's just an internal change in how the file is written to MP4. Where possible, tell x264 to output to .264 (raw) and add the raw .264 in MP4Box at time of muxing to avoid any potential issues in MP4 output from x264.
For existing encodes with an incorrect NAL Unit size, you may extract the streams to raw and remux them using the known good MP4Box build. This does not affect quality at all and only takes a minute or so (depending on filesize and HD speed) - this will be covered later.

How can I identify non compliant files muxed with the old builds?
The thing that these files have in common is a NAL Unit size of 16-bits. I can visually identify these files by their playback behaviour on the KiSS 1600 (where I will just get audio and no video), but since not everyone has that option, you can use MP4Box itself to give you the file information. Extract the files contained in the Zip to a new folder, then drag the file you want to check to the same folder and temporarily rename it to "input.mp4" (without the quotes). Double click the batch file and it will give you something that looks like this:
Image
If where the arrow is pointing it says 16 bits for your file, then it is not fully compatible, and it would be ideal to extract the streams and remux them (check later in the guide for instructions on how to do this).

Can I fix the problem in files already encoded, or do I need to re-encode them?
Yes, you can fix the problem with files that have already been encoded, so no, you do not need to re-encode them. What you need to do is extract the files from the MP4 and mux them to MP4 again using the MP4Box build linked at the beginning. This process typically takes a few seconds to a minute depending on the size of the file and HD speed. I will go through how to do this with MP4Box itself, but once you've replaced your MP4Box builds in software such as YAMB and/or MeGUI, those should be able to do this same operation.

Note: The files must be extracted from the original MP4. Remuxing it and telling the program to use the original MP4 file as input only carries the error across. If in doubt, use MP4Box and the mini guide below.

How do I fix/remux non compliant files with MP4Box?
This is relatively simple, but some knowledge of command lines would be beneficial. Unfortunately I cannot write a batch effectively to automate extraction of files and remuxing, because they can be so diverse. For example one encode might have a H.264 stream at TrackID 1 and an MP3 at TrackID 2. Likewise it could be the case that these TrackIDs are reversed, that the audio is AAC, dual audio, or even in some cases dual audio with mixed codecs. MPEG-4 ASP encodes can be strange too because they can have high TrackIDs like 201, or even have Object Descriptor streams (which you can omit).

Now on to the remuxing.
1) Extract mp4muxtools.zip to your desktop.
2) Copy or move the MP4 you want to remux to the same folder as mp4muxtools.zip was extracted to, and temporarily rename the file to "input.mp4" (without the quotes).
3) Double click mp4info.bat. Take note of the TrackIDs as described back in the section "How can I identify non compliant files muxed with the old builds?".
4) Click Start > Run, type cmd and hit enter. This brings you to a command prompt.
5) At the command prompt type "cd desktop\mp4muxtools" (without the quotes) and press enter. This tells it to change to that directory.
6) Type "mp4box -raw # input.mp4" (without the quotes again) and replace the # with the TrackID number from the mp4info.bat window. Press enter and it will begin extracting the track. Repeat until you have extracted all the tracks you want, changing the TrackID number each time. For example if I had this:
Track # 1 Info - TrackID 2 - TimeScale 25000 - Duration 00:00:22.040
this tells me that the H.264 track is TrackID 2, so you use the command line "mp4box -raw 2 input.mp4".
7) To mux your extracted stream to a new MP4, type "mp4box -add file1.ext -add file2.ext -fps # output.mp4". Replace the # with the actual framerate, so for example it would be "-fps 23.976". You can add as many tracks as you like, by repeating the -add function and pointing to other input files. The extracted files will be suffixed with "_track#.ext", so your command line may look like this: "mp4box -add input_track1.h264 -add input_track2.aac -fps 23.976 output.mp4"


That about covers everything. If you have any questions, or if you just want to "sign" the thread saying you've updated MP4Box or remuxed your AMVs and they are now available, then do so. I'd be happy to play them on the KiSS 1600 to check them out :)

Also, Mods please sticky this.

User avatar
Scintilla
(for EXTREME)
Joined: Mon Mar 31, 2003 8:47 pm
Status: Quo
Location: New Jersey
Contact:
Org Profile

Post by Scintilla » Fri Jun 15, 2007 1:05 pm

How old is "old"? My version is "GPAC version 0.4.3-DEV - compiled by Kurtnoise - Apr 23 2007".
ImageImage
:pizza: :pizza: Image :pizza: :pizza:

User avatar
Zero1
Joined: Fri Jan 02, 2004 12:51 pm
Location: Sheffield, United Kingdom
Contact:
Org Profile

Post by Zero1 » Fri Jun 15, 2007 1:45 pm

0.4.3-DEV should be fine, but as it's name implies it's a dev version, so it's compiled from CVS which is undergoing change constantly, and therefore has no guarantee of stability. That said I used 0.4.3-DEV for some time, but I suggest overwriting any MP4Box versions with the one I linked to, just so you can be 100% sure (I muxed files with that and tested it on my player).

From a quick search I can't find anything relevant in the CVS changelog that refers to the NAL Unit size being changed from 16 to 32-bit.

User avatar
Tab.
Joined: Tue May 13, 2003 10:36 pm
Status: SLP
Location: gayville
Org Profile

Post by Tab. » Fri Jun 15, 2007 2:23 pm

Wow, good catch. Thanks for the well composed information, Zero.

User avatar
NeoQuixotic
Master Procrastinator
Joined: Tue May 01, 2001 7:30 pm
Status: Lurking in the Ether
Location: Minnesota
Contact:
Org Profile

Post by NeoQuixotic » Fri Jun 15, 2007 2:35 pm

Insert clever text/image here.

User avatar
Zero1
Joined: Fri Jan 02, 2004 12:51 pm
Location: Sheffield, United Kingdom
Contact:
Org Profile

Post by Zero1 » Fri Jun 15, 2007 2:43 pm

Tab. wrote:Wow, good catch. Thanks for the well composed information, Zero.
Thanks, I appreciate it.

When I first found this problem I felt like I was hitting my head against a brick wall, because I knew for a fact this KiSS 1600 supports all of high profile and played one of my insane encodes with ease; so I was fortunate in that I knew it wasn't decoder feature related. So by process of elimination I found that there was a discrepency in the muxing.

I might sift through my IRC logs as I was making discoveries and post them sometime. It really was quite odd.
anubisx00 wrote:Some AMVs I've found to have a NAL Unit size of 16.
Absolutely awesome. Not only did you find a bunch of NAL 16's to test, you also found NAL 24's! (Which is something I haven't come across, granted I have a lot of MP4s but rarely check the info in MP4Box unless I have a problem).

I'll be downloading these right away. I really wonder how the 24-bit NALUs will go down.

Considering this, I now wonder if in old builds of MP4Box the NAL size was auto generated based on some criteria (grabbing at straws, but something like duration or filesize), and perhaps the auto generation was wrong, so they just switched to a fixed size of 32-bit.

Anyway, off to test now. Thanks

User avatar
LantisEscudo
Joined: Thu Mar 08, 2001 5:21 pm
Location: Eastern Massachusetts
Contact:
Org Profile

Post by LantisEscudo » Fri Jun 15, 2007 5:38 pm

Interesting. All of the ones I'm distributing are 32, but almost every one I've downloaded (which are older) are 16 or 24.

Thanks for catching this, Zero.

And a few more for your testing that I found in my collection:

16 NALU:
(Direct Link)



24 NALU:

User avatar
NeoQuixotic
Master Procrastinator
Joined: Tue May 01, 2001 7:30 pm
Status: Lurking in the Ether
Location: Minnesota
Contact:
Org Profile

Post by NeoQuixotic » Fri Jun 15, 2007 8:29 pm

What exactly is a NAL unit? A quick google search didn't result in anything that made any sense to me. (If its too hard to describe, don't bother.) Also, your doom9 post talks about being spec compliant. Is there a spec requirement for the NAL unit size? Since the unit size can vary I'd assume it has its purpose.
Insert clever text/image here.

User avatar
Kionon
I ♥ the 80's
Joined: Fri Mar 02, 2001 10:13 pm
Status: Ayukawa MODoka.
Location: I wonder if you know how they live in Tokyo... DRIFT, DRIFT, DRIFT
Contact:
Org Profile

Post by Kionon » Fri Jun 15, 2007 10:40 pm

Crap, I suppose I'll need to check my encodes now. :/
ImageImage
That YouTube Thing.

Locked

Return to “Video & Audio Help”