MKV for adobe premiere pro CS5

[DJ_Izumi]

Hey, perhaps I should start using hours long AVC encodes with a single keyframes as a source, they will load, so it's fine!

But seriously, just because something works doesn't mean it's optimal. Using lossy non intra-only sources to edit is not fast, nor reliable, and I don't see what's the point in making the editing experience a slow and painful one when you could just load blazingly fast utvideo clips and just enjoy the smooth editing afterwords. But hey, not everybody can be a pitcher, so I ain't stopping who likes it the other way.

Actually, it is fast. I'm actually able to seek without major difficulty, or, for the sake of comparison, it's no slower than trying to get the PC to chew through the obscene ammount of data needed to seek through 1920x1080 24fps Lagarith Lossless. That's a LOT of I/O to put through the system afterall, and excluding SSDs, CPU power for decoding has advanced a lot farther than access time on hard drives. I'd say that both experiences are about equal on this box. It's not even putting the CPU past 50% load as I seek and it randomly pulls of frames.

Honestly, have any of you actually sat down and put this to the test? As you dismiss it, have any of you actually TRIED IT? I mean, tried it with Premiere CS5.5, and not that you tried it years ago in the past and assumed the software hasn't evolved?

And... How is it not 'reliable'?

[Pwolf]

Honestly, have any of you actually sat down and put this to the test? As you dismiss it, have any of you actually TRIED IT?

Yes. CS5.

[DJ_Izumi]

Yes. CS5.

And what were your results? (Other than the obvious 10bit thing. I was actually gonna be pretty shocked if 10bit worked.) Because I'm actually fairly suprised with how smoothly this is editing on this box.

[mirkosp]

I laugh at you for using lagarith and not utvideo as a lossless codec for editing. Just as much as editing software evolves, so do lossless codecs. Feel free to compare yourself how having 6 1280x720 (no 1080p editing, this is an AMV site, and aside for ghibli movies and the likes 1080p anime are just upscaled from 720p to begin with) avc clips overlayed on each other on 6 tracks, and then try the same deal with 6 1280x720 utvideo clips (whatever predictor you prefer), then come back here saying that it's just as fast. Really, I wanna see that. Oh, and no cheating. Don't make those avc encodes intra-only or with otherwise editing-optimized settings, that defies the purpose (since it's going to be a valid editing choice anyway, but it's not what you get out of the box with fansubs or BDs).

And please, as soon as you start enabling advanced features in the avc encode, the decoder is gonna fall short because those features were clearly thought for saving space during playback and not for faster than realtime editing. AVC is a lucky format since it specifies how it should be decoded too, but as soon as you wind up loading mpeg-2 or mpeg-4 asp (and maybe vc-1 too, but don't count my word on that because I'm not sure on this one format) encodes you could be getting different looks from a moment to the other since the formats don't specify a bit precise decode so it's all left up to decoders implementations, which might decode differently from what you expect. I don't call that reliable.
You can't just expect any encode to be awesome for editing, especially if it's a fansub encode, which is clearly thought for playback and limiting the required storage space.

And by the way, industry standard is editing with ProRes, so we're talking 10bit 4:2:2, really.

[DJ_Izumi]

Feel free to compare yourself how having 6 1280x720 (no 1080p editing, this is an AMV site, and aside for ghibli movies and the likes 1080p anime are just upscaled from 720p to begin with) avc clips overlayed on each other on 6 tracks, and then try the same deal with 6 1280x720 utvideo clips (whatever predictor you prefer), then come back here saying that it's just as fast. Really, I wanna see that. Oh, and no cheating. Don't make those avc encodes intra-only or with otherwise editing-optimized settings, that defies the purpose (since it's going to be a valid editing choice anyway, but it's not what you get out of the box with fansubs or BDs).

I think I'll stick with 1080p, you seem a tad ignorant on the issue of 1080p vs 720p. While certianly not all, a lot of anime is natively rendered out at 1080p. Though for there's almost no 1080p broadcast systems, they're generally 720p or 1080i, so most material is broadcast from a 720p source and upscaled to 1080i. So while television ripped fansubs are capped at 720p, the BDrips can be 1080p, assuming they are rendered out at 1080p. It's certianly not absolute though, I mean hey, FMA: Brotherhood was only rendered at 540p for some silly reason. Then there's all that lovely 90's anime that was mastered on 35mm film, those make for beautiful 1080p transfers. Have you checked out the Lain BDs? If it wasn't for the sake that it was 4:3, you'd think it was animated just this year. It's just a glorious transfer.

Anyway, here you go, this is using Coalgirl's 1080p Blu-Ray of Kanon, the streams were pulled out of the MKVs without any modification, so this is pretty representive of 'any ol' fansub' since it is just that, any ol' fansub. The top five layers are at 15% opacity while the bottom is at 100%, to ensure all six streams need to be rendered simultaniously. There's a 98% CPU spike that you can see the moment I press play on my preview, but it drops and then playback continues with an average of 50%. This is probably it trying to quickly find the last I frame before the start points so it can deliver the first request frame on each track. Though you can see that through the graph yourself.

http://i254.photobucket.com/albums/hh10 ... /AVCx6.jpg

And please, as soon as you start enabling advanced features in the avc encode, the decoder is gonna fall short because those features were clearly thought for saving space during playback and not for faster than realtime editing. AVC is a lucky format since it specifies how it should be decoded too, but as soon as you wind up loading mpeg-2 or mpeg-4 asp (and maybe vc-1 too, but don't count my word on that because I'm not sure on this one format) encodes you could be getting different looks from a moment to the other since the formats don't specify a bit precise decode so it's all left up to decoders implementations, which might decode differently from what you expect. I don't call that reliable.
You can't just expect any encode to be awesome for editing, especially if it's a fansub encode, which is clearly thought for playback and limiting the required storage space.

And by the way, industry standard is editing with ProRes, so we're talking 10bit 4:2:2, really.

Pro-Res isn't used by any cameras however, so to say it's industry standard' for editing is kinda... Wrong. You're gonna edit AVCHD with pro-sumer cameras. NXCAM on low range professional cameras ($5000~) NXCAM is still 4:2:0 AVC, just in a much more flexable standard. XDCAM, MPEG-2 4:2:2, which you'll see in cameras in a 20k-35k range. You're also forgetting DNxHD, which is stock standard on Avid platforms. While Adobe is pro and Apple was once pro, Avid's stuff is still considdered top tier. I've used ProRes though, but only when capturing from an AJA IOHD box. Fun box to play with, BTW. ProRes is most often used for offline proxies, which you'd use since directly editing something like REDCODE RAW would be exceptionally demanding, but just about every camera below the cinema level uses formats that you can edit directly on any decent box.

[mirkosp]

Feel free to compare yourself how having 6 1280x720 (no 1080p editing, this is an AMV site, and aside for ghibli movies and the likes 1080p anime are just upscaled from 720p to begin with) avc clips overlayed on each other on 6 tracks, and then try the same deal with 6 1280x720 utvideo clips (whatever predictor you prefer), then come back here saying that it's just as fast. Really, I wanna see that. Oh, and no cheating. Don't make those avc encodes intra-only or with otherwise editing-optimized settings, that defies the purpose (since it's going to be a valid editing choice anyway, but it's not what you get out of the box with fansubs or BDs).

I think I'll stick with 1080p, you seem a tad ignorant on the issue of 1080p vs 720p. While certianly not all, a lot of anime is natively rendered out at 1080p. Though for there's almost no 1080p broadcast systems, they're generally 720p or 1080i, so most material is broadcast from a 720p source and upscaled to 1080i. So while television ripped fansubs are capped at 720p, the BDrips can be 1080p, assuming they are rendered out at 1080p. It's certianly not absolute though, I mean hey, FMA: Brotherhood was only rendered at 540p for some silly reason. Then there's all that lovely 90's anime that was mastered on 35mm film, those make for beautiful 1080p transfers. Have you checked out the Lain BDs? If it wasn't for the sake that it was 4:3, you'd think it was animated just this year. It's just a glorious transfer.

I'll throw the ignorance right back at you: while a lot of shows are surely rendered at 1080p, the actual drawings and effects are most often than not still just 720p or so. The 1080p things are limited to text and some specific graphic elements most of the time. See for example baka to test, which is 720p most of the time, but some graphical elements and text (for example, in a few eps a little demon version of the protagonist with some text shows up in the bottom of the picture) are 1080p. Hardly a reason to actually do 1080p. Same reasoning can be done for most other shows, you can do comparisons. I would rather not be using the odd one-offs examples such as the work on the Lain BDs (which iirc involved actually completely redoing the CG as well) as the main example to go by for a standard procedure.
And so, when you have sub-1080p material, what are you going to do? Keep the upscale and waste space? Resize with premiere's resizer and thus losing on better resizing algorithms and possibly having to manually correct the gamma too?

Anyway, here you go, this is using Coalgirl's 1080p Blu-Ray of Kanon, the streams were pulled out of the MKVs without any modification, so this is pretty representive of 'any ol' fansub' since it is just that, any ol' fansub. The top five layers are at 15% opacity while the bottom is at 100%, to ensure all six streams need to be rendered simultaniously. There's a 98% CPU spike that you can see the moment I press play on my preview, but it drops and then playback continues with an average of 50%. This is probably it trying to quickly find the last I frame before the start points so it can deliver the first request frame on each track. Though you can see that through the graph yourself.

http://i254.photobucket.com/albums/hh10 ... /AVCx6.jpg

I would not consider Bloatgirl's unreasonable encoding settings as a standard fansub nor a good encode, and kanon is most certainly one of the shows which are 720p. You didn't provide a comparison with utvideo either, how does it fare? I didn't say that you cannoy get acceptable results with some avc stuff, but that utvideo is gonna be a better choice anyway. Notice how some is italic.

And by the way, industry standard is editing with ProRes, so we're talking 10bit 4:2:2, really.

Pro-Res isn't used by any cameras however, so to say it's industry standard' for editing is kinda... Wrong. You're gonna edit AVCHD with pro-sumer cameras. NXCAM on low range professional cameras ($5000~) NXCAM is still 4:2:0 AVC, just in a much more flexable standard. XDCAM, MPEG-2 4:2:2, which you'll see in cameras in a 20k-35k range. You're also forgetting DNxHD, which is stock standard on Avid platforms. While Adobe is pro and Apple was once pro, Avid's stuff is still considdered top tier. I've used ProRes though, but only when capturing from an AJA IOHD box. Fun box to play with, BTW. ProRes is most often used for offline proxies, which you'd use since directly editing something like REDCODE RAW would be exceptionally demanding, but just about every camera below the cinema level uses formats that you can edit directly on any decent box.

Yeah I admit, I stretched it a bit, but you can't say I'm completely wrong either. It largely depends on the setup and who does what, but I've been confirmed that one of the standard procedures which can be used involves converting to prores before editing, so you can't say just anybody does direct editing. Also, the codecs you list aren't quite the same thing as BD AVC or standard fansub encodes, as they are generally higher bitrate and actually thought to be somewhat usable even in case of direct editing.

[Pwolf]

Yes. CS5.

And what were your results? (Other than the obvious 10bit thing. I was actually gonna be pretty shocked if 10bit worked.) Because I'm actually fairly suprised with how smoothly this is editing on this box.

Assuming it actually worked, it was sluggish and iirc frame accuracy wasn't ideal (of course this was about 2 years ago when CS5 came out). And I'm not talking about moving the play head and waiting for it to move. Premiere pro has been really good at letting you move the playhead have you wait for the video to catch up. That's sluggish. That said, I decided it would be a great idea to do a bit of extended testing to prove my point.

I have a 1080p fansub @23.976fps in an MKV container. For obvious reason I cannot load this into premiere directly. I tried Phant's batch file to remux the streams into an MP4 container. Wouldn't work. So I used mp4Box... Wouldn't work. So then I used x264 (x264 -o test.mp4 test.mkv). WOOT! I have an mp4! I opened it in Premiere CS5... it crashed. So I opned up ZarxGUI and re-encoded the video at the default settings (YEA RE-ENCODING LOSSY TO LOSSY! THATS ALWAYS GOOD! But this is 2012 so it's OK!). Opened in premiere, OMG IT WORKS!.

Playback is bit sluggish compared to my experience with UTVideo but I would consider it acceptable. Example would be during faster seeks, it will skip frames. While this isn't terrible, there are situations where I would like to actually see those frames. There's the issue of frame accuracy as well... I can seek to a point in time and make a cut between two scenes. When I move my playhead infront of the scene I just made a cut before, it displays the frame before it (from the previous scene). So I now I have to make a 1frame cut to compensate. Wasted time and thats assuming it doesn't come back to bite me in the ass when I go to export. Also and this should explain itself:



I don't think I need to explain what's going on here. This alone would be a reason not to use a generic AVC/H264 encode directly. This is a low-action scene as well. That said, in the amount of time it's taken me to actually import into premiere, i could've encoded to UTVideo and started editing with no problems. Even then, now that I have AVC/H264 in premiere now, I had to re-encode it again to a lossy format and it doesn't even decode correctly. I cannot, with 100% certainty put my faith into this source. With lossless, I can.

[DJ_Izumi]

And so, when you have sub-1080p material, what are you going to do? Keep the upscale and waste space? Resize with premiere's resizer and thus losing on better resizing algorithms and possibly having to manually correct the gamma too?

I say stick with 1080p, cause any motion graphics or other AE work you do will be preserved. 1080p really is the benchmark now so why downscale? Unless the show is a blatent upscale that is. And even if characters arn't quite up to 1080p, backgrounds might be and such. I don't think 720p vs 1080p really causes thaaaaaat much of an increase of space, unless you're converting to lossless first. Though it's really more at the editor's discretion.

I would not consider Bloatgirl's unreasonable encoding settings as a standard fansub nor a good encode, and kanon is most certainly one of the shows which are 720p. I didn't say that you cannoy get acceptable results with some avc stuff, but that utvideo is gonna be a better choice anyway. Notice how some is italic.

I figured they'd be a pretty good extreme example, considdering how much they try to pointlessly trick their stuff out. They once did a direct mux release, because their attempts to re-encode proved to be LARGER than streams from the Blu-Ray disc they were sourcing from. Any other fansub group's stuff would probably be easier to chew on. 720p stuff even more so. I think this is more a matter of 'It'll work, if you have enough CPU power to do it'. What I've demonstrated would probably not work on a dual core system of any kind, maybe 2-3 layers but that's it. You'd start having preview lag after that I imagine.

You didn't provide a comparison with utvideo either, how does it fare?

I didn't bother, it's not like we're really disputing the usefulness of converting to lossless if possible, but the OP is working with less than 1TB of storage so it's a problem for him. Though in the case of lossless vs direct editing, it'd simply be a matter of how many layers and which one saturates the system first. AVC saturing the CPU or lossless saturing the storage device? I'd be curious to see which one maxed out first, but I'm not going to invest the time into finding out.

Yeah I admit, I stretched it a bit, but you can't say I'm completely wrong either. It largely depends on the setup and who does what, but I've been confirmed that one of the standard procedures which can be used involves converting to prores before editing, so you can't say just anybody does direct editing. Also, the codecs you list aren't quite the same thing as BD AVC or standard fansub encodes, as they are generally higher bitrate and actually thought to be somewhat usable even in case of direct editing.

No, but I think that this proves that this is doable and it shouldn't be entirely verboten like it has been for a long time. That said, 10bit encodes don't work at all and every fansubber and their grand mother is rushing to start exclusively using 10bit. Annoys the hell out of me, because my HTPC relies on DXVA for playback, 10bit stuff is unplayable on it for that reason.

[DJ_Izumi]

I have a 1080p fansub @23.976fps in an MKV container. For obvious reason I cannot load this into premiere directly. I tried Phant's batch file to remux the streams into an MP4 container. Wouldn't work. So I used mp4Box... Wouldn't work. So then I used x264 (x264 -o test.mp4 test.mkv). WOOT! I have an mp4! I opened it in Premiere CS5... it crashed. So I opned up ZarxGUI and re-encoded the video at the default settings (YEA RE-ENCODING LOSSY TO LOSSY! THATS ALWAYS GOOD! But this is 2012 so it's OK!). Opened in premiere, OMG IT WORKS!.

I'm not sure what's worse, that you're solution to get over your inability to transmux it into an MP4 container was to utterly rape the footage by re-encoding it, or that just demuxing the stream using mkvextract or something didn't occur to you... o.O Premiere does natively support .264 files afterall and it's what I did for simplicity's sake. How about you give that a shot and report your results?

There's the issue of frame accuracy as well... I can seek to a point in time and make a cut between two scenes. When I move my playhead infront of the scene I just made a cut before, it displays the frame before it (from the previous scene). So I now I have to make a 1frame cut to compensate. Wasted time and thats assuming it doesn't come back to bite me in the ass when I go to export.

I experienced no issues with frame accuracy in my case.

http://www.pwolfamv.com/other/onereason.png[/url]

You didn't by chance transcode a 10bit fansub to 8bit did you? I've noticed that some software for such conversions actually mess up decoding the 10bit encode and hardcode all sorts of garbage into the outputted file. I've had this issue when trying various software when looking to re-encode 10bit to 8bit so that material will playback on my HTPC.

[mirkosp]

I don't completely condemn AVC editing hey, especially if it's intra-only. I just want it to be encoded with settings thought out for editing, though. Really, in the end editing speed is everything to me and a boost in decode is gonna be a saviour. Now, 1 TB space limit could be troublesome without clipping, but I can get over 300fps decode speed with 720p predict median utvideo, and I just can't get that far with avc without it getting to bloat more than utvideo predict median ends up being (lossy intra-only ends up bloating a lot more to preserve quality compared to lossless in my testing, so yeah).

Now, I admit things aren't quite as bad as they used to be with direct lossy non-intra-only decoding, but there are a number of reasons for which lossless encoding is still a superior choice, hence why it should be encouraged at all costs.

Also, as far as custom effects go: I say downscale. A good AMV should try to merge the custom effects with the anime. Having sharp and aliased effects on top of slightly blurry anime isn't as pretty as having the effects match the sharpness level of the anime, which make it merge much more. This means that editing at 720p and applying a slight blur or lowpass filter to the effects as well is the best way to amalgamate the two.