HD video additions to the guides - Ripping

This forum is for video and audio help and discussion.

HD video additions to the guides - Ripping

Postby BasharOfTheAges » Sun Dec 13, 2009 11:06 am

I know the general workflow for working with BluRays is pretty brain dead simple (and could be worked into zarxfag with a few paragraphs distributed over about 5 or 6 pages), but I've noticed the new standard for ripping, DVDFab(HD), is only trialware for BRDs. Has anyone had any luck with a better piece of software, or are we going to be ("officially") stuck for years with the legally dubious parallel to MPEG2 encoding with TMPEGEnc?
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Re: HD video additions to the guides - Ripping

Postby Zarxrax » Sun Dec 13, 2009 11:45 am

Good question. I haven't personally ripped any BRDs yet, so I hadn't noticed this.
User avatar
Zarxrax
 
Joined: 01 Apr 2001
Location: Concord, NC

Re: HD video additions to the guides - Ripping

Postby Qyot27 » Sun Dec 13, 2009 12:32 pm

BasharOfTheAges wrote:I know the general workflow for working with BluRays is pretty brain dead simple (and could be worked into zarxfag with a few paragraphs distributed over about 5 or 6 pages), but I've noticed the new standard for ripping, DVDFab(HD), is only trialware for BRDs. Has anyone had any luck with a better piece of software, or are we going to be ("officially") stuck for years with the legally dubious parallel to MPEG2 encoding with TMPEGEnc?

Honestly, I'd think it's about time to change the recommended MPEG-2 encoder to HCenc. But that's just me.

I've not had to rip any Blu-rays yet (nor do I even have a BD-ROM drive, or any Blu-rays to start with), but I know there is work on open-source libraries to handle it, for example so mplayer supports playback like it does with libdvdcss. Once that comes to fruition it'll be possible.
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog
User avatar
Qyot27
Surreptitious fluffy bunny
 
Joined: 30 Aug 2002
Location: St. Pete, FL
Status: Creepin' between the bullfrogs

Re: HD video additions to the guides - Ripping

Postby BasharOfTheAges » Sun Dec 13, 2009 3:49 pm

Qyot27 wrote:
BasharOfTheAges wrote:I know the general workflow for working with BluRays is pretty brain dead simple (and could be worked into zarxfag with a few paragraphs distributed over about 5 or 6 pages), but I've noticed the new standard for ripping, DVDFab(HD), is only trialware for BRDs. Has anyone had any luck with a better piece of software, or are we going to be ("officially") stuck for years with the legally dubious parallel to MPEG2 encoding with TMPEGEnc?

Honestly, I'd think it's about time to change the recommended MPEG-2 encoder to HCenc. But that's just me.

I've not had to rip any Blu-rays yet (nor do I even have a BD-ROM drive, or any Blu-rays to start with), but I know there is work on open-source libraries to handle it, for example so mplayer supports playback like it does with libdvdcss. Once that comes to fruition it'll be possible.

It's already possible. DVDFabHD does it. Possible isn't Free (or free for that matter).

As for after ripping, I've just been using dss2() to pop it into AVIsynth. It's all the same from there. Only other stuff that needs to go into the guides are resize numbers for 720 and 1080 (the former because a good deal of people's systems can't handle the latter). Maybe a note about how pre-processing takes a bit longer with the increase in frame size (6x larger for full 1080).

The ripping is the only issue - and only really an ethical one at that. Suggesting to so many people that you should use a non-free tool results in alarming rates of piracy of that tool. :amv:
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Re: HD video additions to the guides - Ripping

Postby Zarxrax » Sun Dec 13, 2009 8:27 pm

Qyot27 wrote:Honestly, I'd think it's about time to change the recommended MPEG-2 encoder to HCenc. But that's just me.


Feel free to write the guide for it.
There are a number of reasons that I haven't done so already:
1. A few years back I tried making some mpeg2 encodes for awa using quenc and hcenc, and awa kept telling me that me encodes were screwed up. I never figured out what the heck I was doing wrong, so I decided to just play it safe and stick with tmpegenc.
2. mpeg-2 is mostly useless for amvs except for sending to conventions. And for that, the bitrate is generally high enough that you wont see serious artifacts when its shown at the convention.
3. Hcenc only encodes the video stream, meaning users will have to encode audio seperately, and then mux them with something. This complicates things a fair bit. I also don't know a good muxer that is both easy to use and works well.
4. I keep hoping the conventions will all upgrade to mpeg4, and then this will all be moot.
User avatar
Zarxrax
 
Joined: 01 Apr 2001
Location: Concord, NC

Re: HD video additions to the guides - Ripping

Postby Qyot27 » Sun Dec 13, 2009 11:48 pm

BasharOfTheAges wrote:
Qyot27 wrote:Honestly, I'd think it's about time to change the recommended MPEG-2 encoder to HCenc. But that's just me.

I've not had to rip any Blu-rays yet (nor do I even have a BD-ROM drive, or any Blu-rays to start with), but I know there is work on open-source libraries to handle it, for example so mplayer supports playback like it does with libdvdcss. Once that comes to fruition it'll be possible.

It's already possible. DVDFabHD does it. Possible isn't Free (or free for that matter).

As for after ripping, I've just been using dss2() to pop it into AVIsynth. It's all the same from there. Only other stuff that needs to go into the guides are resize numbers for 720 and 1080 (the former because a good deal of people's systems can't handle the latter). Maybe a note about how pre-processing takes a bit longer with the increase in frame size (6x larger for full 1080).

The ripping is the only issue - and only really an ethical one at that. Suggesting to so many people that you should use a non-free tool results in alarming rates of piracy of that tool. :amv:

I was referring to a solution that doesn't contribute to software piracy. I know DVDFabHD can do it.

As for the actual opening it part, I'd go for DGAVCIndex or FFmpegSource2 (although if you don't remux to MKV first, you need Haali's splitter installed so it can handle TS). Apparently there actually are some situations where DSS2 is not frame-accurate.


Zarxrax wrote:Feel free to write the guide for it.
There are a number of reasons that I haven't done so already:
1. A few years back I tried making some mpeg2 encodes for awa using quenc and hcenc, and awa kept telling me that me encodes were screwed up. I never figured out what the heck I was doing wrong, so I decided to just play it safe and stick with tmpegenc.
2. mpeg-2 is mostly useless for amvs except for sending to conventions. And for that, the bitrate is generally high enough that you wont see serious artifacts when its shown at the convention.
3. Hcenc only encodes the video stream, meaning users will have to encode audio seperately, and then mux them with something. This complicates things a fair bit. I also don't know a good muxer that is both easy to use and works well.
4. I keep hoping the conventions will all upgrade to mpeg4, and then this will all be moot.

I actually did get a guide sort-of written up, but it got to the point where trying to lead people through the steps visually and actually present the info in an understandable way to a beginner was grating on my nerves - also for the fact that I decided it would also be best covered with a section describing batch scripting for more advanced users, seeing as that way video encoding, audio encoding, and muxing can be handled all at once and without user input (and for those tasks, the scripts are really not much more complicated than using AviSynth). I packed up the files and archived them, so I could finish it one of these days, but I just haven't yet.

The most I could settle on doing was distribute my pack of profiles to use with HCenc (which could use some degree of updating in the face of the newest betas of v0.24). There is a very rudimentary primer that I posted on Wordpress, though. It was also from March, so the MPEG Default vs. FOX1 thing I mention in regard to how HCenc performs compared to TMPGEnc is outdated and in need of re-evaluation. As are some of the options used there (like INTRAVLC 1, as that parameter is now set on 2 - Auto, and Auto is a more optimal setting than 1 or 0).

1./2. For HCenc, the main focus is DVD compliancy (although ostensibly it could be used for Blu-ray compliant MPEG-2, since it allows up to 1920x1152) - and for that purpose, I've had no issues with it. Both TMPGEnc DVD Author/Authoring Works and dvdauthor accept the streams just fine, and I've not had issues with them on my DVD players either. However, I only started using HCenc as of v0.23, when I switched to it in summer 2008 - I don't know what sorts of compatibility issues may have existed before then.

3. The two muxers I use are mplex1 (which is kind of old, and I guess is also the same as tc_mplex; it was based on bbmpeg) and mplex from the mjpegtools project, which is the only solution I've found that handles custom pulldown (24->29.97 and 25->29.97) properly. Both CLI, although there are some apps that use them in their toolchains, like DVD Flick. For the audio, I know that batch encoding programs often interface with twolame and/or aften (or ffmpeg) to handle it, possibly by way of piping the audio from wavi like I do.
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog
User avatar
Qyot27
Surreptitious fluffy bunny
 
Joined: 30 Aug 2002
Location: St. Pete, FL
Status: Creepin' between the bullfrogs

Re: HD video additions to the guides - Ripping

Postby BasharOfTheAges » Mon Dec 14, 2009 7:31 am

Qyot27 wrote:
BasharOfTheAges wrote:
Qyot27 wrote:Honestly, I'd think it's about time to change the recommended MPEG-2 encoder to HCenc. But that's just me.

I've not had to rip any Blu-rays yet (nor do I even have a BD-ROM drive, or any Blu-rays to start with), but I know there is work on open-source libraries to handle it, for example so mplayer supports playback like it does with libdvdcss. Once that comes to fruition it'll be possible.

It's already possible. DVDFabHD does it. Possible isn't Free (or free for that matter).

As for after ripping, I've just been using dss2() to pop it into AVIsynth. It's all the same from there. Only other stuff that needs to go into the guides are resize numbers for 720 and 1080 (the former because a good deal of people's systems can't handle the latter). Maybe a note about how pre-processing takes a bit longer with the increase in frame size (6x larger for full 1080).

The ripping is the only issue - and only really an ethical one at that. Suggesting to so many people that you should use a non-free tool results in alarming rates of piracy of that tool. :amv:

I was referring to a solution that doesn't contribute to software piracy. I know DVDFabHD can do it.

As for the actual opening it part, I'd go for DGAVCIndex or FFmpegSource2 (although if you don't remux to MKV first, you need Haali's splitter installed so it can handle TS). Apparently there actually are some situations where DSS2 is not frame-accurate.

I started with DGAVCIndex and it decided it didn't like certain BRDs. I re-ripped one of them several times to be sure and it didn't work any of the times. Gave several dozen errors (I stopped hitting Yes/No after about 80 or so)

As to the non-frame accuracy, is that for playback? Because I'm just using it to load, filter, and dump the entire disc as a single file anyways. I'd never recommend editing with the scripts themselves with HDDs so cheap nowadays. It's silly.
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Re: HD video additions to the guides - Ripping

Postby Qyot27 » Mon Dec 14, 2009 10:29 am

BasharOfTheAges wrote:I started with DGAVCIndex and it decided it didn't like certain BRDs. I re-ripped one of them several times to be sure and it didn't work any of the times. Gave several dozen errors (I stopped hitting Yes/No after about 80 or so)

As to the non-frame accuracy, is that for playback? Because I'm just using it to load, filter, and dump the entire disc as a single file anyways. I'd never recommend editing with the scripts themselves with HDDs so cheap nowadays. It's silly.

I do recall that the version of libavcodec it depended on had some caveats, but there seems to be enough buzz around the -NV versions.

It may be for playback, or it may be in general. The underlying point was that it relies on PTS timestamps, and if the PTS is wrong or absent, DSS2 ceases to be frame-accurate. For a quick pass-through it may be okay, but I'm not sure how it would react with filtering that manipulates frames like IVTC or other analysis routines do.
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog
User avatar
Qyot27
Surreptitious fluffy bunny
 
Joined: 30 Aug 2002
Location: St. Pete, FL
Status: Creepin' between the bullfrogs

Re: HD video additions to the guides - Ripping

Postby BasharOfTheAges » Sat Dec 19, 2009 1:07 pm

Qyot27 wrote:
BasharOfTheAges wrote:I started with DGAVCIndex and it decided it didn't like certain BRDs. I re-ripped one of them several times to be sure and it didn't work any of the times. Gave several dozen errors (I stopped hitting Yes/No after about 80 or so)

As to the non-frame accuracy, is that for playback? Because I'm just using it to load, filter, and dump the entire disc as a single file anyways. I'd never recommend editing with the scripts themselves with HDDs so cheap nowadays. It's silly.

I do recall that the version of libavcodec it depended on had some caveats, but there seems to be enough buzz around the -NV versions.

It may be for playback, or it may be in general. The underlying point was that it relies on PTS timestamps, and if the PTS is wrong or absent, DSS2 ceases to be frame-accurate. For a quick pass-through it may be okay, but I'm not sure how it would react with filtering that manipulates frames like IVTC or other analysis routines do.

I can totally support the buzz around the -NV versions... if you have a decent GPU, using it in your workflow will do so much more than any new CPU on the market could dream of doing. It doesn't stop it from just not working with certain discs though (Paprika in my case). I certainly wouldn't recommend using something that has documented cases of just not working in practice (as opposed to ones that wouldn't work in theory - unless there's a list of movies that have come out with borked timestamps somewhere).

How common is the last point, though? I've yet to see a BRD that isn't progressive scan (i.e. there's no reason to IVTC) and I can't say I know enough about various filters that play with ordered frames that aren't all about de-interlacing.
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Re: HD video additions to the guides - Ripping

Postby Qyot27 » Sat Dec 19, 2009 2:56 pm

BasharOfTheAges wrote:How common is the last point, though? I've yet to see a BRD that isn't progressive scan (i.e. there's no reason to IVTC) and I can't say I know enough about various filters that play with ordered frames that aren't all about de-interlacing.

Blu-ray can only store interlaced video at 30fps. Hence, anything that's supposed to be 30fps can only be stored as interlaced. For 720 this is less of a concern because 720p60 is supported, but 1080 is restricted to just 1080p24 or 1080i30 (or 1080i60, if you go with the way the spec says it - still the same thing because they're talking fields, not frames).

Whether you see it in the wild is another matter, but the possibility exists (and what are the chances that the weirdness that occurs with DVD hybridism is rendered impotent on Blu-ray?).
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog
User avatar
Qyot27
Surreptitious fluffy bunny
 
Joined: 30 Aug 2002
Location: St. Pete, FL
Status: Creepin' between the bullfrogs

Re: HD video additions to the guides - Ripping

Postby BasharOfTheAges » Sat Dec 19, 2009 4:05 pm

Qyot27 wrote:
BasharOfTheAges wrote:How common is the last point, though? I've yet to see a BRD that isn't progressive scan (i.e. there's no reason to IVTC) and I can't say I know enough about various filters that play with ordered frames that aren't all about de-interlacing.

Blu-ray can only store interlaced video at 30fps. Hence, anything that's supposed to be 30fps can only be stored as interlaced. For 720 this is less of a concern because 720p60 is supported, but 1080 is restricted to just 1080p24 or 1080i30 (or 1080i60, if you go with the way the spec says it - still the same thing because they're talking fields, not frames).

Whether you see it in the wild is another matter, but the possibility exists (and what are the chances that the weirdness that occurs with DVD hybridism is rendered impotent on Blu-ray?).

Must be my sample size then. I'm 4/4 with 1080p24. Mayhaps I'll eventually find something that rips as 1080i30. I take it that's the old film/broadcast dichotomy showing up in this format again?

It's all rather far-reaching to discuss at this point. Until GPU rending becomes commonplace, people aren't going to have the patience to use such large formats - the filtering takes for fucking ever to do. :lol:
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Re: HD video additions to the guides - Ripping

Postby Qyot27 » Sat Dec 19, 2009 5:16 pm

Or at least until AviSynth (and external filters) gets proper 64-bit integration. 2.6 at least will have native multithreading once it reaches final, but adding 64-bit support requires a lot of code gutting and revision from the way I've seen it discussed. The existing AviSynth64 doesn't do enough to fix the problem once and for all.

In the meantime, I'd settle for multi-pass filtering and then just natively use 64-bit interfaces (like outputting to .y4m and giving that file directly to 64-bit x264, or piping).
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog
User avatar
Qyot27
Surreptitious fluffy bunny
 
Joined: 30 Aug 2002
Location: St. Pete, FL
Status: Creepin' between the bullfrogs

Re: HD video additions to the guides - Ripping

Postby mirkosp » Sun Dec 27, 2009 12:57 pm

I worked with the Blu-ray release of Mushi-Shi, which is telecined. Other telecined BDs, for example, are Air and CANAAN, as far as I know. By the way DSS2 did great with mushi-shi, if that's any concern. But well, in my experience I haven't had any frame accuracy issue with DSS2, although I am now aware that there might be (thanks for the link, qyot). An issue I did have with it, though, was when I was trimming. If I had the trim start at 0, everything is fine, but if I had a trim that started at something different than 0, avisynth would shit brix on me. Now since DGAVCIndex isn't an option anymore (no longer supported etc., so I don't trust the use), and I prefer my cpu over my gpu (admittedly, my GPU has CUDA, but it's just a geforce 9600GS, so...), I decided to go with dss2, even more 'cause loading with ffmpegsource/2 would just crash avisynth. >_<
So basically what I do is use dss2 to convert the whole m2ts to lossless, with the quality filters in the chain (generally just antialiasing and/or debanding, if anything, really), and then I do the trimming from the lossless.
I think that in blu-ray cases the PTS timestamps are supposedly reliable, or at least experience so far makes me hope so, although the trimming issue (which I only had on the Macross Zero blu-ray, by the way) admittedly might have to do with it...
Image
User avatar
mirkosp
MODkip
 
Joined: 24 Apr 2006
Location: Gallarate (VA), Italy
Status: (」・ワ・)」(⊃・ワ・)⊃

Re: HD video additions to the guides - Ripping

Postby Qyot27 » Mon Dec 28, 2009 7:54 am

mirkosp wrote:I decided to go with dss2, even more 'cause loading with ffmpegsource/2 would just crash avisynth. >_<

It would crash during the indexing phase or playback? I do find it's far easier (and more convenient) to use ffmsindex to render the index files for FFMS2, although the downside is that it's still just CLI* - albeit there are pushes for a Qt-based GUI for it down the line. That was mentioned on Doom10.

*ridiculously easy:
ffmsindex inputvideo.ext
is all that's required for using FFVideoSource by itself.
ffmsindex -t -1 inputvideo.ext
will act as FFmpegSource("video.ext",atrack=-1) does.
ffmsindex -t -1 inputvideo.ext index.ffindex
can be specified if you want to keep your index files in a specific place and use the cachefile parameter to load them.

The benefit, of course, is that using the indexing app is a lot faster and less hefty than trying to write the index by playing the script in WMP or MPC. And it gives you a progress meter. I added C:\Program Files\AviSynth 2.5\plugins to Windows' PATH, so I don't have to worry about updating it manually every time FFMS2 updates. Otherwise you could just throw it in C:\WINDOWS and be done with it, but have to copy new versions there on each update. The Open Command Prompt Here powertoy from Microsoft or the alternate shell extension from code.kliu.org can really make your life easier with that. Note: Vista, and I also assume Win7, users can just right-click while holding Shift to bring up the native Open Command Window Here option, but it is nice to not have to remember that every time you want to use it.

I think maybe I should write a more general primer on doing CLI work and writing batch scripts so I don't have to keep repeating myself on that.
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog
User avatar
Qyot27
Surreptitious fluffy bunny
 
Joined: 30 Aug 2002
Location: St. Pete, FL
Status: Creepin' between the bullfrogs

Re: HD video additions to the guides - Ripping

Postby Willen » Thu Feb 11, 2010 2:17 am

Qyot27 wrote:Blu-ray can only store interlaced video at 30fps. Hence, anything that's supposed to be 30fps can only be stored as interlaced. For 720 this is less of a concern because 720p60 is supported, but 1080 is restricted to just 1080p24 or 1080i30 (or 1080i60, if you go with the way the spec says it - still the same thing because they're talking fields, not frames).

Whether you see it in the wild is another matter, but the possibility exists (and what are the chances that the weirdness that occurs with DVD hybridism is rendered impotent on Blu-ray?).


I think you meant to type 1080p30.

The reason the BD specs say 1080i60 and not 1080p30 is because if done properly, you can use that interlaced signal to store true 60i and 30p footage. If you are encoding native 1080i60 ATSC video you are golden. If you have 1080p footage shot at 30 frames per second, possibly from some AVCHD cameras or video-capable DSLRs, it would be stored as interlaced field pairs on the BDs (i.e. 1080p30 converted to 1080i60). It should be no different than how DVDs handle pure interlaced material derived from 30 (29.97) frames per second original footage. De-interlacing on playback should be pretty straightforward to reconstruct the original 30 frames.

As for if the footage is properly flagged as pure interlaced or field pairs for correct playback, I don't know...

BTW, most 1080i60 BDs I've encountered are of the National Geographic documentary type. Most of them are shot with HD video cameras which operate at 60i (NTSC/ATSC). I would imagine the expense of shooting in 35mm film (or even IMAX) would go above the budgets of most of the programs.
Having trouble playing back videos? I recommend: Image
User avatar
Willen
Now in Hi-Def!
 
Joined: 10 Jul 2005
Location: SOS-Dan HQ
Status: Melancholy

Next

Return to Video & Audio Help

Who is online

Users browsing this forum: No registered users and 4 guests