All right, so I'm toying around with the idea for a Haruhi AMV... and since I've been collecting the 1280x720 raws, I figure it'll look a lot better than most stuff gotten raw.
But then I run into problems. The biggest being, there are occasionally errors on straight lines.
Examples:
http://img247.imageshack.us/img247/7100 ... rez8iw.png
http://img192.imageshack.us/img192/3619 ... uki4an.png
On the prez, it's on his collar and his mouth. On Yuki, it's her chin and that collar section. These happen all over the place randomly.
So, has anyone worked with this source before? How did you deal with these? Leave 'em in, or can they be filtered out?
Thanks, everybody.
The Melancholy of Haruhi Suzumiya [HD Raw Question]
-
shiro_clanclan
- Joined: Sun Jun 09, 2002 8:59 pm
- Location: Concord, CA
-
Qyot27
- Surreptitious fluffy bunny
- Joined: Fri Aug 30, 2002 12:08 pm
- Status: Creepin' between the bullfrogs
- Location: St. Pete, FL
- Contact:
You could try TIsophote (on this page: http://www.missouri.edu/~kes25c/), but I don't know how much that'll do. It actually kinda looks like somebody already tried using it on the source, but I'm not sure.
If the episodes are at 120fps like a lot of RAWs seem to be, there's also a utility on that page I linked to that takes out all the dummy frames. It sets everything at the same frame-rate, so some sections will play a little fast (or maybe slow, can't remember what sections get modified), but since it's meant for editing that's not a problem. You can slow those parts back down in the editing program.
If the episodes are at 120fps like a lot of RAWs seem to be, there's also a utility on that page I linked to that takes out all the dummy frames. It sets everything at the same frame-rate, so some sections will play a little fast (or maybe slow, can't remember what sections get modified), but since it's meant for editing that's not a problem. You can slow those parts back down in the editing program.
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog
-
shiro_clanclan
- Joined: Sun Jun 09, 2002 8:59 pm
- Location: Concord, CA
-
DJ_Izumi
- Joined: Wed Oct 03, 2001 8:29 am
- Location: Canada
- Contact:
-
shiro_clanclan
- Joined: Sun Jun 09, 2002 8:59 pm
- Location: Concord, CA
Seeing the raws for .hack//ROOTS leads me to believe that you're absolutely right about that. ROOTS is the same res as these Haruhi raws but looks infinitely better when it comes to lines and such. It's clearer.
I agree, they have issues. But it's not like the final video's gonna be 1280x720 anyway. XP I'll end up with 640x360 or 856x480, probably.
I agree, they have issues. But it's not like the final video's gonna be 1280x720 anyway. XP I'll end up with 640x360 or 856x480, probably.
-
DJ_Izumi
- Joined: Wed Oct 03, 2001 8:29 am
- Location: Canada
- Contact:
Upscale -> Downscale is not an ideal route to go for quality. Since in the upscale you lose some detail and it's not recovered by down scaling. (Unless you did a nearest neighbour resize up and down, to what was exactly TWICE the original resolution.
I suggest you try to find some cleaner source material, HD or not HD. Cause, to be honest, that footage isn't that great looking.
I suggest you try to find some cleaner source material, HD or not HD. Cause, to be honest, that footage isn't that great looking.
-
shiro_clanclan
- Joined: Sun Jun 09, 2002 8:59 pm
- Location: Concord, CA
-
Zero1
- Joined: Fri Jan 02, 2004 12:51 pm
- Location: Sheffield, United Kingdom
- Contact:
Recently HD or High Definition is becoming more and more of a buzzword, and everyone wants to get in on it. This is no exception for AMV editors, or raw cappers (who capture the TV stream in Japan and distribute it online).
Being a fansubber and encoder, I usually check out raws even if it's not a project I am working on, I do it as a matter of interest, and what I have been seeing recently is an increasing number of "HD raws". These "HD raws" are more often than not, upscales. Upscales are basically standard resolution video (720x480) that is resized to another resolution, which in this case happens to be 1280x720 which is a HD resolution.
So just why would someone upscale a video?
It really is beyond me why anyone would do such a thing, but perhaps I am too involved in encoding to be able to look from the outside in. The most obvious reason I could think of would be "raw capper rivalry" commonly known as e-penis. "LAWL my caps are HD and yours are 640x360". It's similar to 119.88fps files, I'm sure you've come across them at some time. Basically 119.88fps is a convienient way of making a variable frame rate video (such as MKV with timecodes), constant framerate by adding in null/placeholder frames (for example AVI). With it being 119.88, it means that it can cleanly be decimated to 29.97 or 23.976 (if you intend to put out a true CFR version of it).
However, some raw cappers are putting out 119.88fps files when they don't need to be. Out of curiosity I took a 119.88 fps Gundam Seed Destiny OP (ew) episode and transmuxed it to MP4 (the original intent was just to see how much space it would save). MP4box automatically drops NVOPs; NVOPs are basically the null or placeholder frames which were originally intended for VFR, but later hacked by DivX as one of the workarounds for B-frames in AVI/VfW. The resulting file was pure CFR 23.976 fps which means the null frames had no affect on the video and were added for no purpose.
Raw cappers seem to like doing weird things that make their caputres look elite. Unfortunately 119.88fps for the sake of it, and upscales are about the biggest failures around.
Other than that, I cannot think of a legitimate reason for upscaling it. For those of you who are curious, upscaling is bad for a number of reasons. The two biggest reasons are:
a) A higher resolution requires more bitrate or filesize (which means "HD" raws of the same filesize invariably look worse)
b) Resizing filters often introduce artifacts to the image such as ringing (usually known as haloing)
The top and bottom of it, is you simply cannot increase the quality by upscaling it. You cannot add detail that did not exist to start with.
How do you come to the conclusion that these are mostly upscales?
I will say with confidence that the images of Haruhi posted are from and upscale.
Firstly there's the visual side of it. The resolution might be large, but most good encoders would be able to spot the side effects of AVISynth filters, such as awarpsharp (haloing, inconsistant sharpening and "lumpy lines" for want of a better word), lanczos resize (more haloing) etc. The other thing, noticed by shiro_clanclan is other artifacts in the video. The inconsistant lines he talks about are IVTC/deinterlacing artifacts... Hold that thought, I will come to this later.
Now the visual side of it is all well and good, but not 100% reliable since perception varies from person to person.
Even if you don't know for a fact that this is an upscale, with some knowledge of video coding/broadcast, you can tell this is an upscale. Let me explain.
Now lets come back to those IVTC/deinterlacing artifacts...
1) True 720p video is progressive, this means no interlacing which in turn means there can be no interlacing artifacts since there wasn't any interlacing to get rid of to begin with!
2) Additionally, there is no such thing as 720i, and if there was, it would be pure interlaced (not hybrid like NTSC DVDs). Pure interlaced means that you would not have any progressive frames at all, and it's a dog to look at on a progressive monitor. This raw has a mix progressive and interlaced frames, which points to a 29.97fps telecined source.
3) 1080i is also pure interlaced, so they cannot claim it's a downscaled 1080i. Even if you discarded one field, you would only get 544 lines anyway (true 1080i resolution is 1088 pixels).
4) 1080p exists, but due to it's high bandwith usage and high processor requirements, no one broadcasts it. This again means that they cannot call it a downscaled 1080p. Similarly watch out for "1080p raws", these are bullshit too. If no one is broadcasting it, how can people capture it? Chances are it's an upscaled true 720p.
But I know for a fact this show airs in HD!
I'm not discounting whether it's aired in HD or not, but I'm discounting that the end product is true HD or not. There are certain limiting factors to whether a raw capper can get the video in it's true resolution.
There are two ways you can capture TV programs airing in Japan, one would be with a DVB/Cable/Satellite tuner in your PC, or an external DVB/Cable/Satellite decoder (such as the Satellite receiver you use to watch Sky or something) hooked up to an analog capture card via S-Video cable for example.
PC DVB/Cable/Satellite cards have no way of accessing your subscription information, which means you can only watch the free channels since the others are encrypted. If a show airs in HD on a free to view channel, you may use your DVB/Cable/Satellite card and software to be able to dump the raw data as it's received. It's usually MPEG-2 video with MPEG-1 audio in an MPEG-2 transport stream (.ts). Dumping the stream does not alter the quality at all, nor the resolution. This is how you get true HD raws.
What about premium/PPV channels? Well obviously you pay for or subscribe to them, and the information is contained on a credit card like subscriber identity module. When you come to watch them on your TV, the receiver checks that you have the rights to view that channel and decrypts/descrambles it so you can watch it. As I said before, the PC counterpart does not have access to this module so in turn even if you subscribe to these, the PC doesn't know about it, or is able to know, therefore all non free channels are locked. This is where the analog method comes into play. With no way to view the encrypted channel on the PC or stream dump it, the only option left is to connect your external DVB/Cable/Satellite receiver into an analog TV card and capture it.
Now here's the interesting part.
Even if the show is being broadcast in HD, even if you have your receiver hooked up to the TV capture card, it's still not possible to get a HD capture. Why? Well simply because TV capture cards are designed for either PAL and/or NTSC, this means the vertical resolution will be at best 480 or 576 pixels.
This method has 2 or 3 possible points of quality loss. First off you have the cable quality, then connection quality in the TV card and receiver and lastly some software is restrictive and forces you to encode on the fly to a certain format (usually MPEG1 or 2).
So to summerise; if the show is broadcast in HD and shown on a free to air channel, it's possible to streamdump true HD raws. If the show is on a premium channel, the PC receiver will not be able to descramble it, which means you have to use an external receiver hooked up to an analog TV capture card, which has a max resolution of 480/576 pixels high, meaning no true HD raws.
Don't get too excited about so called HD raws, a lot of them are bullshit right now, but you occasionally get true HD raws, and when you see a HD raw you will know that it is. There's no mistaking real HD.
Being a fansubber and encoder, I usually check out raws even if it's not a project I am working on, I do it as a matter of interest, and what I have been seeing recently is an increasing number of "HD raws". These "HD raws" are more often than not, upscales. Upscales are basically standard resolution video (720x480) that is resized to another resolution, which in this case happens to be 1280x720 which is a HD resolution.
So just why would someone upscale a video?
It really is beyond me why anyone would do such a thing, but perhaps I am too involved in encoding to be able to look from the outside in. The most obvious reason I could think of would be "raw capper rivalry" commonly known as e-penis. "LAWL my caps are HD and yours are 640x360". It's similar to 119.88fps files, I'm sure you've come across them at some time. Basically 119.88fps is a convienient way of making a variable frame rate video (such as MKV with timecodes), constant framerate by adding in null/placeholder frames (for example AVI). With it being 119.88, it means that it can cleanly be decimated to 29.97 or 23.976 (if you intend to put out a true CFR version of it).
However, some raw cappers are putting out 119.88fps files when they don't need to be. Out of curiosity I took a 119.88 fps Gundam Seed Destiny OP (ew) episode and transmuxed it to MP4 (the original intent was just to see how much space it would save). MP4box automatically drops NVOPs; NVOPs are basically the null or placeholder frames which were originally intended for VFR, but later hacked by DivX as one of the workarounds for B-frames in AVI/VfW. The resulting file was pure CFR 23.976 fps which means the null frames had no affect on the video and were added for no purpose.
Raw cappers seem to like doing weird things that make their caputres look elite. Unfortunately 119.88fps for the sake of it, and upscales are about the biggest failures around.
Other than that, I cannot think of a legitimate reason for upscaling it. For those of you who are curious, upscaling is bad for a number of reasons. The two biggest reasons are:
a) A higher resolution requires more bitrate or filesize (which means "HD" raws of the same filesize invariably look worse)
b) Resizing filters often introduce artifacts to the image such as ringing (usually known as haloing)
The top and bottom of it, is you simply cannot increase the quality by upscaling it. You cannot add detail that did not exist to start with.
How do you come to the conclusion that these are mostly upscales?
I will say with confidence that the images of Haruhi posted are from and upscale.
Firstly there's the visual side of it. The resolution might be large, but most good encoders would be able to spot the side effects of AVISynth filters, such as awarpsharp (haloing, inconsistant sharpening and "lumpy lines" for want of a better word), lanczos resize (more haloing) etc. The other thing, noticed by shiro_clanclan is other artifacts in the video. The inconsistant lines he talks about are IVTC/deinterlacing artifacts... Hold that thought, I will come to this later.
Now the visual side of it is all well and good, but not 100% reliable since perception varies from person to person.
Even if you don't know for a fact that this is an upscale, with some knowledge of video coding/broadcast, you can tell this is an upscale. Let me explain.
Now lets come back to those IVTC/deinterlacing artifacts...
1) True 720p video is progressive, this means no interlacing which in turn means there can be no interlacing artifacts since there wasn't any interlacing to get rid of to begin with!
2) Additionally, there is no such thing as 720i, and if there was, it would be pure interlaced (not hybrid like NTSC DVDs). Pure interlaced means that you would not have any progressive frames at all, and it's a dog to look at on a progressive monitor. This raw has a mix progressive and interlaced frames, which points to a 29.97fps telecined source.
3) 1080i is also pure interlaced, so they cannot claim it's a downscaled 1080i. Even if you discarded one field, you would only get 544 lines anyway (true 1080i resolution is 1088 pixels).
4) 1080p exists, but due to it's high bandwith usage and high processor requirements, no one broadcasts it. This again means that they cannot call it a downscaled 1080p. Similarly watch out for "1080p raws", these are bullshit too. If no one is broadcasting it, how can people capture it? Chances are it's an upscaled true 720p.
But I know for a fact this show airs in HD!
I'm not discounting whether it's aired in HD or not, but I'm discounting that the end product is true HD or not. There are certain limiting factors to whether a raw capper can get the video in it's true resolution.
There are two ways you can capture TV programs airing in Japan, one would be with a DVB/Cable/Satellite tuner in your PC, or an external DVB/Cable/Satellite decoder (such as the Satellite receiver you use to watch Sky or something) hooked up to an analog capture card via S-Video cable for example.
PC DVB/Cable/Satellite cards have no way of accessing your subscription information, which means you can only watch the free channels since the others are encrypted. If a show airs in HD on a free to view channel, you may use your DVB/Cable/Satellite card and software to be able to dump the raw data as it's received. It's usually MPEG-2 video with MPEG-1 audio in an MPEG-2 transport stream (.ts). Dumping the stream does not alter the quality at all, nor the resolution. This is how you get true HD raws.
What about premium/PPV channels? Well obviously you pay for or subscribe to them, and the information is contained on a credit card like subscriber identity module. When you come to watch them on your TV, the receiver checks that you have the rights to view that channel and decrypts/descrambles it so you can watch it. As I said before, the PC counterpart does not have access to this module so in turn even if you subscribe to these, the PC doesn't know about it, or is able to know, therefore all non free channels are locked. This is where the analog method comes into play. With no way to view the encrypted channel on the PC or stream dump it, the only option left is to connect your external DVB/Cable/Satellite receiver into an analog TV card and capture it.
Now here's the interesting part.
Even if the show is being broadcast in HD, even if you have your receiver hooked up to the TV capture card, it's still not possible to get a HD capture. Why? Well simply because TV capture cards are designed for either PAL and/or NTSC, this means the vertical resolution will be at best 480 or 576 pixels.
This method has 2 or 3 possible points of quality loss. First off you have the cable quality, then connection quality in the TV card and receiver and lastly some software is restrictive and forces you to encode on the fly to a certain format (usually MPEG1 or 2).
So to summerise; if the show is broadcast in HD and shown on a free to air channel, it's possible to streamdump true HD raws. If the show is on a premium channel, the PC receiver will not be able to descramble it, which means you have to use an external receiver hooked up to an analog TV capture card, which has a max resolution of 480/576 pixels high, meaning no true HD raws.
Don't get too excited about so called HD raws, a lot of them are bullshit right now, but you occasionally get true HD raws, and when you see a HD raw you will know that it is. There's no mistaking real HD.
7-zip // x264 (Sharktooth's builds) // XviD (Koepi's builds) // MP4box (celtic_druid's builds) // Firefox // CCCP
-
shiro_clanclan
- Joined: Sun Jun 09, 2002 8:59 pm
- Location: Concord, CA
I have to agree wholeheartedly with Zero1. I was re-watching an episode of Haruhi today (the raw for episode 2), and noticed the one thing that should've tipped me off to it being an upscale. At one point, Haruhi raises her right arm... and it gets very, very bright. Like the white bleeds out of her sleeve more than the blood of guard #1 in Elfen Lied. I should've seen that. Yes, the lines actually look quite clean in a few places--the tenth episode is a great example of a miraculous result--but overally the quality is... not... HD.
.hack//ROOTS, though, I am fairly certain is HD. It's broadcast in HD (has the watermark for Japanese HDTV), and doesn't have all the problems found in the upscaled Haruhi raws. Not that it matters what the end result is, but I noticed that .hack is recorded in h264 whereas Haruhi is done in (yep, 119.88) XviD.
So yeah. I noticed that l33t-raws was much quicker than I when it came to figuring all this out, and have been getting the 704x396 raws. Guess I'll wait for them to release a batch, then re-collect raws from scratch.
Thanks for all the help--hopefully this'll assist the two planned Haruhi projects I have to look as good as possible... I just can't stand the artifacts on slowly-moving objects (shirts, mouths, writing...).
.hack//ROOTS, though, I am fairly certain is HD. It's broadcast in HD (has the watermark for Japanese HDTV), and doesn't have all the problems found in the upscaled Haruhi raws. Not that it matters what the end result is, but I noticed that .hack is recorded in h264 whereas Haruhi is done in (yep, 119.88) XviD.
So yeah. I noticed that l33t-raws was much quicker than I when it came to figuring all this out, and have been getting the 704x396 raws. Guess I'll wait for them to release a batch, then re-collect raws from scratch.
Thanks for all the help--hopefully this'll assist the two planned Haruhi projects I have to look as good as possible... I just can't stand the artifacts on slowly-moving objects (shirts, mouths, writing...).
-
DJ_Izumi
- Joined: Wed Oct 03, 2001 8:29 am
- Location: Canada
- Contact:
Typically you can just tell it's an upscale just by looking at it, to be HD it sorta HAS to be crisp and clear.
Not that people havn't made mistakes. Zero1 wouldn't stop saying my HD PlanetES episodes were upscales untill I gave him a screen shot from the HD's where you could read every word on a page from a book on screen where as in the DVD screen s hot at 480p, it was mostly blurry and there was no ammount of filtering that would have made it readable.
Not that people havn't made mistakes. Zero1 wouldn't stop saying my HD PlanetES episodes were upscales untill I gave him a screen shot from the HD's where you could read every word on a page from a book on screen where as in the DVD screen s hot at 480p, it was mostly blurry and there was no ammount of filtering that would have made it readable.