MD5 / CRC check sum to confirm upload

Locked
User avatar
Eek-1
Joined: Sat Jun 22, 2002 10:06 am
Status: 0xc00000e9
Location: Cyberjaya
Org Profile

Post by Eek-1 » Thu Sep 15, 2005 10:19 am

bum wrote: - So is using this forums search function and google :roll:
Alright. My fault.

http://www.animemusicvideos.org/phpBB/v ... hp?t=53062
http://www.animemusicvideos.org/phpBB/v ... hp?t=47767
http://www.animemusicvideos.org/phpBB/v ... hp?t=42909


The reason I asked for this feature is because I had videos downloaded from here which later I found they got corrupted during the transfer. Euphoria was one of them.

trythil
is
Joined: Tue Jul 23, 2002 5:54 am
Status: N͋̀͒̆ͣ͋ͤ̍ͮ͌ͭ̔̊͒ͧ̿
Location: N????????????????
Org Profile

Post by trythil » Thu Sep 15, 2005 10:28 am

Eek-1 wrote: The reason I asked for this feature is because I had videos downloaded from here which later I found they got corrupted during the transfer. Euphoria was one of them.
Um, this won't save you much time at all (and it certainly won't reduce the amount of data transferred) if you're getting errors when downloading videos, especially when they play fine for other people. MD5 sums aren't error-correction codes.

User avatar
Eek-1
Joined: Sat Jun 22, 2002 10:06 am
Status: 0xc00000e9
Location: Cyberjaya
Org Profile

Post by Eek-1 » Thu Sep 15, 2005 11:34 am

trythil wrote:
Eek-1 wrote: The reason I asked for this feature is because I had videos downloaded from here which later I found they got corrupted during the transfer. Euphoria was one of them.
Um, this won't save you much time at all (and it certainly won't reduce the amount of data transferred) if you're getting errors when downloading videos, especially when they play fine for other people. MD5 sums aren't error-correction codes.
Oh yeah, I totally thought about that. Maybe I should add some more to my reason:

From those experiences I've become concerned about my videos might get corrupted whenever I upload them. However so far they were all fine since I always d/l and then hash them. But most of my AMVs are above 50MBs so the downloads were always slow and tedious and had wasted my bandwidth $.

User avatar
Zarxrax
Joined: Sun Apr 01, 2001 6:37 pm
Contact:
Org Profile

Post by Zarxrax » Thu Sep 15, 2005 11:46 am

bum wrote:
Zarxrax wrote:
trythil wrote:MD5 sums are kept, I think, but I wonder how many people would actually take the time to generate MD5 or CRC hashes of their videos and compare, rather than going through the old way and just downloading the thing again.
I would...
You wouldnt be stupid enough to screw up an upload in the first place.
It's not a matter of being stupid. File transfers can screw up, no matter who you are. I used to have some serious problems with my connection, where practically ANY file that I tried uploading or downloading would end up corrupted most of the time. File corruption can happen at anytime, particularly if you have to stop and resume your upload. There is no way to actually verify that your upload is actually correct without either downloading it and checking it, or checking its checksum. Just because the client says it uploaded, and the number of bytes means the same... that doesn't really mean anything.

User avatar
bum
17747114553
Joined: Sat Nov 08, 2003 9:56 pm
Org Profile

Post by bum » Thu Sep 15, 2005 8:12 pm

I didnt mean that against you. What I meant is that most threads on the help boards could be answered within 15 seconds of using the forum search. Even the forum stickies dont seem to do much. If we add something like MD5 it will result in more repeate threads.

User avatar
derobert
Phantom of the .Org
Joined: Wed Oct 24, 2001 8:35 am
Location: Sterling, Virginia
Contact:
Org Profile

Post by derobert » Fri Sep 16, 2005 11:21 pm

Eek-1 wrote:The reason I asked for this feature is because I had videos downloaded from here which later I found they got corrupted during the transfer. Euphoria was one of them.
Ummm. Fix your computer, your network, whatever. If they are corrupting data they are _broken_ and I certainly wouldn't trust a computer, network, etc. that routinely destroyed my data.
Key 55EA59FE; fingerprint = E501 CEE3 E030 2D48 D449 274C FB3F 88C2 55EA 59FE
A mighty order of ages is born anew.              http://twitter.com/derobert

User avatar
derobert
Phantom of the .Org
Joined: Wed Oct 24, 2001 8:35 am
Location: Sterling, Virginia
Contact:
Org Profile

Post by derobert » Fri Sep 16, 2005 11:29 pm

FYI:

The reason we don't display the MD5 checksum when you're confirming the video is because we haven't calculated it yet. Butter calculates the MD5 *after* you have confirmed the video.

The reason we don't display the MD5 on the video info pages is because, well, it'd confuse 99% of the people who visit that page. I suppose it could be added here, labeled 'checksum' or somesuch.
Key 55EA59FE; fingerprint = E501 CEE3 E030 2D48 D449 274C FB3F 88C2 55EA 59FE
A mighty order of ages is born anew.              http://twitter.com/derobert

User avatar
SpringBoy
Joined: Sat Mar 26, 2005 8:03 am
Location: Melbourne, Australia
Org Profile

Post by SpringBoy » Sat Sep 17, 2005 12:16 am

So hard would it be to add the checksum calculation *before* confirming the upload?

If it could be done it can just be that people can choose if they want to confim via redownloading, confirm via checksum or just take their chances.

User avatar
derobert
Phantom of the .Org
Joined: Wed Oct 24, 2001 8:35 am
Location: Sterling, Virginia
Contact:
Org Profile

Post by derobert » Sat Sep 17, 2005 12:58 am

SpringBoy wrote:So hard would it be to add the checksum calculation *before* confirming the upload?
It'd unfortuantely take a reasonable bit of work. The reason is that (a) a checksum calculation can be time consuming (e.g., up to 15s or so, depending on system load); (b) the program on Kero that handles things like that can only do one thing at a time. So while a 15s md5sum calculation is going, none of the things that Pancake does would happen. These include:

- Looking up videos in the database to start downloads
- Figuring out upload status
- Inserting "video is done downloading" into the database
- Finding videos to do test downloads for confirming

Those are rather important things, so it'd be a bad idea to block them for 15s. That's why it'd take some work to do.
Key 55EA59FE; fingerprint = E501 CEE3 E030 2D48 D449 274C FB3F 88C2 55EA 59FE
A mighty order of ages is born anew.              http://twitter.com/derobert

User avatar
Zarxrax
Joined: Sun Apr 01, 2001 6:37 pm
Contact:
Org Profile

Post by Zarxrax » Sat Sep 17, 2005 12:00 pm

So the server cant do multitasking? O.o

Locked

Return to “Site Help & Feedback”