i was banned for an unjust cause

This forum is for the general discussion of Anime Music Videos.

Postby Hakura » Fri Nov 15, 2002 5:13 pm

Hey Phade, speaking of Bandwidth issuses, I just got my site back up today, we have 400 gigs a month to use, but I don't want people abusing it...if you di find a way to stop bandwidth usages or can help detect when that could happen, it would be a great help ^^

I'm new to the deticated server thing ^^ also with dl accelators I'd like to know about that too ^^
User avatar
Hakura
 
Joined: 30 May 2001
Location: Winnipeg, Canada

Postby AbsoluteDestiny » Fri Nov 15, 2002 6:01 pm

Hakura wrote:Hey Phade, speaking of Bandwidth issuses, I just got my site back up today, we have 400 gigs a month to use, but I don't want people abusing it...if you di find a way to stop bandwidth usages


say it's an indirect link

people are lazy.
User avatar
AbsoluteDestiny
 
Joined: 15 Aug 2001
Location: Oxford, UK

Postby Phade » Fri Nov 15, 2002 6:03 pm

Hey,

From what I know of these "download accelerators", many try to circumvent per-connection bandwidth caps placed on downloads by the hosting server. Server bandwidth caps placed by the server owner try to more evenly distribute the available bandwidth to users by limiting the amount of bandwidth used by an individual download. The accelerator opens multiple connections to the download server and uses the combined connections to download different parts of the file simultaneously.

For example, if an accelerator is set to spawn 4 downloads at once; the first download will try to download the first 25%, the second download connection the second 25% and so on. When each download connection finishes downloading their parts, the accelerator combines the parts into one complete file. Assuming all connections were successful, the download took 1/4 the time it normally would because it had allocated to itself 4x the connections to get the file.

This is all fine and dandy (if you don't count being a dick by bumping off 3 other user's connections to get the file 4x faster). But if the accelerator user screws up the settings, the accelerator may try to reconnect or redownload segments of the file unnecessarily. The end result here is that it costs much more time and resources to get the user the file than if they just downloaded it normally. I believe this was the case with this person trying to download my video several hundred times in a row (that, or they were intentionally being a dick).

There is a different type of product called a "download resumer" which is rather useful, especially to dial-up users. The resumer allows downloads to be paused and resumed later if the connection is broken (like you have to use the phone in the middle of a 3 hour download). The resumer picks up where the download left off once the connection is restored.

Something that can go wrong with resumers is the time interval between resumes. If the "check for resume" is set to once a minute or so, things are nice for the server. But if the "check for resume" is set for 60 times each second, this hammering of the server kills the server performance for everyone else using the server.

Now the trick is to detect when someone is doing a normal download vs. trying to skrew everyone else by trying to hog bandwidth or hammer the server. Figuring this out is one of the things on the Golden Donut list of stuff to do before rolling it out.

Phade.
User avatar
Phade
Site Admin
 
Joined: 20 Oct 2000
Location: Little cabin in the woods...

Postby Hakura » Fri Nov 15, 2002 6:09 pm

Ahh...I see.. ^^ thanks for the info Phade ^^ :wink:
User avatar
Hakura
 
Joined: 30 May 2001
Location: Winnipeg, Canada

Postby Nappy » Fri Nov 15, 2002 6:16 pm

might this be the reason I see some vids with like 2k hits and 10k d/l? or are they just trying to get on the under rated list?
User avatar
Nappy
 
Joined: 25 Jun 2002
Location: S. CA (helLA) Posts :0

Download Accelerators

Postby XStylus » Fri Nov 15, 2002 8:11 pm

Phade wrote:Hey,

From what I know of these "download accelerators", many try to circumvent per-connection bandwidth caps placed on downloads by the hosting server. Server bandwidth caps placed by the server owner try to more evenly distribute the available bandwidth to users by limiting the amount of bandwidth used by an individual download. The accelerator opens multiple connections to the download server and uses the combined connections to download different parts of the file simultaneously.


I use a program called StarDownload. It's a decent program (has a few bugs, but there's no adware like Download Accelerator), although I have a 56k connection, so I use it mainly as a download resumer and have the download 'accelerating' feature disabled since it does me no good. Unfortunately most 56k users who use such programs aren't as considerate.

However, I really like the idea of download accelerators to overcome bandwidth caps. It makes perfect sense to me. Some cablemodem and DSL ISPs have bandwidth caps, so people use those download accelerators to overcome those too. I do understand a site-host's standpoint in putting bandwidth caps so that more people can download a file, but you need to think logically about it for a sec: Which is better?: Having mutiple users downloading a file slowly, each with their thumb up their ass having to wait for it to finish downloading, or one user downloading a file REALLY quickly, gets off the connection and enjoys it while another user downloads it quickly?

The problem is the 56k users who use Download accelerators. They download files slowly regardless of whether there is a bandwidth cap or not, and somehow think a download accelerator will help them. Another problem is the idiots who tinker with the settings in these download accelerators without knowing what their doing, setting them to download at 1/100 x100 rather than the default 1/4 x4. Site hosts probably wouldn't have such a problem with DLAs if it weren't for those morons.

I don't know a lot about site hosting, but I do know that some have their sites set up so that only one connection from the same IP may be connected. Some standard FTP programs will hiccup over that (such as FTPX/FTP Explorer), but that will quite effectively put the kabosh on multiple FTP connections and download accelerators, if that's what you really want to do.

However, It's my suggestion that if you do put such a block on your site, set it for a max of 4 connections from one IP, rather than 1. The next thing you should do is put a minimum speed limit on your site... 2kbps sounds about right. This should (hopefully) take care of 56k idiots downloading at 1/4 x4, while benefiting those blessed with fast connections. Also, if a person 'hammers' a site with repeated connection attempts after getting speed-bumped or trying to do more than 4 connections, you can set up software to automatically add that person to a ban list temporarily or permanently (depending on your mood).

Don't ask me how to do any of that stuff though. That's just stuff I've seen other FTP hosts do with varied degrees of effectiveness.
User avatar
XStylus
 
Joined: 22 Aug 2001
Location: A quaint little village.
Status: Enjoying the salty air.

Postby Zarxrax » Fri Nov 15, 2002 9:54 pm

There's one way that I know of to prevent people from hammering. However I'm sure EVERYONE would hate this method. Still, if there's nothing else, it might be the only thing you could do. You ever see those sites where the only way to download files from the site is if you first download and install their proprietary downloading software? You download this program from them, then the program will connect and download whichever file it was that you wanted.
User avatar
Zarxrax
 
Joined: 01 Apr 2001
Location: Concord, NC

Postby Phade » Fri Nov 15, 2002 10:03 pm

Hey xstylus,

Cable/DSL caps are "client side" and do not affect server performance. The assumption here is that the server is set to a lower speed than the max speed of the cable/DSL. The max speed of the cable/DSL cannot be overcome with "accelerators".

You assume that the person hogging the bandwidth will go away quickly after a download. Suppose that they queue up 50 or so videos to download so that while they are watching one, they are downloading more videos. So now instead of sharing a connection, one person gets all the bandwidth for an hour while everyone else gets "server is full" error messages and gets no bandwidth at all. Sure the one guy is happy to be downloading, but there are a bunch of other people very unhappy because they get nothing whatsoever.

I plan on making the max connections to be 2: one to download a video and one to surf the site, or download two videos at a time and that's it.

There are always going to be somthing new that will effectively circumvent the max connection limit as well. It will be a race, but I want to be prepared.

Phade.
User avatar
Phade
Site Admin
 
Joined: 20 Oct 2000
Location: Little cabin in the woods...

Postby klinky » Sat Nov 16, 2002 1:15 am

I had the idea for a download accelerator way back in what I think it was 1999, two weeks later FlashGet appears.... I don't know exactly who was the first to make them, but I swear I had the idea FIRST :p, I just didn't know how to implement it... Thank god, They are aweful devices >_<


~klinky
User avatar
klinky
 
Joined: 23 Jul 2001
Location: Cookie College...

Postby Anime Jedi » Sat Nov 16, 2002 12:51 pm

There's only one vid that I'm almost completely sure used footage from another music video. And it only makes sense that it did too. Since both vids are EXACTLY the same, execpt for the music. Of course, because of this, one had good sync, and the other did not. Weirdest thing I ever saw. :lol: But I'm not pointing any fingers, incase I'm wrong, which I really doubt it. :wink:
User avatar
Anime Jedi
 
Joined: 19 May 2002
Location: Wandering Aimlessly (Canada)

Postby RadicalEd0 » Sat Nov 16, 2002 2:46 pm

heheheh. quu feels like an idiot now >:]
btw, about the 200 ip business, while it may suck being on aohell and don't think I use it by choice, the one advantage is all the IPs are routed back to aol HQ in virginia and every time a member logs on a new random one is assigned to them, so it would be impossible to ban someone using aol wthout banning all aol users (like irc.aniverse.com does)
even crappy isps come in handy :lol:
not that phade would ever ban me
NMEAMV: PENIS
NMEAMV: IN
NMEAMV: YO
NMEAMV: MIXED
NMEAMV: DRINK
User avatar
RadicalEd0
 
Joined: 24 Jun 2002

Postby Tycho2k » Sat Nov 16, 2002 3:01 pm

Don't know if anyone will read this one, especially Phade (whom is is directed at), but although I am by far not a server guru, I run an FTP off my home comp with Serv-U FTP, and I can limit a certain number of logins from the same IP, and I find this limits Download Accelerators quite well, since whenever I see someone using one I get multiple logins from the same IP. Don't know if this helps any, just a thought (and trying to get myself noticed by Phade. I am the guy who emailed you a couple times last year about my AMV database program. :D )
"To Sleepy-Time, to Death's embrace,
To all the Horrors I must face.
Zombie hordes, Terrors unknown,
Psyche scars are deeply sown.
But worry not, I can defend,
Always Me and my MAC10."

Written late at night after playing way too much Resident Evil. :)
Tycho2k
 
Joined: 27 Jan 2002

Postby RadicalEd0 » Mon Nov 18, 2002 3:53 pm

oh, sorry :mrgreen: ;;;;
I meant akirahhz

dunno why I said quu... mustve been one of those days
damn non-edit button
NMEAMV: PENIS
NMEAMV: IN
NMEAMV: YO
NMEAMV: MIXED
NMEAMV: DRINK
User avatar
RadicalEd0
 
Joined: 24 Jun 2002

Previous

Return to General AMV

Who is online

Users browsing this forum: No registered users and 2 guests