More Bandwidth and Revenue Needed

This forum is for site announcements. Please go here to read the SITE & FORUM RULES.

Postby Coderjoe » Tue Oct 04, 2005 9:49 am

Coderjoe wrote:Hmm.. Within the month of august, Animix was downloaded 768 times, using a total of 557GB of bandwidth. There are 10 videos that, in August alone, used over 100GB apiece. Perhaps BitTorrent might be worthwhile for very popular vids. All but three of the top ten bandwidth users from August had over 1000 downloads.

Thank you, (someone), for providing the statistics.


hmm.

On the other hand... AniMix only used 1.93% of the total local download amount in August...

xort wrote: And just how much memory do you think it takes to track lets say 25,000 torrents? Do you know?


Do you know? Also, how many downloaders do you expect per torrent, on average?

Bittorrent really isn't wothwhile for the mass of videos being downloaded a few times a month. Especially when many people are likely to close the torrent once their download is complete. Additionally, who do you think would be seeding the torrents? More than likely the same download server that is storing the video and currently sends it out via http already.

Bittorrent might be worthwhile for videos with a large number of downloads per month, or for top bandwidth consumers in a month, but not for the peek on Phade's curve.
User avatar
Coderjoe
 
Joined: 03 Mar 2001

Postby RECONstiPated » Tue Oct 04, 2005 3:05 pm

I never thought I'd actually want to donate, my letter is being sent in now!
RECONstiPated
 
Joined: 17 Mar 2004

Postby derobert » Tue Oct 04, 2005 9:42 pm

xort wrote:And just how much memory do you think it takes to track lets say 250,000 torrents?
I doubt there is much of a technical issue with tracking that many torrents. Now, let's talk about seeding even 30,000 torrents. I'm pretty confident the disk arrays required to support random access to 2TiB at 30 MB/sec are confortably out of our price range.
Key 55EA59FE; fingerprint = E501 CEE3 E030 2D48 D449 274C FB3F 88C2 55EA 59FE
A mighty order of ages is born anew.              http://twitter.com/derobert
User avatar
derobert
Phantom of the .Org
 
Joined: 24 Oct 2001
Location: AOD Command Fortress

Postby Mroni » Wed Oct 05, 2005 5:05 am

I like the idea of booths at the cons. I think you would get a great response selling hats there too. I would easily plunk down 20 for a hat and make a donation too. I am very bad at mailing things out Ive had a hd waiting to get shipped for almost a month now. I want to help the org Ive been leeching since 2001. And I don't use a credit card so having a booth at Otakon or Katsucon would be great for me I would also donate time in watching it. I don't know how you would secure the money I certainly wouldn't rip off anybody maybe only the big name creators could run it.


Mr Oni
Purity is wackable!
"Don't trust me I'm over 30!"
User avatar
Mroni
 
Joined: 14 Aug 2001
Location: Heading for the 90s living in the 80s sitting in a back room waiting for the big boom

Postby bum » Wed Oct 05, 2005 6:46 am

Coderjoe wrote:
Coderjoe wrote:Hmm.. Within the month of august, Animix was downloaded 768 times, using a total of 557GB of bandwidth. There are 10 videos that, in August alone, used over 100GB apiece. Perhaps BitTorrent might be worthwhile for very popular vids. All but three of the top ten bandwidth users from August had over 1000 downloads.

Thank you, (someone), for providing the statistics.


hmm.

On the other hand... AniMix only used 1.93% of the total local download amount in August...



Considering how much bandwidth this place goes through each month (roughly 3TB) 1.93% is alot. What I believe would be a good idea is if torrents of the top 10% in 25 video blocks are put up. As the list changes (which it rarely does, esspecialy closer to the top) little signs can be placed where appropriat.
User avatar
bum
17747114553
 
Joined: 08 Nov 2003

Postby Coderjoe » Wed Oct 05, 2005 11:41 am

bum wrote:(roughly 3TB).


Try closer to 30TB. (in august 28,826,947,411,928 bytes)
User avatar
Coderjoe
 
Joined: 03 Mar 2001

Postby celibi87 » Wed Oct 05, 2005 9:17 pm

Coderjoe wrote:
bum wrote:(roughly 3TB).


Try closer to 30TB. (in august 28,826,947,411,928 bytes)

damn.....
User avatar
celibi87
 
Joined: 11 Oct 2004
Location: That one place

Postby xort » Wed Oct 05, 2005 10:48 pm

Coderjoe wrote:Do you know? Also, how many downloaders do you expect per torrent, on average?

I do know, however I wouldn't expect anyone to take my word. It would be more worth while to research an answer than to be given one that you would never honestly believe. For the mean average about 150 downloads evenly spread over 30 days.
Bittorrent really isn't wothwhile for the mass of videos being downloaded a few times a month. Especially when many people are likely to close the torrent once their download is complete. Additionally, who do you think would be seeding the torrents? More than likely the same download server that is storing the video and currently sends it out via http already.
Those that close without providing a full return should have their acounts limited, to upload only or download equal to upload. Management for anti leech control is highly effective. Members that can't upload should be able to donate money for download credits or an upgrade to an unlimited acount. Although even the tighest ISP could handle a few megs uploaded over a month. Or maybe your acount will have download bandwidth added ever 15 days or something for free just for being a member. The seeding would be handled in part by the current server and in part by people downloading. As their is a steady flow of downloads, their will always be someone downloading a file that would be able to upload to others.
Bittorrent might be worthwhile for videos with a large number of downloads per month, or for top bandwidth consumers in a month, but not for the peek on Phade's curve.

Think of it like this: if you already serve all that content each month, by changing the way it's downloaded from HTML transfer to BT, what is lost?

After talking this problem over, the solution offered was to convert the whole database into a single superset torrent, and provide single file access torrents (IE one video) for download. People would be able to grab single files and might in passing upload to others, (for sure if they actively choise to help) while the very popular files would mostly transfer themself.

The seeding of the whole database would be covered by the current http servers while the tracking might need to be offloaded onto another system or one of the freed transfer servers.

Try closer to 30TB

30TB in 30 days. I could offer a link to a tracker that passed 70TB in a day.
I doubt there is much of a technical issue with tracking that many torrents. Now, let's talk about seeding even 30,000 torrents. I'm pretty confident the disk arrays required to support random access to 2TiB at 30 MB/sec are confortably out of our price range.

And yet somehow this site currently runs... Further the most common vidoes would not require any access help.

However this is speculation, Phade having taken the effort to dig up a graph and post it, Combinded with his own bad BT experience will likely mean that AMV dot org will never have any BT support, ever. That's my own experience on human nature talking here so I might be wrong.

My parting suggestion is that someone finds what are the top 100 videos by bandwidth, and just go ahead and set up a tracker and make the torrents for them. Ask nicely and have a BT section added in the download options and take at least some of the load off.

If even a fraction of 1% of the people here are willing to offer money, I know that many more would offer bandwidth.
xort
 
Joined: 29 Jun 2001

Postby Kalium » Wed Oct 05, 2005 11:10 pm

Let me explain why BT is a bad idea for the Org.

It works like this. BitTorrent solves the problem of a number of users wanting a small number of files of significant size (200+ meg seems to be the sweet spot) simultaneously. To be really effective, you need hundreds at a time. Aside from the most popular of videos (ex: AMV Hell 3) just after release, the simultaneous demand just doesn't exist.

HTTP solves the problem of a relatively low number of users wanting a relatively high number of files.

"But wait!" you protest, "the Org has lots of users!"

"Ah," I retort "yes, but only a handful of users (at most!) are likely to be downloading a random video a random moment."

"As a result, the simultaneous demand neccessary for BitTorrent to be useful doesn't exist. This is, in fact, precisely the situation our old and dear friend HyperText Transport Protocol was created for!"

Need I continue?
User avatar
Kalium
Sir Bugsalot
 
Joined: 03 Oct 2003
Location: Plymouth, Michigan

Postby Phade » Wed Oct 05, 2005 11:20 pm

Hey,

It also seems that web site + BT = high on "nuke" list. Until that changes, I think we really can't raise our risk exposure even more than it is now.

Phade.
User avatar
Phade
Site Admin
 
Joined: 20 Oct 2000
Location: Little cabin in the woods...

Postby Coderjoe » Thu Oct 06, 2005 12:58 am

(sorry, long post)

xort, how about I post a nifty little table built from the data for august?
Code: Select all
+------+-------------+---------------+---------------+----------+
| dls  | video_count | bandwidth     | pct_bandwidth | pct_vids |
+------+-------------+---------------+---------------+----------+
|    0 |       28858 | 9514086774818 |         33.00 |    79.39 |
|   50 |        6232 | 8798511199207 |         30.52 |    17.15 |
|  100 |         779 | 2910870116770 |         10.10 |     2.14 |
|  150 |         244 | 1557879731843 |          5.40 |     0.67 |
|  200 |          78 |  752857051190 |          2.61 |     0.21 |
|  250 |          38 |  452363007786 |          1.57 |     0.10 |
|  300 |          25 |  343384593580 |          1.19 |     0.07 |
|  350 |          21 |  369657854308 |          1.28 |     0.06 |
|  400 |          16 |  446654281788 |          1.55 |     0.04 |
|  450 |          11 |  228720772244 |          0.79 |     0.03 |
|  500 |           4 |  117028038332 |          0.41 |     0.01 |
|  550 |           7 |  192141559032 |          0.67 |     0.02 |
|  600 |           5 |  163223949786 |          0.57 |     0.01 |
|  650 |           2 |   71273460700 |          0.25 |     0.01 |
|  700 |           5 |  256076784162 |          0.89 |     0.01 |
|  750 |           2 |  572297822208 |          1.99 |     0.01 |
|  800 |           1 |   59119312896 |          0.21 |     0.00 |
|  850 |           2 |   90148339320 |          0.31 |     0.01 |
|  950 |           3 |  127361215000 |          0.44 |     0.01 |
| 1000 |           1 |   65488539648 |          0.23 |     0.00 |
| 1050 |           3 |  225380063412 |          0.78 |     0.01 |
| 1250 |           1 |  104206521200 |          0.36 |     0.00 |
| 1300 |           1 |   88189874392 |          0.31 |     0.00 |
| 1350 |           1 |   83647723520 |          0.29 |     0.00 |
| 1400 |           1 |  126969472540 |          0.44 |     0.00 |
| 1500 |           1 |   81662140416 |          0.28 |     0.00 |
| 1550 |           1 |   55376169526 |          0.19 |     0.00 |
| 1750 |           2 |  307129124864 |          1.07 |     0.01 |
| 1900 |           1 |  168448131072 |          0.58 |     0.00 |
| 2800 |           1 |  250930956288 |          0.87 |     0.00 |
| 3100 |           1 |  245862830080 |          0.85 |     0.00 |
+------+-------------+---------------+---------------+----------+


:arrow: dls - low end of number of downloads for the group. 0 means 0-49, 50 means 50-99, and so on.
:arrow: video_count - number of videos in this group.
bandwidth - amount of bandwidth consumed by this group.
:arrow: pct_bandwidth - the percentage of the overall bandwidth for the month used by this group.
:arrow: pct_vids - the percentage of the unique videos downloaded in the month that fall within this group.

if you look at this data, you can see:

:arrow: 33% of the bandwidth was used by videos that were downloaded 0-49 times in the entire month. also, 79.39% of the videos downloaded fall within this group.
:arrow: another 30.52% of the bandwidth was used by videos that were downloaded 50-99 times in the entire month. there were 17.15% of the overall number of videos in this group.
:arrow: adding the previous two points together shows that 63.52% of the bandwidth was used by videos downloaded 0-99 times, and that 96.53% of the unique videos downloaded were downloaded 0-99 times.

bittorrent really doesn't become worthwhile until you have at least (as in the abolute minimum of) 2 people downloading a file at the same time, which isn't likely to happen until you get up to 400 downloads or so (guestimating). the overhead incurred by putting the lesser-downloaded videos out via bittorrent is simply not worth it, as it puts more wear and tear on the raid arrays, makes the tracker memory usage go up, increases the number of HTTP requests that must be handled, and wastes bandwidth on the peer-to-peer traffic overhead.

you may also see the following information, if you add the pieces of information together from above...
Code: Select all
videos downloaded more than 200 times:
+-----------+----------+---------------+---------------+
| vid_count | PCT_VIDS | bandwidth     | pct_bandwidth |
+-----------+----------+---------------+---------------+
|       185 |     0.51 | 5618453425972 |         19.49 |
+-----------+----------+---------------+---------------+

videos downloaded 200 times or less:
+-----------+----------+----------------+---------------+
| vid_count | PCT_VIDS | bandwidth      | pct_bandwidth |
+-----------+----------+----------------+---------------+
|     36163 |    99.49 | 23208493985956 |         80.51 |
+-----------+----------+----------------+---------------+


These tables show that there are 185 videos, making up 0.51% of the unique videos downloaded, that used 19.49% of the bandwidth. Torrenting these might be worthwhile. But then there is also the risk of someone undesireable taking notice and spoiling the party for everyone.
User avatar
Coderjoe
 
Joined: 03 Mar 2001

Postby celibi87 » Thu Oct 06, 2005 11:48 am

Phade wrote:Hey,

It also seems that web site + BT = high on "nuke" list. Until that changes, I think we really can't raise our risk exposure even more than it is now.

Phade.

i dont know if our exposure is already big or small. all i know is that when i go to a con and check out the amv contest they talk about this site in paticular. Are we exposed? I do believe we are. I am against putting bt on this site cause i love how it is right now.
User avatar
celibi87
 
Joined: 11 Oct 2004
Location: That one place

Postby 7sigma » Thu Oct 06, 2005 10:08 pm

celibi87 wrote:I am against putting bt on this site cause i love how it is right now.


Well, something must change if we want the site to keep going. BitTorrent could be useful in lightening the load of the most popular vids, but that's only half the problem (and it creates a lot of new ones, like drawing big guns in our direction).

So, I guess we could think about advertising again. The fact that we have a lot of people with some cash (we have at least a PC and an internet connection) that all share the same hobby ought to be worth something. Again, the somewhat gray area we live in might really hinder this idea, but it should be considered anyway.

The perfect solution, however, would be simply increasing donations, since the cost-per-user of the site seems to be really low, as Phade said. I don't have any bloody idea on how to do this without hurting users in a way or another, though.
When the gods wish to punish us, they answer our prayers - Oscar Wilde
User avatar
7sigma
 
Joined: 20 Dec 2003
Location: Brazil

Postby SQ » Fri Oct 07, 2005 6:26 pm

Well the envelope art contest is a step in the right direction.
The only problem is that the donations that come with those envelopes will probably be used to fund the prize for the contest instead of the site. >_<
-SQ™
Changes are good. Wait until you've seen mine.
Latest video - Family (Oct 3, 2013)
User avatar
SQ
Doesn't have a title
 
Joined: 08 Nov 2002
Status: Forever Navy

Postby Phade » Fri Oct 07, 2005 7:24 pm

Hey SQ,

That depends entirely on the amount of donations and if the prizes themselves are donated. ;-)

Phade.
User avatar
Phade
Site Admin
 
Joined: 20 Oct 2000
Location: Little cabin in the woods...

PreviousNext

Return to Site Announcements

Who is online

Users browsing this forum: No registered users and 0 guests