*tosses in 2 cents*
As much as most people insist that flashy, effect-ridden videos are not by default
better than simpler ones, just the fact that the editor spent (probably) a good deal of time dealing with potentially complicated extras tends to make an impact on the viewer. I think its worthwhile to remember that [blank] anime combined with [blank] song can be done a number of ways and come out really good. All other things being equal (sync, concept, capture quality etc.), scoring the effects-heavy AMV higher than a simpler one doesn't make sense unless you're really impressed by the extras - in which case the objectiveness of the scoring system wobbles a little and becomes more a reflection of the viewer's personal preference.
I wonder how many people would be equally impressed by well done, simple effects and well done, complicated effects. I can think of at least one very good AMV (VHD: Bloodlust/Moonlight Shadow) that used simple effects SO well that I never even noticed them (aka their presence did not distract from the AMV), but I don't think it would be scored very high today because people would watch it and think, "The editor could have used [blank] effect or done [blank] to make it even cooler, but didn't, so that's disappointing."

On that note, being part of the MTV generation isn't so "cutting edge," anymore, and the age of the viewer has a LOT to do with how high an AMV is scored. Wouldn't it be interesting to have stats on the age of AMV watchers around here recorded when stars/opinions are given? It would be another way to sort and rank AMVs that might better reflect an their "true" worth. Unfair, discriminatory? Well, not really. One thousand 10-15 year olds giving an AMV 2 stars and 300 16-24 year olds giving it 4 stars is kind of significant, IMO, but there's no way, as it stands now, to find out this kind of information.
It's been said before: some kind of measure should be taken to make sure the immature, easily-impressed, less-wordly voters out there don't swamp the ratings. Though, it may be too late, for some AMVs. This also would help those browsing for good AMVs to watch; knowing that an AMV received really low scores from preteens, but really high scores from teens/adults, might steer a kid (sorry, that's what you are!) away from that AMV, but direct an older individual towards it. This could help good AMVs out there being ignored by the younger crowd (particularly if an older/more obscure anime is used) receive more exposure, better ratings etc.