I'll go at this from the perspective of changing the way the fanime contest was last year...
1) Don't have a semifinals and a finals. Just have either Two showings of the same set of videos and do ballot voting, or have only one showing if you're gonna do audience vote by noise.
2) Actually HAVE a staff for this. Have a person to run tech, a person to run MC, and a person behind the scenes. Both years I've been to fanime the creators have wound up running things, much to the *ahem* disapproval of certain others. Last year Brad ran tapes while I queued them and John Bahker MC'd. This year John MC'd both showings, and I ran tapes on the second one (we queued them up ahead of time). THIS SHOULD NOT HAPPEN.
3) Set the deadline for when entries must be received
to at least a month before the competition. Judge the videos shortly thereafter with a panel of judges. I'd suggest that half of them be creators, and half be viewers, but please make them educated viewers. Filter it down to a certain number of videos per category.
4) Speaking of categories, have them listed before hand, and CLEARLY defined. Some cons think "Action" videos must be only action, no drama, otherwise they belong in the drama category. Other cons think otherwise. However, do not let people submit a category they want their video in. This should be decided by the judges to avoid videos being miscategorized. The category a video goes in should be decided by a general vote of the judges.
5) Just a note about judging here - for whatever Drama category you have, I would highly suggest trying to ignore entirely whether or not you've seen a series or not. One of the biggest problems judging drama videos is that often times you cannot see the drama unless you've seen the series. This gives an unfair advantage to the Fushigi Yuugi/Cowboy Bebop/Trigun/etc Drama videos because most people have seen some of those, while giving the Gunbuster/Magical Princess Minky Momo/Pretty Combat Communist Rika-chan*
videos a distinct disadvantage. If you view the videos from the perspective of never having seen a series before this helps to even the playing field.
6) On the technical side, DON'T accept Digital submissions unless you REALLY know what you're doing. This bit AX in the ass this year, and is continually biting other contests in the ass. If you are not properly prepared to handle and display digital submissions, you wind up screwing yourself and the creators who submitted this way. Audio dropouts, bad video, not-powerful-enough computers, all these things can really screw up a contest using digital submissions. I would definately recommend ONLY accepting Authored DVD, VCD, VHS, SVHS, DV or MiniDV, and possibly SVCD if you have a player that can handle it. These are all things which can be played independantly of a computer, and can be dubbed straight onto another tape when you...
7) Compile a contest tape. Since I know it'll come down to some guy with a VCR hitting stop and play at Fanime, my suggestion is to find the best videos and then copy them all onto a nice S-Deck with flying head erase so you can make one seamless tape with all the contest entries in the order they will be viewed. If you go over 2 hours of videos then have 2 tapes. But this is a FAR better prospect than this or last year where we had about 30 tapes we had to deal with and queue up ahead of time, and then shuffle around in the dark trying to find them.
8) Don't be afraid to disqualify videos for technical problems. Don't let a video that looks like crap into the finals. AX disqualified many entries this year based on video quality, and I think that was a good move since you're going to be blowing up these videos to a large screen via a projector in a dark room, and any major video problems (macroblocks, MPEG artifacts, interlacing issues) will be amplified by orders of magnitude. Putting those kinds of videos up on the big screen makes the contest look far less professional.