Not necessarily true. My first two videos were done within one week as well, but judging from the comments I got they're decent at least. If you don't use any digital effects and are working with Windows Movie Maker you can produce quite a few videos very quickly because WMM is so easy to use. Sure the only effect you can do is cross-fading with overlays but that can work pretty well at times.
I'm taking a little bit longer on my latest video however, since now I'm using Adobe Premiere for it and because it's a song with a lot of beats. I think if you really know what you're doing and know exactly where each note, beat, and word falls it helps a lot to consolidate your time and organize your movie layout. That's why the first step I do is to create a written layout that lists what to watch for specifically like:
<beginning guitar riff> 00:00:70 - 00:03:40
<line 1, something about being funky, match monkey waltzing around scene> 00:04:20 - 00:09:30
<orchestra hit> 00:10:20 - 00:11:30
Anyway, basically what I'm saying is don't worry too much about how much time you spend as long as you put effort into making it a quality video people will be able to tell. And take pride in your creations no matter how badly others may think of them because completing the creation is an accomplishment in itself.