extreme sync effects

[Clodo]

Hi to all,i'm hope this is the correct forum for that.
I'm thinking about an "extreme" syncronization between images/video and music.
I'm trying to explain with a video:

(click herefor download an XVID/AVI, it's an external link, 20MB).
(artist: Stratovarius, song: Black Diamond).

To create this, i'm simply capture a midi-player, and after replace the midi output with the real output (the odd trick is found a midi like this, already syncronized; in this i made some adjustement..).
Now, i'm thinking about using some portion of this video as mask-layer for other footage, or link it with properties of some effects, or link each note to a specific frame, or ... ?
I can post here the loseless version, if anyone want to try some experiments.
I'm a code developer, and my base idea is to create some script that automatically pick some random footage and apply effects, to understand if can be interesting before loosing too much time

My question is simple: can be cool or i'm crazy? bye!

[requiett]

There are plugins for After Effects which can already do this based on waveforms.

[-GfN-]

I wrote three interrelated expressions within after effects which make the whole thing work on a much more manual basis than the plugins req mentoned. other than that, there are possibilities to use the method for natural animations (such as fog moving to noise in the music) and much more. it's just that you have to have a good sense for what it can be used for.

[Clodo]

There are plugins for After Effects which can already do this based on waveforms.

I already do some effects with a plugin in AE based of waveforms in my first AMV.
But it's very difficult to create a "clean" effects; here we have each single notes, that can be used for example to map a scale of notes to frames of a sequence (for example by mapping a sequence of a character that sound an instrument to the same notes).

Effects based on waveforms, or scene changes based on threshold of that, doesn't "feel good" in my experience (too much "artificial"), for this i'm here to ask a feedback, i'm not sure that this technique can be "give a nice sensation" for an AMV

Thanks!

[BasharOfTheAges]

Aside form fractals, what mathematically generated art does "feel good" though?