AMV Generated by Artifical Intelligence

[vkamv]

I’ve always been fascinated with AI, and the recent rise of deep learning, and wondered if one day an AI could create an AMV, which is why I conducted this experiment. I used a random number generator to create this AMV. The following things were randomly selected by the generator:

• Random start time. Range between 4:00 and 17:00
• Random duration of clip. Between 1-3 seconds.
• Random episode number of which the clip would be taken. 1-13 (13 being PMMM Rebellion)

I compiled the above results into an excel file which I then used to create this AMV. The only human element was cutting the clip from the episode according to data in excel and pasting the clip into Adobe Premiere timeline. After all the clips were assembled I selected this song to fit with it. I also added a simple fade in at the start and fade out at the end. The only other human involvement was if a scene had a random cut between scenes at the beginning that made the look glitched I didn't started the clip one the new scene.

I know it’s not true AI but more of random generated AMV, but I found the experiment and results interesting. The AMV turned out pretty good in my opinion, especially with the song.

I'm not a programmer so I couldn't write a program to actually cut the clips out for me. But if such a program were made than truly random AMVs with no human involvement could be created.

Any thoughts on this experiments? Is this AMV an abomination?

[snapxynith]

Four years ago, I implemented a random amv generator that uses either premiere or avisynth. It uses bpm to determine clip length, can pan and zoom, lists the clip frame range, and the number of clip.

Here is a limited selection of his work.
"Toby AMV - YouTube" https://youtube.com/user/tob39amv

Ai on video would be massively heavy, so I never intended to implement it..

Based on results, ai is not required, just brute force and a person to filter vauge winners.

[vkamv]

Four years ago, I implemented a random amv generator that uses either premiere or avisynth. It uses bpm to determine clip length, can pan and zoom, lists the clip frame range, and the number of clip.

Here is a limited selection of his work.
"Toby AMV - YouTube" https://youtube.com/user/tob39amv

Ai on video would be massively heavy, so I never intended to implement it..

Based on results, ai is not required, just brute force and a person to filter vauge winners.

Interesting. I think the process can be useful for brainstorming and seeing clips and scenes in a different light when they are randomly paired together. It also caused me to realize that much of AMV creating can be random. I find myself a lot scrubbing through episodes and trying out random scenes to see if they work together or not, much like this random selection. Some of my favorites sequences in my AMVs were put together by pure chance it seemed without in planning before hand.

It would also be interesting if a program helped you find scenes that had similar styles, whether similar in colors, or in orientation of shapes or character motions.

[lloyd9988]

Four years ago, I implemented a random amv generator that uses either premiere or avisynth. It uses bpm to determine clip length, can pan and zoom, lists the clip frame range, and the number of clip.

Here is a limited selection of his work.
"Toby AMV - YouTube" https://youtube.com/user/tob39amv

Ai on video would be massively heavy, so I never intended to implement it..

Based on results, ai is not required, just brute force and a person to filter vauge winners.

Interesting. I think the process can be useful for brainstorming and seeing clips and scenes in a different light when they are randomly paired together. It also caused me to realize that much of AMV creating can be random. I find myself a lot scrubbing through episodes and trying out random scenes to see if they work together or not, much like this random selection. Some of my favorites sequences in my AMVs were put together by pure chance it seemed without in planning before hand.

It would also be interesting if a program helped you find scenes that had similar styles, whether similar in colors, or in orientation of shapes or character motions.

I've thought it best to break up these problems into simpler terms for different AI bots to work through.

Like, one AI bot does the scrubbing, one AI bot does pan and zooming, one AI bot does masking (Would be useful... Just saying), and one AI bot acts as a master-mind for it all.

I think it best to start with one AI bot and train just the one AI bot. I'd love to try this out for myself. This seems like an interesting research subject to work through.

For the scrubbing AI bot, maybe instead of random scenes, you could input phrases of words and the AI bot best selects the scenes that it thinks fits the description you input. But... That Method would take hours... Perhaps input a number and the AI bot scrubs through random scenes.

Yeah, we would need to define different parts for an AI personally because there are many tasks that go into making an AMV and I think assigning different bots to do one specific task and then creating a 'Master-Mind' AI bot would be ideal imo.

Any thoughts?

P.S. I remember watching a 3-part youtube video on AI and he states that AI is more 'Artificial' than 'Intelligent'. I can link the youtube video by 3Blue1Brown.

[seasons]

A long time ago (between 2008 and 2011?) there was a thread about some website where users would upload video clips and pictures and the site itself would generate some kind of music video (glorified slide show, basically) out of it. Some users were calling it the death of AMVs. Obviously, nothing ever really came of this. But I've been looking for that thread forever.

Kind of afraid to watch the OP's video, what if it's good (or even "good")?

[CrackTheSky]

That video was pretty much exactly what I would expect a video like this to look like. Honestly it's not terrible, but it's really random and it's not something I would, y'know, recommend, exactly. That said, it's better than a lot of videos I've seen where the editor is actually trying, so...

[vkamv]

That video was pretty much exactly what I would expect a video like this to look like. Honestly it's not terrible, but it's really random and it's not something I would, y'know, recommend, exactly. That said, it's better than a lot of videos I've seen where the editor is actually trying, so...

I think a lot has to do with the anime itself and the music you select. Electronic music seams to work for random scenes. I redid the process with Serial Experiments Lain and a different song and liked the results even more. The anime itself is made up of very trippy scenes which match the music nicely.

I hypothesize that when a lot of people start making their first AMV it's a pretty random process for them as well. I know mine was. So I don't think it's necessarily surprising that an AMV created randomly can look better than some posted out there b y new editors.

[Lylaaz]

In the end i think the question remains - can AI generated AMV tell a story? Probably not. At least for a long time.

Although the song was nice and thought provoking.. In the end it just feels random

[Dragonrider1227]

What a fascinating concept