Avisynth MT (for multiple cores, and its fucking awesome!)

This forum is for questions and discussion of all the aspects of handling your footage. If you have questions about capturing/ripping footage, AviSynth, or compression/encoding/converting, look here.

Postby Corran » Sun Sep 09, 2007 1:27 am

BasharOfTheAges wrote:It seems just for simple directshowsource transcoding with nothing else i don't get any speed increase (24:15 [mm:ss] of footage 1024x68 h264 in MKV to HuffYUV in AVI took around 11.5 minutes both with and without MT).


Multithreading the source filter probably isn't going to help if decoding a video where keyframes are spread out such as H.264 as you need to rebuild the frames sequentially.

Leak from the Doom9 forums wrote:Well, I suppose it could open the same file several times in different threads and fetch several frames at once, but I'm honestly not sure if that's going to work too well, at least when you have a codec where frames depend on each other.

It could work if you start a thread at every keyframe (although I'm not sure how to get that information from a source filter) or with keyframe-only formats like HuffYUV or MJPEG, but probably not too well with MPEG4 ASP or H.264 with several hundred frames of max keyframe distance...
User avatar
Corran
 
Joined: 14 Oct 2002

Postby Qyot27 » Sun Sep 09, 2007 4:59 am

Corran wrote:
BasharOfTheAges wrote:It seems just for simple directshowsource transcoding with nothing else i don't get any speed increase (24:15 [mm:ss] of footage 1024x68 h264 in MKV to HuffYUV in AVI took around 11.5 minutes both with and without MT).


Multithreading the source filter probably isn't going to help if decoding a video where keyframes are spread out such as H.264 as you need to rebuild the frames sequentially.

Leak from the Doom9 forums wrote:Well, I suppose it could open the same file several times in different threads and fetch several frames at once, but I'm honestly not sure if that's going to work too well, at least when you have a codec where frames depend on each other.

It could work if you start a thread at every keyframe (although I'm not sure how to get that information from a source filter) or with keyframe-only formats like HuffYUV or MJPEG, but probably not too well with MPEG4 ASP or H.264 with several hundred frames of max keyframe distance...

There isn't any way around that though? Like DGAVCIndex or FFmpegSource (to be honest FFmpegSource wins that contest for right now since it actually supports reading from the containers and isn't limited to just H.264), for example - both write index files which get read in place of the straight video, and depend on libavcodec to do their decoding. I could see how DirectShowSource would be deficient, but with ones that write indexes it'd be vastly simpler to implement multithreaded decoding, as it'd be more up to the development of libavcodec to achieve that, or so I would think.

Whether AviSynth MT supports either one is something someone else will have to test, I don't have a computer that uses multithreading.
User avatar
Qyot27
Surreptitious fluffy bunny
 
Joined: 30 Aug 2002
Location: St. Pete, FL
Status: Creepin' between the bullfrogs

Postby Corran » Sun Sep 09, 2007 6:32 am

Indexing allows for frame accurate seeking but the video still needs to be rendered in the same fashion; i.e. starting at a keyframe and applying the changes til the current requested frame is rendered.

Out of curiousity I ran the following two scripts and encoded them both to MJPEG. When encoding the one using MTSource, I noticed the frame rendering rate drop to 0fps several times. Thread lock I'm guessing... Even so, I was still shocked that it took so long to render compared to the single thread. It is honestly my first time personally using ffmpegsource and mtsource so maybe the default settings shouldn't have been used in the test?

Code: Select all
MTSource("""FFMPEGSource("F:\Video\AMVs\[Koop] Skittles (1280x720).mp4")""")
converttorgb32()

Time to render: 12:49
Code: Select all
FFMPEGSource("F:\Video\AMVs\[Koop] Skittles (1280x720).mp4")
converttorgb32()

Time to render: 3:23
User avatar
Corran
 
Joined: 14 Oct 2002

Postby Pwolf » Sun Sep 09, 2007 5:00 pm

Corran wrote:Indexing allows for frame accurate seeking but the video still needs to be rendered in the same fashion; i.e. starting at a keyframe and applying the changes til the current requested frame is rendered.

Out of curiousity I ran the following two scripts and encoded them both to MJPEG. When encoding the one using MTSource, I noticed the frame rendering rate drop to 0fps several times. Thread lock I'm guessing... Even so, I was still shocked that it took so long to render compared to the single thread. It is honestly my first time personally using ffmpegsource and mtsource so maybe the default settings shouldn't have been used in the test?

Code: Select all
MTSource("""FFMPEGSource("F:\Video\AMVs\[Koop] Skittles (1280x720).mp4")""")
converttorgb32()

Time to render: 12:49
Code: Select all
FFMPEGSource("F:\Video\AMVs\[Koop] Skittles (1280x720).mp4")
converttorgb32()

Time to render: 3:23


try changing the mode to 3 and see what it does.

Pwolf
ImageImage
ImageImage
Like the AMV .Org App? Think about donating to help me make it better.
User avatar
Pwolf
Friendly Neighborhood Pwaffle
 
Joined: 03 May 2001
Location: Some where in California, I forgot :\

Postby Corran » Mon Sep 10, 2007 6:57 am

Code: Select all
SetMTMode(3)
MTSource("""FFMPEGSource("F:\Video\AMVs\[Koop] Skittles (1280x720).mp4")""")
converttorgb32()


Render time ~40 minutes x_x

The number is a little tainted as at the 20 minute mark I started watching random stuff on you-tube thus pulling some cpu cycles away from the encode, but even so...
User avatar
Corran
 
Joined: 14 Oct 2002

Postby Pwolf » Mon Sep 10, 2007 1:15 pm

do:

setmtmode(3,2) (2 being the number of cores so if you have 4, use 4)


Pwolf
ImageImage
ImageImage
Like the AMV .Org App? Think about donating to help me make it better.
User avatar
Pwolf
Friendly Neighborhood Pwaffle
 
Joined: 03 May 2001
Location: Some where in California, I forgot :\

Postby Corran » Mon Sep 10, 2007 4:26 pm

Pwolf wrote:do:

setmtmode(3,2) (2 being the number of cores so if you have 4, use 4)


Pwolf


The default is 0. 0 sets it to use all available cores.
User avatar
Corran
 
Joined: 14 Oct 2002

Postby Pwolf » Mon Sep 10, 2007 5:12 pm

Corran wrote:
Pwolf wrote:do:

setmtmode(3,2) (2 being the number of cores so if you have 4, use 4)


Pwolf


The default is 0. 0 sets it to use all available cores.


i must have missed that part >.>

Pwolf
ImageImage
ImageImage
Like the AMV .Org App? Think about donating to help me make it better.
User avatar
Pwolf
Friendly Neighborhood Pwaffle
 
Joined: 03 May 2001
Location: Some where in California, I forgot :\

Previous

Return to Footage Help

Who is online

Users browsing this forum: No registered users and 1 guest