Avisynth MT (for multiple cores, and its fucking awesome!)

This forum is for questions and discussion of all the aspects of handling your footage. If you have questions about capturing/ripping footage, AviSynth, or compression/encoding/converting, look here.

Postby Ayanefan » Sun Jul 29, 2007 9:32 am

I managed to get 7-8fps with this script! It used to be 3fps.
An 800mb +/- 20mb was done in 2 minutes!!!

SetMTMode(2)
MPEG2Source("C:\AMV_Project\FireFight\VOBs\K_S_Vol1\K_S_Vol1.d2v", cpu=6)
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\FluxSmooth.dll")
TFM().TDecimate(mode=1)
MT("""Deen("a3d",4,10,12).TFM().TDecimate(mode=1).vmToon(strength=164).Crop(8,0,-8,-0).UnDot().Tweak(cont=1.1,sat=1.05,bright=0, hue=.2).TemporalSoften(4,4,8,15,2)""")
Image
Image
User avatar
Ayanefan
 
Joined: 24 Oct 2005

Postby Ayanefan » Sun Jul 29, 2007 9:37 am

ayanefan wrote:I managed to get 7-8fps with this script! It used to be 3fps.
An 800mb +/- 20mb was done in 2 minutes!!!

SetMTMode(2)
MPEG2Source("C:\AMV_Project\FireFight\VOBs\K_S_Vol1\K_S_Vol1.d2v", cpu=6)
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\FluxSmooth.dll")
TFM().TDecimate(mode=1)
MT("""Deen("a3d",4,10,12).TFM().TDecimate(mode=1).vmToon(strength=164).Crop(8,0,-8,-0).UnDot().Tweak(cont=1.1,sat=1.05,bright=0, hue=.2).TemporalSoften(4,4,8,15,2)""")

Woops! Made a mistake with the TFM(), I've got it in twice.

There, 946.51mb of uncompressed output took- 2:37.
At approx 7 or 8 fps.
Image
Image
User avatar
Ayanefan
 
Joined: 24 Oct 2005

Postby Ayanefan » Sun Jul 29, 2007 9:40 am

This test, without MP()

MPEG2Source("C:\AMV_Project\FireFight\VOBs\K_S_Vol1\K_S_Vol1.d2v", cpu=6)
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\FluxSmooth.dll")
Deen("a3d",4,10,12).TFM().TDecimate(mode=1).vmToon(strength=164).Crop(8,0,-8,-0).UnDot().Tweak(cont=1.1,sat=1.05,bright=0, hue=.2).TemporalSoften(4,4,8,15,2)

3-4fps , 4:52.
946.51mb

MT is AWSOME!
Image
Image
User avatar
Ayanefan
 
Joined: 24 Oct 2005

Postby Ayanefan » Sun Jul 29, 2007 10:54 am

I've tried different SetMTmodes 1-6, they all took the filters but didn't make much of a speed difference in the processing. Only Mode 1 made a slight difference, going up to 9 fps once in a while, but only made about 5 seconds difference.
I'd stick to mode 2 because it goes frame by frame rather than 1/2 frames.
Image
Image
User avatar
Ayanefan
 
Joined: 24 Oct 2005

Postby BasharOfTheAges » Sun Jul 29, 2007 2:59 pm

Well, there are some things that mode 2 an do that mode 1 can't, so =/
Another Anime Convention AMV Contest Coordinator 2008-2014 & Head of the AAC Fan-works Theater - follow us on Twitter: https://twitter.com/#!/AACFanTheater
:sorcerer: :sorcerer: |RD: "Oh, Action!" (side-by-side) | |
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Postby Ayanefan » Sun Jul 29, 2007 4:29 pm

BasharOfTheAges wrote:Well, there are some things that mode 2 an do that mode 1 can't, so =/


Was that an attempt to make me look silly?
Image
Image
User avatar
Ayanefan
 
Joined: 24 Oct 2005

Postby BasharOfTheAges » Sun Jul 29, 2007 6:10 pm

ayanefan wrote:
BasharOfTheAges wrote:Well, there are some things that mode 2 an do that mode 1 can't, so =/


Was that an attempt to make me look silly?


It was an attempt at explaining why it was still useful despite being slower. The wiki page also suggests more memory use.

I need to actually try a few scripts out myself - the last baseline i have to compare to is my old laptop with < 1GB of ram and an old P4 for processing... I'm running a core2duo with 4GB of RAM now so the degree of awesome would jump with just the old version of avisynth at my disposal. As I mentioned before though, getting clips into batch mode will still be a several hour long process. :lol:
Another Anime Convention AMV Contest Coordinator 2008-2014 & Head of the AAC Fan-works Theater - follow us on Twitter: https://twitter.com/#!/AACFanTheater
:sorcerer: :sorcerer: |RD: "Oh, Action!" (side-by-side) | |
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Postby Ayanefan » Sun Jul 29, 2007 6:52 pm

BasharOfTheAges wrote:I'm running a core2duo with 4GB of RAM now so the degree of awesome would jump with just the old version of avisynth at my disposal.


You make me sick! All I have is an AMD X2 6000 w/ 1gb ram. :(
Image
Image
User avatar
Ayanefan
 
Joined: 24 Oct 2005

Postby Ayanefan » Sun Jul 29, 2007 7:19 pm

ayanefan wrote:
BasharOfTheAges wrote:I'm running a core2duo with 4GB of RAM now so the degree of awesome would jump with just the old version of avisynth at my disposal.


You make me sick! All I have is an AMD X2 6000 w/ 1gb ram. :(


But I do have another wonder about this. It's actually a MultiThread version of AviSynth, right? Now, MT has been out before dual core so could there be a Multi-Core version of it too in the future?? It might make it go totally to the moon awesome!
Image
Image
User avatar
Ayanefan
 
Joined: 24 Oct 2005

Postby BasharOfTheAges » Tue Jul 31, 2007 4:37 pm

ayanefan wrote:But I do have another wonder about this. It's actually a MultiThread version of AviSynth, right? Now, MT has been out before dual core so could there be a Multi-Core version of it too in the future?? It might make it go totally to the moon awesome!


This is for multiple cores. The fact that hyper threaded pre-multicore processors also can make use of it is just an added bonus.

I just realized this only works with mpeg2source(), not any other modes (right?) The only thing i'm working on atm is a h264 stream in MKV so i'm stuck with directshowsource. Guess the testing will have to wait until i rip some other discs.
Another Anime Convention AMV Contest Coordinator 2008-2014 & Head of the AAC Fan-works Theater - follow us on Twitter: https://twitter.com/#!/AACFanTheater
:sorcerer: :sorcerer: |RD: "Oh, Action!" (side-by-side) | |
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Postby Kariudo » Wed Aug 01, 2007 7:04 pm

from the readme (typos and all)
Code: Select all
  MTsource

MTsource(string filter,int delta,int threads,int max_fetch)

All parameters are named. Function parameters:

filter string = No default
source filter to run multithreaded. Currently only internal and external source filters are supported (like DirectShowSource, Avisource, MPEG2Source) . You can use an avs defined filter or a non-source filter but it might crash or produce frame corruption.

delta int = 1
this is how many frames there are between each frame request so if you are only going to read every second frame set it to 2 or if you are reading the frames backwards set it to -1. More complex frame access pattern like SelectEvery(10,3,6,7) are not supported (but might work anyway as the requested frames are in the cache, there will just be some waisted memory from non requested frame in the cache)

threads int = 2
number of threads to run. Set this to the number of threads your computer is able to run concurrently.

max_fetch int = 30
This is the maximum number of frames ahead of the currently requested frame that MTsource will fetch. Setting it to low will leaving the threads idle for most of the time and setting it to high will waste to much memory.


I've done some testing on individual filters (almost all the ones I have, all the ones I use regularly)
download the results

the filters I didn't use the default settings for (and the values I used)
Code: Select all
Mpeg2Source("D:\Goddess\disk1.d2v",cpu=4)
TFM(d2v="D:\Goddess\disk1.d2v",order=-1,mode=5,PP=7,field=-1,slow=2)
vaguedenoiser(threshold=2,method=1,nsteps=6,Wiener=true,chromaT=2.0)
removegrain(mode=5)
deen("a3d",3,4,4)
limitedsharpenfaster(edgemode=1,wide=true,strength=250,smode=1)
vmtoon(strength=25)
crop(120,0,-120,0)


I didn't run tests for setMTmode(2,0) on all of them because it looked like the same thing over and over (slight to no performance loss for most)

resizers like lanczos4resize don't work even if you don't change the height or width. This applies for MT.dll version 0.1-0.7 (maybe later versions as well)

fastLineDarken doesn't work on my desktop. I always get a greenscale image after I run the filter (like greyscale/black & white but green)
I've tried getting masktools 1.5.1 again and again(instead of using 1.5.8, which says in its properties that it's really 1.5.1) and then it'l give me an error saying that there is no such function as expand()

I realized after I had done all the non-MT testing that I had lagarith set to multithreaded mode. This could have skewed some of the values for the non-MT tests.

generally speaking, when the non-MT outperformed or performed on par with the MT() values the cpu utilization was at 100% for the non-MT test.
ImageImage
Image
User avatar
Kariudo
Twilight prince
 
Joined: 15 Jul 2005
Location: Los taquitos unidos
Status: 1924 bots banned and counting!

Postby BasharOfTheAges » Wed Aug 01, 2007 7:45 pm

wait... so directshowsource works? O.o Hell yea i 'm back in the testing. I'll probably do that right now in fact. I just hope the color banding i'm seeing in VdubMod is a result of poor playback and not how the rendered version will look - because my source seems beautiful normally but ugly as all hell when i open it up in vdubmod.
Another Anime Convention AMV Contest Coordinator 2008-2014 & Head of the AAC Fan-works Theater - follow us on Twitter: https://twitter.com/#!/AACFanTheater
:sorcerer: :sorcerer: |RD: "Oh, Action!" (side-by-side) | |
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Postby Scintilla » Wed Aug 01, 2007 7:49 pm

BasharOfTheAges wrote:I just hope the color banding i'm seeing in VdubMod is a result of poor playback and not how the rendered version will look - because my source seems beautiful normally but ugly as all hell when i open it up in vdubmod.

Usual solution for this problem: Options --> Preferences --> Main tab, then set "Output color depth" to "Use output setting".

For some reason, VDubMod defaults to only 16-bit output color depth, which is insufficient for most any video you find these days.
ImageImage
:pizza: :pizza: Image :pizza: :pizza:
User avatar
Scintilla
(for EXTREME)
 
Joined: 31 Mar 2003
Location: New Jersey
Status: Quo

Postby BasharOfTheAges » Wed Aug 01, 2007 8:55 pm

Scintilla wrote:
BasharOfTheAges wrote:I just hope the color banding i'm seeing in VdubMod is a result of poor playback and not how the rendered version will look - because my source seems beautiful normally but ugly as all hell when i open it up in vdubmod.

Usual solution for this problem: Options --> Preferences --> Main tab, then set "Output color depth" to "Use output setting".

For some reason, VDubMod defaults to only 16-bit output color depth, which is insufficient for most any video you find these days.


Thanks, that's Good to know. (I still need to do some filtering though, the output isn't as great as i would have hoped.

It seems just for simple directshowsource transcoding with nothing else i don't get any speed increase (24:15 [mm:ss] of footage 1024x68 h264 in MKV to HuffYUV in AVI took around 11.5 minutes both with and without MT).
Another Anime Convention AMV Contest Coordinator 2008-2014 & Head of the AAC Fan-works Theater - follow us on Twitter: https://twitter.com/#!/AACFanTheater
:sorcerer: :sorcerer: |RD: "Oh, Action!" (side-by-side) | |
User avatar
BasharOfTheAges
Just zis guy, you know?
 
Joined: 14 Sep 2004
Location: Merrimack, NH
Status: Extreeeeeeeeeme

Postby Ayanefan » Thu Aug 02, 2007 7:27 pm

Scintilla wrote:Usual solution for this problem: Options --> Preferences --> Main tab, then set "Output color depth" to "Use output setting".

For some reason, VDubMod defaults to only 16-bit output color depth, which is insufficient for most any video you find these days.


The God of Avisynth!!!! :thumbs up:
Image
Image
User avatar
Ayanefan
 
Joined: 24 Oct 2005

PreviousNext

Return to Footage Help

Who is online

Users browsing this forum: No registered users and 1 guest