Rewriting AVS job scripts

This forum is for questions and discussion of all the aspects of handling and cleaning up your footage with Avisynth.
Locked
User avatar
DriftRoot
Joined: Mon Jun 09, 2003 7:18 pm
Status: As important as any plug-in.
Location: N.H.
Org Profile

Rewriting AVS job scripts

Post by DriftRoot » Sat Oct 23, 2010 11:04 am

Problem: A bunch of the job control scripts I've saved in case I need to ever remake some clips contain inaccuracies, mostly having to do with compression settings. I need to reload and re-run these job scripts with the correct settings.

Question: What is the fastest, easiest way to output new clips at the correct settings using the old job scripts? Is there some function in AVS I don't know about that will rewrite the job script to my current AVS settings, or do I need to edit the scripts by hand, replacing applicable settings with the correct ones? If it's the latter, is the VirtualDub.video.SetCompression line the only one I need to worry about editing?

Code: Select all

// $input "C:\...Disc5.avs"
// $output "C:\...TEST.avi"
// $state 0
// $id 1c495e6009bf1c0
// $start_time 00000000 00000000
// $end_time 00000000 00000000
// $script

VirtualDub.Open("C:\...Disc5.avs","",0);
VirtualDub.audio.SetSource(0);
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,500,1,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.audio.EnableFilterGraph(0);
VirtualDub.video.SetInputFormat(0);
VirtualDub.video.SetOutputFormat(7);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetSmartRendering(0);
VirtualDub.video.SetPreserveEmptyFrames(0);
VirtualDub.video.SetFrameRate2(0,0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetCompression(0x7367616c,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.audio.filters.Clear();
VirtualDub.subset.Clear();
VirtualDub.subset.AddRange(0,109116);
VirtualDub.video.SetRangeFrames(908,940);
VirtualDub.project.ClearTextInfo();
  // -- $reloadstop --
VirtualDub.SaveAVI("C:\...TEST.avi");
VirtualDub.audio.SetSource(0);
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done
Thanks in advance for any assistance!
Image

Mister Hatt
Joined: Tue Dec 25, 2007 8:26 am
Status: better than you
Contact:
Org Profile

Re: Rewriting AVS job scripts

Post by Mister Hatt » Sat Oct 23, 2010 11:38 am

What exactly are you trying to do? It looks like you're using virtualdub to do what avisynth normally does, and that you think avisynth is capable of editing vdub job control scripts...

User avatar
Phantasmagoriat
Joined: Mon Feb 06, 2006 11:26 pm
Status: ☁SteamPunked≈☂
Contact:
Org Profile

Re: Rewriting AVS job scripts

Post by Phantasmagoriat » Sat Oct 23, 2010 1:17 pm

Sounds like you are mixing up avisynth scripts (.avs file extension) with virtualdub job control scripts (.job file extension). Avisynth scripts simply serve your footage to virtualdub [and apply their own filters in addition to the basic mpeg2source("index.d2v") line]. Virtualdub job control scripts take the footage that is served to virtualdub, and records a list of everything you want to do to it later, including virtualdub's own filters and final compression settings. I recently made a Visual Aid that may help explain this. Although job control is not on there... it would go at the arrow between [AvsPmod or Virtualdub] ---> [(.avi) Lossless Video]

So basically, avisynth occurs well beforehand and has nothing to do with this. :P

Regarding the .job script you posted, it sounds like you are right about changing the VirtualDub.video.SetCompression line, but I would have to know what both the old and new settings are. So save a new .job file with the settings you want, and compare it with the old one. If you used all the same vdub filters, I think everything would be the same except the compression line. Still, pay close attention to all differences. I haven't looked at a .job script in a very long time, but I believe the first value (0x7367616c in this case) references the codec you want to compress with, and the following numbers are miscellaneous settings used by the codec.

If you need to change anything, first make a backup copy, then in notpad use ctrl+h to replace the old settings with the new ones. Save, then run your new job with vdub.

Hope this helps
Image
Org Profile | AMVGuide | Phan Picks! | THE424SHOW | YouTube | "Galactic Escape"

"Effort to Understand; Effort to be Understood; to See through Different Eyes."

User avatar
DriftRoot
Joined: Mon Jun 09, 2003 7:18 pm
Status: As important as any plug-in.
Location: N.H.
Org Profile

Re: Rewriting AVS job scripts

Post by DriftRoot » Sat Oct 23, 2010 6:22 pm

Yeah, I meant VDM job scripts, not AVS scripts...sorry about that. I've been toiling very hard with both today and wound up lumping my aggravation into one word that doesn't really describe either. :sweat:

I have done some tests and the only differences in the script are that VirtualDub.video.SetCompression line and another line, called out as VirtualDub.video.SetMode. From what you said, Phantasmagoriat, I agree that VirtualDub.video.SetCompression(); seems to set the codec. I have confirmed that - when this setting is undefined - no codec is being set. There's another line that also changes, though, that I suspect specifies the processing mode - VirtualDub.video.SetMode(3); presumably being Full Processing Mode, since Fast Recompress changes this to VirtualDub.video.SetMode(1);.

Whether these lines contain all the information required for the correct compression settings is sort of up in the air, but I would HOPE that anything not set by the script would simply be controlled by whatever settings are in the AVS script and VDM itself at the time of export.
Image

User avatar
Phantasmagoriat
Joined: Mon Feb 06, 2006 11:26 pm
Status: ☁SteamPunked≈☂
Contact:
Org Profile

Re: Rewriting AVS job scripts

Post by Phantasmagoriat » Sat Oct 23, 2010 7:31 pm

Sounds about right, although avisynth scripts don't hold any compression settings. They just make modifications to the picture itself. Maybe I should have asked first, but what "settings" were you trying to change anyway? If I had to take a wild guess, you want to change compression from Lagarith/HuffYUV to the Ut Video Codec. I only bring this up because I'm thinking of doing that for all my lossless footage myself.
Image
Org Profile | AMVGuide | Phan Picks! | THE424SHOW | YouTube | "Galactic Escape"

"Effort to Understand; Effort to be Understood; to See through Different Eyes."

User avatar
DriftRoot
Joined: Mon Jun 09, 2003 7:18 pm
Status: As important as any plug-in.
Location: N.H.
Org Profile

Re: Rewriting AVS job scripts

Post by DriftRoot » Sat Oct 23, 2010 9:13 pm

I know it's not the AVS script - I got my acronyms messed up earlier. This is the VDM job list file I'm messing with. :)

To answer your question about what the heck I'm up to and why, I want to change my compression settings, which isn't necessarily just the codec being used...but I don't know for sure. What I DO know is that when I make clips and forget to don't set the compression type or codec, the job file contains:

Code: Select all

VirtualDub.video.SetMode(3)
VirtualDub.video.SetCompression();
And when I do set Fast Recompress with Lagarith, the file contains:

Code: Select all

VirtualDub.video.SetMode(1);
VirtualDub.video.SetCompression(0x7367616c,0,10000,0);
What I want is the Fast Recompress/Lagarith results, so I'm taking the approach that the files must be edited to the settings which give me those results. As a double precaution, I'm setting VDM TO those settings before running the job list, just in case something isn't being controlled by that job file. So far it seems to be working out properly.

No idea if any of this is in the Read <a href=http://www.a-m-v.org/guides/avtech31/>ErMaC & AbsoluteDestiny's Friendly AMV Guides</a>, but I never gave much thought to what a job list controls. I've routinely changed compression settings, loaded an old job file, and never though twice about the results actually were. I only noticed something was wrong recently because of the enormous file sizes I was getting. I should check to see whether the compression settings are saved when you save the job list, or when the job list is being built...I would hope it's the former, because...well, that's scary to think of all the screwed up clips I've made, otherwise!
Image

User avatar
Qyot27
Surreptitious fluffy bunny
Joined: Fri Aug 30, 2002 12:08 pm
Status: Creepin' between the bullfrogs
Location: St. Pete, FL
Contact:
Org Profile

Re: Rewriting AVS job scripts

Post by Qyot27 » Sat Oct 23, 2010 10:00 pm

I've always found VDub's scripting abilities needlessly cumbersome and just a general mess whenever I've needed to use it (which is why I don't, and moved to other, simpler-to-control solutions). There are much simpler solutions than relying on VDub's job control for this.
DriftRoot wrote:No idea if any of this is in the Read ErMaC & AbsoluteDestiny's Friendly AMV Guides, but I never gave much thought to what a job list controls. I've routinely changed compression settings, loaded an old job file, and never though twice about the results actually were. I only noticed something was wrong recently because of the enormous file sizes I was getting. I should check to see whether the compression settings are saved when you save the job list, or when the job list is being built...I would hope it's the former, because...well, that's scary to think of all the screwed up clips I've made, otherwise!
I actually am about 99% sure that the job control file overrides anything previously set (since it does store compression settings), but the best way to find out is use the method you're currently using, and then perform a manual test to see if you get the same result. If they match, then the situation is ok. If they don't match, then you've got a problem. But that's somewhat inconsequential due to the first part of your response.
DriftRoot wrote:What I want is the Fast Recompress/Lagarith results, so I'm taking the approach that the files must be edited to the settings which give me those results. As a double precaution, I'm setting VDM TO those settings before running the job list, just in case something isn't being controlled by that job file. So far it seems to be working out properly.
Then there is, like I said, a far simpler way of doing this than even messing with VDub at all.

Download avs2avi here, unpack it and put the .exe file in C:\WINDOWS

Then, all that's required is (for Lagarith):

Code: Select all

avs2avi input.avs output.avi -c LAGS
You can batch script this by repeating that command over and over in a plain text file with a .bat extension, only changing the names of the input.avs and output.avi files (you can also change the codec by changing the -c part). Like so:

Code: Select all

avs2avi ep1.avs ep1.avi -c LAGS
avs2avi ep2.avs ep2.avi -c ULY0
avs2avi ep3.avs ep3.avi -c MJPG
(ULY0 is the YUV 4:2:0 codec in Ut Video)

Then all that's required is to double-click on it and Windows will automatically run the commands. Leaving out the -c parameter will allow you to choose the compression format from the standard VCM dialog box. This can be used to save custom codec settings because -c uses the codec's defaults. To create a configuration file, the following would be used:

Code: Select all

avs2avi test.avs test.avi -s lagarith.conf -e
And then all that would be necessary to use that configuration file is to do:

Code: Select all

avs2avi test.avs test.avi -l lagarith.conf
Just be aware that loading that .conf file will also change the settings in anything else (read: VDub, et al) that also uses the Video for Windows framework.


So say you needed to convert 20 AviSynth scripts to Lagarith AVI files. The batch script to do that could look like:

Code: Select all

avs2avi 1.avs 1.avi -l lagarith.conf
avs2avi 2.avs 2.avi -l lagarith.conf
avs2avi 3.avs 3.avi -l lagarith.conf
avs2avi 4.avs 4.avi -l lagarith.conf
avs2avi 5.avs 5.avi -l lagarith.conf
avs2avi 6.avs 6.avi -l lagarith.conf
avs2avi 7.avs 7.avi -l lagarith.conf
avs2avi 8.avs 8.avi -l lagarith.conf
avs2avi 9.avs 9.avi -l lagarith.conf
avs2avi 10.avs 10.avi -l lagarith.conf
avs2avi 11.avs 11.avi -l lagarith.conf
avs2avi 12.avs 12.avi -l lagarith.conf
avs2avi 13.avs 13.avi -l lagarith.conf
avs2avi 14.avs 14.avi -l lagarith.conf
avs2avi 15.avs 15.avi -l lagarith.conf
avs2avi 16.avs 16.avi -l lagarith.conf
avs2avi 17.avs 17.avi -l lagarith.conf
avs2avi 18.avs 18.avi -l lagarith.conf
avs2avi 19.avs 19.avi -l lagarith.conf
avs2avi 20.avs 20.avi -l lagarith.conf
There are ways of contextualizing things like this through the use of shell variables, but I generally don't bother with that since I almost never do things that general. Depending on your workflow, composing a script like the above can be simplified by copying/pasting all those filenames at once with something like Ninotech Path Copy or the Copy File Name to Text option in ExplorerXP, and duplicating the other parameters through the use of Notepad's or Wordpad's 'Replace' function, or duplicating cells in a spreadsheet program.
My profile on MyAnimeList | Quasistatic Regret: yeah, yeah, I finally got a blog

User avatar
Phantasmagoriat
Joined: Mon Feb 06, 2006 11:26 pm
Status: ☁SteamPunked≈☂
Contact:
Org Profile

Re: Rewriting AVS job scripts

Post by Phantasmagoriat » Sat Oct 23, 2010 10:22 pm

@DriftRoot:
I see. Well at least I was right about Lagarith being used :o
Glad to hear it's working so far :)
As a side note, you can totally bait-and-switch the Lagarith-encoded files with Ut-encoded files at any time if you want to try it out [preliminary tests are very promising]

@Qyot: haha, I used to do something similar back when I still edited with clips. Now I just frameserve everything into my editor and don't bother with any of that [unless making proxy files].
Image
Org Profile | AMVGuide | Phan Picks! | THE424SHOW | YouTube | "Galactic Escape"

"Effort to Understand; Effort to be Understood; to See through Different Eyes."

Mister Hatt
Joined: Tue Dec 25, 2007 8:26 am
Status: better than you
Contact:
Org Profile

Re: Rewriting AVS job scripts

Post by Mister Hatt » Sat Oct 23, 2010 11:11 pm

I'd just use sed to rewrite chunks of my vdub job files. A simple regex replace should do it.

User avatar
DriftRoot
Joined: Mon Jun 09, 2003 7:18 pm
Status: As important as any plug-in.
Location: N.H.
Org Profile

Re: Rewriting AVS job scripts

Post by DriftRoot » Sun Oct 24, 2010 7:51 am

@Qyot27: That is indeed a handy method - I knew there had to be something less cumbersome and more tweakable than those job files. Unfortunately in my case, sifting through the old VDM job files and extracting the proper input/output file names for avs2avi would probably take more time than a find/replace on the target code. Going forward, however, I will definitely give avs2avi a try!

@Phantasmagoriat: Interesting to hear the bait-and-switch is an option. Are any editing programs like Premiere having issues with the file sizes? I still am haunted by the days when bait-and-switch was a necessity and switching to the high-quality footage included crossing my fingers and saying a few prayers.
Image

User avatar
DriftRoot
Joined: Mon Jun 09, 2003 7:18 pm
Status: As important as any plug-in.
Location: N.H.
Org Profile

Re: Rewriting AVS job scripts

Post by DriftRoot » Sun Oct 24, 2010 10:28 am

Also wanted to add one thing that I discovered, which perhaps more frequent coders already are quite aware of - this is for people who aren't aware (or, like me, are aware, but haven't had their coffee, yet): watch out editing a job file in programs which try to interpret quote marks "properly." To err on the safe side, you only want straight quotes in the job file (""), NOT open or closed quotes (“”). Some command lines it doesn't make a difference, but some it does.
Image

Mister Hatt
Joined: Tue Dec 25, 2007 8:26 am
Status: better than you
Contact:
Org Profile

Re: Rewriting AVS job scripts

Post by Mister Hatt » Sun Oct 24, 2010 10:50 am

It also makes a difference if you use single or double quotes, especially under POSIX compliant applications. That said, sed or perl are still the fastest way to go because you can write a transform function that'll turn the entire VDM script into an avs script, or at least change some parts of it in one go. It's quite nifty when you know how to use regular expressions.

User avatar
Phantasmagoriat
Joined: Mon Feb 06, 2006 11:26 pm
Status: ☁SteamPunked≈☂
Contact:
Org Profile

Re: Rewriting AVS job scripts

Post by Phantasmagoriat » Sun Oct 24, 2010 4:02 pm

DriftRoot wrote:Interesting to hear the bait-and-switch is an option. Are any editing programs like Premiere having issues with the file sizes? I still am haunted by the days when bait-and-switch was a necessity and switching to the high-quality footage included crossing my fingers and saying a few prayers.
If you're talking about the 4GB size limit, I think most editing applications have gotten past that problem. Even Magix fixed that recently. If it's any consolation, just yesterday, I bait-and-switched a 5GB HuffYUV file with a 4GB Ut file in After Effects and had no problems. Of course, you'll never know if it'll work until you try it :amv:

DriftRoot wrote:you only want straight quotes in the job file (""), NOT open or closed quotes (“”). Some command lines it doesn't make a difference, but some it does.
This reminds me of when I first joined the org. IIRC the forums used (“”) not ("") at the time. Needless to say this was the cause of much hair loss whenever people offered scripts that didn't work :lol: or if they used Word to type their scripts :dino:

Mister Hatt wrote:It also makes a difference if you use single or double quotes, especially under POSIX compliant applications. That said, sed or perl are still the fastest way to go because you can write a transform function that'll turn the entire VDM script into an avs script, or at least change some parts of it in one go. It's quite nifty when you know how to use regular expressions.
Well yeah, if you want to learn sed or perl first :P
Another thing you can do is load your vdub filters directly through avisynth and build your script that way. Although I rarely use vdub filters since the ones made for avisynth are generally more customizable. However, in this case you would still need to make a job queue of sorts, so what DriftRoot is doing is probably the most straight-forward method.
Image
Org Profile | AMVGuide | Phan Picks! | THE424SHOW | YouTube | "Galactic Escape"

"Effort to Understand; Effort to be Understood; to See through Different Eyes."

Mister Hatt
Joined: Tue Dec 25, 2007 8:26 am
Status: better than you
Contact:
Org Profile

Re: Rewriting AVS job scripts

Post by Mister Hatt » Mon Oct 25, 2010 1:34 am

sed -e's/lanczos/spline36/g' -e's/1280,720/848,480/g' -i file1.avs -i file2.avs -i file9001.avs

^magical sed line that turns an ugly lanczos upscale into a sugoi splined pretty encode and is capable of dealing with mixed non-upscaled and already splined content over several files~

And this is like easymodo :V

User avatar
Phantasmagoriat
Joined: Mon Feb 06, 2006 11:26 pm
Status: ☁SteamPunked≈☂
Contact:
Org Profile

Re: Rewriting AVS job scripts

Post by Phantasmagoriat » Mon Oct 25, 2010 11:33 am

magical sed line is magical :awesome:
Image
Org Profile | AMVGuide | Phan Picks! | THE424SHOW | YouTube | "Galactic Escape"

"Effort to Understand; Effort to be Understood; to See through Different Eyes."

Locked

Return to “AviSynth Help”