Dupped() - yet another frame duplication function

[Corran]

Alright... so the title says it all. Hopefully someone out there will find this useful.

This function attempts to find frames that are exactly the same as the last frame that changed, a "new frame" so to speak, and then overwrite them with that "new frame".

In other words it works like this:
(Assume captial letters represent frames with actual changes)
AbcdeFGhiJkLmnop

This filter will produce:
AAAAAFGGGJJLLLLL

When the adjusted video is played back looks exactly the same as the original. This works because most anime does not have new movement in every single frame.

What this ultimately means is that when the filtered footage is sent to the encoder, it is much easier to compress because a lot of the frames are mathematically identical. The noise that occurs between each frame is now limited to the frames that actually change.

I realize I'm not actually creating something new. Donald Graft and others have made similar plugins. I've only tested Donald's... but I found that his did not accurately identify very small changes in the video. If I adjusted the threshold to identify very slow pans, zooms, or other motion, it would start picking up on the noise in my source. Therefore my goal with this was to create a function that required little effort to configure assuming a relatively clean source, and the ability to detect small changes accurately. The small mouth movement of the news anchor on the TV in the background? This filter detects it. The slow moving clouds? Same. Granted, there are some situations where the change is way too subtle to detect, but they are far and few between I think.

So here are the details that probably matter to anyone reading this...

How can I use this practically?
Well, it depends really. The biggest benefit will probably be realized if you are the kind of person that uses avisynth to create lossless filtered AVI files and then edit with those, not the scripts themselves. The Lagarith codec has a nifty option called null frames.

[When null frames is enabled,] if the previous frame is mathematically identical to the current, the current frame is discarded and the decoder will simply use the previous frame again.

I took The Girl Who Leapt Through Time at 640x368, cleaned it up, and encoded it with lagarith twice using the null frames option. The first time was without dupped() and the second time was with.
The first encode was <strike>21.0GB.</strike> 10.9GB
The second one was 6.3GB. For a 98 minute lossless encode...

AMV Distrobution benefits?
I haven't tested this, but I imagine tacking this on the end of your script might not hurt in most cases. An AMV such as Bleach Technique Beat won't have much benefit with as much constant motion that there is, but simpler videos would see bigger benefits. However, if you have slow fades to white or black on a still frame of footage, you probably won't want to use this filter as the fade will likely end up looking choppy.

To use this function:

Right Click, Save as, and place in your avisynth plugins directory
dupped.avsi

Currently there are only three arguments.
Thresh=20 will determine that a new frame exists when YMinMax(a,b) is >= 20. The value of YMinMax(a,b) can be found by setting stats=true. I've currently set the default for this parameter to 16. I've only used this on one clean source so it probably needs a little tweaking.
panthresh=1.65is used in calculating the dynamic threshold of low motion scenes. The first two YMinMax values shown in the stats are averaged and then multiplied by panthresh. If the third YMinMax is equal to or greater than this calculation then the current frame is considered a new frame. The dynamic threshold is now displayed in the stats to the right of YMinMax(last,next).
stats=True will display the stats used to determine if there was a change in the frame.
Showme=True will show the video that the stats are created from.

I suggest using this function at the end of your script and only after you have cleaned up your source. Also, the video needs to be in a YUV colorspace. In otherwords, if get an error saying the "Image must be planar" then you need to put converttoyv12() right before the dupped() function.

Here is an example of such usage if needed.

Code: Select all

AVISource("Videofile.avi")
converttoyv12()
dupped()

Also, don't use SetMTMode() if you are using a multi-threaded version of avisynth. (If you don't know if you are, then you aren't running it.) Instead, perform multi-threading on a per-filter basis using MT() like this:

Code: Select all

AVISource("Videofile.avi")
MT("ttempsmooth()",4)
fft3dgpu()
dupped()

Also, it may be obvious... but make sure stats and showme are not enabled during your actual encodes otherwise the frames won't be identical where they should be.

Here are some screenshots of the stats and showme parameters in use.

Code: Select all

dupped(stats=true)

Code: Select all

dupped(stats=true,showme=true)

Here is the code in case the link above ever goes bad.

Code: Select all

# Dupped() by Corran.  Feel free to modify as you see fit. 
# Attribution not required.
#
# This filter requires a YUV source. Use converttoyv12 if needed
# before calling dupped()
#
## Parameters
# thresh = Used to determine when to delare a frame as new
# Use stats=true to help determine the best value for your source
# Lower number = more sensitive. (Default=16)
#
# panthresh=1.7 This is used to determine when to consider a frame
# part of a low motion scene. (Default=1.65)
#
# stats = Enable/disable display of stats. (Default=false | Not shown)
# (Last = last frame, this = this frame, next = next frame)
#
# showme = Enable/disable display of video that most stats are
# derived from. (Default=false | Not shown)
#
## Notes
# Do not use with SetMTMode(). This function requires the frames to be
# accessed sequentially. Instead, use MT() to mulit-thread on a per-filter basis.
#
# Example:
# dupped(thresh=20,panthresh=1.7,stats=true,showme=true) 

#Function to return the requested frame
function getFrame(clip c, int frameNum)
{
    default(c, last)
    finalframe = c.frameCount-1
    
    Assert(frameNum >= 0, "GetFrame(): FrameNumber supplied is less than 0.")
    Assert(frameNum < finalframe, "GetFrame(): FrameNumber supplied is larger than video's framecount.")
    
    c = trim(frameNum,frameNum)
    return c
}

function dupped(clip c, int "thresh", float "panThresh", bool "stats", bool "showme")
{
    default(c, last)
    global dupThresh = default(thresh, 16)
    global panThresh = default(panThresh, 1.65)
    global dupStats = default(stats, false)
    showme = default(showme, false)
    a = c
    c = scriptclip(last,"""
        c = last
    
        finalframe = c.framecount-1
        prev_frame = current_frame != 0 ? current_frame-1 : 0
        next_frame = current_frame != finalframe ? current_frame+1 : finalframe
            
        #Get Y related stats as needed
        prevCurrentYMinMax = subtract(getframe(prev_frame),c).YPlaneMinMaxDifference
        currentNextYMinMax = prevCurrentYMinMax < dupThresh ? subtract(c,getframe(next_frame)).YPlaneMinMaxDifference : 256
        prevNextYMinMax = currentNextYMinMax < dupThresh ? subtract(getframe(prev_frame),getframe(next_frame)).YPlaneMinMaxDifference : 0
    
        #calculate dynamic pan threshold.   Set to 256 if looking of a pan is pointless
        neighborAvgMinMax = (prevCurrentYMinMax + currentNextYMinMax) / 2
        dynPanThresh = prevNextYMinMax != 0 ? Abs(neighborAvgMinMax * panthresh) : 256

        #determine if the current frame is new
        newframe = prevCurrentYMinMax >= dupThresh || prevNextYMinMax >= dynPanThresh ? true : false
        lastNewFrame = newframe ? current_frame : lastNewFrame
    
        c = getFrame(lastNewFrame)
        c = dupStats ? c.subtitle("prevframe: "+string(prev_frame)+"  currentframe: "+string(current_frame)+"  nextframe: "+string(next_frame)) : c
        c = dupStats ? c.subtitle("Last new frame: "+string(lastNewFrame)+" Newframe: "+string(newframe),y=15) : c
        c = dupStats ? c.subtitle("prevCurrentYMinMax: "+string(prevCurrentYMinMax)+" (Threshold:"+string(dupThresh)+")",y=45) : c
        c = dupStats && currentNextYMinMax != 256 ? c.subtitle("currentNextYMinMax: "+string(currentNextYMinMax),y=60) : c
        c = dupStats && prevNextYMinMax >= dynPanThresh ? c.subtitle("prevNextYMinMax: "+string(prevNextYMinMax),y=90) : c
        c = dupStats && prevNextYMinMax >= dynPanThresh ? c.subtitle("Pan Threshold: "+string(dynPanThresh)+" ("+string(neighborAvgMinMax)+" * "+string(panThresh)+")",y=105) : c
        c = dupStats && prevNextYMinMax >= dynPanThresh ? c.subtitle("Slow pan detected",y=120) : c

        return c
    """)
    c = showme ? stackvertical(c,subtract(a.duplicateframe(0),a)) : c
    
    return c
}

[BasharOfTheAges]

I'm getting the following error:

Code: Select all

I don't know what "a" means.
([ScriptClip], line 2)

Tried redownloading it multiple times - no change.

I'm using VdubMod 1.5.10.2

[Corran]

I'm getting the following error:

Code: Select all

I don't know what "a" means.
([ScriptClip], line 2)

Tried redownloading it multiple times - no change.

I'm using VdubMod 1.5.10.2

Ah damn... I'll look into it tomorrow after I get some sleep. For now write your script with variables like this:

Code: Select all

a = AVISource("Videofile.avi").converttoyv12()
a.dupped()

EDIT: Scratch that. Turns out I just needed to declare 'a' a global variable. You can either change the following line or redownload the avsi file.

Change

Code: Select all

a = default(clip, last)

to

Code: Select all

global a = default(clip, last)

[Zarxrax]

Interesting. I'll test this out and see what kind of results I get.

[BasharOfTheAges]

Ok - well, that did get rid of one error, but the other one is a bit more internal - throwing errors for "newframe" on line 17.

[BasharOfTheAges]

Edit - I think you may have fixed that one too. But i can't tell yet.

It's taking a while to get stuff done - SetMTmode(2) is upping my CPU load from roughly 1/3 to 2/3 though.

[Corran]

Ok - well, that did get rid of one error, but the other one is a bit more internal - throwing errors for "newframe" on line 17.

Did you try seeking through the file a frame at a time? I made it so that newframe will equal 0 if the current requested frame was 0 in order to avoid such an undefined variable. If for some reason you started the script at a frame other than the first one, that variable might not be defined until the first changed frame is detected.

It's taking a while to get stuff done - SetMTmode(2) is upping my CPU load from roughly 1/3 to 2/3 though.

I'm not sure how MT will behave with this function. I'd be wary of using it if making an actual encode to edit with though I guess for testing purposes it really doesn't hurt to try it.

Are you only noticing a major slow down when debug and/or showme is enabled? When I removed those, it runs much faster on my PC. I don't plan on trying to optimize this unless someone can teach me how. It would definately run faster if someone made it into an avisynth .dll plugin... but I don't really know how to do that.

[BasharOfTheAges]

Ok - well, that did get rid of one error, but the other one is a bit more internal - throwing errors for "newframe" on line 17.

Did you try seeking through the file a frame at a time? I made it so that newframe will equal 0 if the current requested frame was 0 in order to avoid such an undefined variable. If for some reason you started the script at a frame other than the first one, that variable might not be defined until the first changed frame is detected.

Nope, i am starting from frame 0. Seeking through in Vdub doesn't show the error, but it appears on the output file.

It's taking a while to get stuff done - SetMTmode(2) is upping my CPU load from roughly 1/3 to 2/3 though.

I'm not sure how MT will behave with this function. I'd be wary of using if an actual encode it though I guess testing it really doesn't hurt.

Are you only noticing a major slow down when debug and/or showme is enabled? When I removed those, it runs much faster on my PC.

I have debug and showme off and it's not giving me a "slowdown" perse - it's just not using my processor to it's fullest.

[Corran]

What is the first newFrame that is reported when debug is enabled? If you are using trim() on the source at some point, it may mess with avisynth's internal current_frame counter. If so, I'm not exactly sure how to counteract that off the top of my head.

[Zarxrax]

Hmmm, I did some tests, and your filter appears to be working, but it appears that lagarith's null frame option is not working for me. Corran, what version of lagarith have you tested with?