TDecimate - want to be able to see on the screen when specify a large cycle

[Ikotas]

In TDecimate, if specify very large numbers for cycleR and cycle (ex. cycleR=7779,cycle=38895), cannot see the result on the screen.
(To begin with, setting display=true in this case will force AvsPmod to close.)

Is it possible to split the result display on the screen only, while leaving the cycleR and cycle specifications as they are?
For example, even if cycleR=7779,cycle=38895 is specified, if the screen display can show 250 frames at a time, etc., can check and make progress.

[pinterf]

Could you give me a full script and the used parameters and probably the video you are using it on? I'm not too familiar with the usage, I always like a ready to reproduce scenario, thanks.

[Ikotas]

Any video can be reproduced.
The script is as follows

TDecimate(mode=0,cycleR=7779,cycle=38895,display=true,sdlim=-4)

With a Japanese TV broadcast source (1440x1080), the maximum frame length is 600 frames, which is the limit of what can be seen on the screen display.

TDecimate(mode=0,cycleR=120,cycle=600,display=true,sdlim=-4)

Even when large numbers are specified, such as cycleR=7779,cycle=38895,
want to check which frames have been decimated on the screen, it would be helpful if the decimation result could be divided and displayed by the specified number of frames.

For example,
TDecimate(mode=0,cycleR=7779,cycle=38895,display=true,xxxxx=250,sdlim=-4)
the results will be displayed, separated by 250 frames from the beginning.
(The results are displayed 250 frames at a time, similar to the following script.)
(TDecimate(mode=0,cycleR=50,cycle=250,display=true,sdlim=-4)

[pinterf]

I see. The freeze is just an unexpected side effect.

[pinterf]

Yep, for 1440x1080 even
TDecimate(mode=0,cycleR=120,cycle=600,display=true,sdlim=-4)
is freezing from avspmod, unlike 1920x1080 which is still O.K.

[Ikotas]

Whether it freezes or not is not the purpose of this issue.
I would like to check the result of decimation of the out of screen area that cannot be displayed on the screen when a large cycle is specified.
I don't think there is a way to check this.

[pinterf]

There is no freeze, just did not expect it to be so slow until the first frame appears
Anyway, do you expect something like this? (see the new displayFrom parameter and its effect)

and

[Ikotas]

Thank you for your suggestion.
The method you suggest might work, but it looks like I need to specify the frame number to be displayed on the screen each time, so if I use "cycleR=7779,cycle=38895" and so on, it would be difficult to check all the results.
What I am imagining is,
For example, "TDecimate(mode=0,cycleR=120,cycle=600,display=true,sdlim=-4,displayDecimation=250)",
If frame 150 is currently displayed, a list of frames 0-249 will be displayed,
If frame 300 is currently displayed, a list of frames 250-499 will be displayed, and so on.
The list is paged by the number specified in the parameter from the frame 0, and the range that includes the current frame number is displayed.
The display of the list is similar to "TDecimate(mode=0,cycleR=50,cycle=250,display=true,sdlim=-4)", but the result of decimation is to display what is specified in cycleR and cycle.
With this method, can use Ctrl + cursor keys in AvsPmod to check as move around.

[pinterf]

Oh, I see, so the parameter is not for the positioning, but rather for specifing the frames displayed around the current frame. it makes sense now.

One more question if I understand your idea correctly: the paging would always start at 0, 250, 500, etc (at multiples of displayDecimation), so it would display 0-249 data for frames 0-249 until we reach frame 250, then 250-499 data for frame numbers between 250-499 and so on.
First I thought that a rolling middle would be what you'd like, e.g. display 0-249 data for frame 150, 1-250 for frame 151, etc, as if the actual frame would be the middle of the displayed range.

[Ikotas]

Okay, I did not think of the idea of always displaying the list around the frame that is currently displayed.
However, I think the idea I indicated is a good one.
The reason is that if need to specify a large cycle, want to compare it with a small cycle, and it is easier to compare if the frame number of the display range is the same in both cases.
For example, there is a source with irregular periodic changes.
If there are only 49 overlapping frames in the range of frames 0-249, "cycleR=50,cycle=250" specification would require the decimation of an extra non-overlapping frame (say frame 125), but "cycleR=500,cycle=2500" specification would allow the decimation of frame 2499 can be decimated instead of frame 125.
This example is a simple case, but similarly, if want to decimate duplicate (or unwanted) frames instead of some non-overlapping frames, can easily see that the same frame range is displayed when comparing large and small cycles to see which frames have actually been decimated.

[Ikotas]

As an additional idea, it would be helpful to have the number of frames actually decimated within the range displayed by the displayDecimation I wrote.
However, this may not be necessary if the calculation of the process makes the display heavy.

[pinterf]

Try this test build, please. Setting displayDecimation to other than 0 will display only frame info within the grouping range.
https://drive.google.com/uc?export=download&id=1wEEqNX98eEA7qWmfpi32JP3vdTLP6CTX

[Ikotas]

Thank you very much!
I immediately tried the following settings.
TDecimate(mode=0,cycleR=7779,cycle=38895,displayDecimation=250,display=true)
After the initial long loading, the listings are displayed very quickly and wonderfully!
One thing I noticed is that for 1440x1080 display, the original has the same value for the end of the resulting column and the beginning of the next column (for example, if the end of the column is frame 25000, the beginning of the next column is also 25000), but this version has a +1 value.
The display seems to have changed regardless of whether displayDecimation is used or not.
In the original display, the beginning of the column always starts from 0 at the beginning, and the periodic change is easy to discern.

[Ikotas]

It is not the same as the original, but it would be just as easy to see if the column ended with a 9.

[pinterf]

Actually that was a bug, a regression when I rewrote the screen display part. I noticed that the last value appeared at the beginning of the next column, and the first line of the FILM:... space separated values was repeated as well. Then I checked the original 1.05 version, and it work without duplicating, so I fixed that part.

[Ikotas]

Ah, it was a bug...
However, as stated, at 1080 pixels in height, the first digit was always displayed from 0, so it was easy to notice the cycle change.
Since you fixed the bug, this is an additional request, but would it be difficult to wrap the last digit with 9 or 4?
Also, would it be difficult to make the additional requests listed above?
I mean to display the actual number of decimated digits in the range specified by displayDecimation.
TDecimate(mode=0,cycleR=7779,cycle=38895,displayDecimation=250,display=true)
In this case, usually 50 decimations are performed within the range of 250, but it could be 49, 51, or of course 0 to 250, but it is not clear at a glance how many decimations were performed, so it would be useful if that number is displayed.

I would appreciate it if you would consider my two requests.
At this point, I am very grateful to be able to see information that could not see before.
Thank you very much.

[pinterf]

Displaying the actual number of decimations within the frame range must be easy I think.
The request to display the columns in a nicer way (nice for humans, grouped by rounded column lengths which are dividable by 5 or 10 ), well, this have to be a bit more generalized. Obviously I wouldn't like to hardcode the limits just because of this very use case (height=1080, decimation=250 (5x50)), anyway, there will be a solution.

[Ikotas]

You are right.
In some cases, may want to use "cycleR=50,cycle=300" and so on.
In this case, it is better to wrap at multiples of 6 to make it easier to see.
This is a more selfish request, but it would be useful if the wrapping could be specified arbitrarily.

[Ikotas]

I have come up with a way to make it easier to find the cycles without changing the fold position of the listings.
I would appreciate it if you could adopt it for consideration.

Proposal 1:

Display the interval of decimation as a numerical value.

Example display)

  ...
**103(x)
  104
  105
  106
  107
**105(5)
  106
  107
  108
**109(4)
  110
  111
  112
  113
  114
**115(6)
  ...

The number to be displayed can be either the number of frames counted from the front or the number of frames in between.

Proposal 2:

Specify the interval of decimation as a parameter in advance, and use a symbol to indicate how large or small the interval is compared to the number of frames.
(Because the list display expands horizontally with proposal 1)

Parameter example
displayDecimationInterval=5 (too long name...)

Example of display

  ...
**103
  104
  105
  106
  107
**105
  106
  107
  108
*-109
  110
  111
  112
  113
  114
*+115
  ...

If the number of decimations is less than "5" set in the parameter, "-" is displayed; if the number is greater, "+" is displayed.

If neither of the above is adopted, please implement only the idea of displaying the actual number of decimations within the range of displayDecimation and make it an official release.
Thank you in advance.

[pinterf]

I see, this is a busy week but it's not forgotten.
Regarding the formatting: the '**' can be replaced with any two-character marker. The reason for displaying two asterisks is just having a properly visible marker.
Then we have a place for #.## formatted metrics (e.g. 0.00), then come various optional properties (MTC character, 'mdup' or '(dvdup)' strings)

So the columns are not of the same widths, they are adaptive to the maximum length displayed in that columns. There is some freedom in it.

[Ikotas]

I understand that as a display specification, either is fine.
Proposal 1, if not linked to the TFM, would be easy to view, with no problem with a slightly stretched width.
However, there is a concern about how much the display will stretch horizontally when linked to the TFM.
Proposal 2 will not be a problem because the width will remain the same regardless of whether it is linked to the TFM or not.

I thought of other ways to make it easier to find cycles, but could only come up with ideas for further additions.
The following is a fantasy that I do not fully expect to implement.

1. display the cumulative number of decimations up to the current display range out of the total number specified by cycle.

yyy/xxxx/zzzzz

yyy...Number of decimations within the current displayDecimation range
xxxx...Cumulative number of decimations up to the current display range out of the total number specified by cycle
zzzzz...Total number of decimations specified by cycle

2. color "**xxx" in the list of decimation targets. (I have never seen anything other than white, so I don't think it exists as a specification.)

[pinterf]

Regarding "1. display the cumulative..." could you please edit a theoretically expected screenshot from the above "example of display"?
As I remember, coloring is not an option, unless I made it available when I replaced the old display routines with the one I put in Avisynth and brought it back in this project.

[Ikotas]

This is not a screenshot, but this is what it would look like.

TDecimate v1.0.10 by Tritical
Mode: 0  Cycle: 7779  CycleR: 38895  Hybrid: 0  #ofDecimations: xx / yyyy
inframe: 9221  useframe: 11542 displayDecimation: 250 (11500-11749)

xx: Number of decimations in the current list
yyyy: Total number of decimations from the beginning to this point (including xx)
Omit "zzzzz" since it is listed in "CycleR".
If it is strange to separate by "/", other symbols can be used.
You said before that getting "xx" is easy, but I imagine that getting "yyyy" would be difficult.

[Ikotas]

There was a slight error.
The "zzzzz" is "Cycle", not "CycleR".
Also, I changed the presentation a little.

TDecimate v1.0.10 by Tritical
Mode: 0  Cycle: 7779  CycleR: 38895  Hybrid: 0  #ofDecimations: 49 (2347:5432)
inframe: 9221  useframe: 11542 displayDecimation: 250 (11500-11749)

The left side of the parentheses is "yyyy" and the right side is the remaining value after subtracting "yyyy" from "zzzzz".

[Ikotas]

Oh, it's handwritten... The first one was correct...
The "zzzzz" is "CycleR", not "Cycle"!!

TDecimate v1.0.10 by Tritical
Mode: 0  Cycle: 38895  CycleR: 7779  Hybrid: 0  #ofDecimations: 49 (2347:5432)
inframe: 9221  useframe: 11542 displayDecimation: 250 (11500-11749)

[pinterf]

Check this one, please
https://drive.google.com/uc?export=download&id=15S9IGHxjStxLVeLA7uHkA8y0giT6GFel

[Ikotas]

Oooh! It works exactly as I imagined it would!
Thank you!
I will check the behavior for a while.
I noticed a very small point: the "displayDecimation" and "#ofDecimations" are preceded by " " (single space) and the original parameter interval seems to be " " (double spaces).
Of course, there is no problem at all, as it has nothing to do with the operation.

How about a proposal to make it easier to find a cycle changes in the list?
If it is difficult to implement that, I will close this issue if there are no bugs in this fix.

[pinterf]

Single space, yes. You have eyes like a hawk.
Next build, see also TDecimate - READ ME.txt: use displayOpt parameter.
Please check the parameter description in the readme as well, I guess my description would not be 100% appropriate.
Check this:
https://drive.google.com/uc?export=download&id=1I84AQ5T2RIXeMTSYFhIWZ6KdIem8L9Jj

[Ikotas]

dysplayOpt! Operation confirmed.
It will be very useful as expected as well!
Thank you very much!

How about this explanation of the options?
I am not confident in the accuracy of the text as I am not an English speaker, so please feel free to modify it as you see fit.
Also, I think the explanation is a bit too long, so please use what you think is necessary.

---

displayDecimation -

If more than zero then starting at the first value specified in Cycle, frames are displayed in a list of frames for the value specified in displayDecimation.
The list is fixed as in the case of simply setting display=true, and when a frame outside the list is displayed, the display switches to the next range.

e.g. displayDecimation = 250

data from frames 0-249 are displayed for frame #0
data from frames 0-249 are displayed for frame #1
...
data from frames 0-249 are displayed for frame #249
data from frames 250-499 are displayed for frame #250
data from frames 250-499 are displayed for frame #251
...

If the Cycle range crosses within the range specified by displayDecimation, the pages are displayed separately.(Page means that the listings are displayed separately.)

e.g. CycleR=20,Cycle=100,displayDecimation = 40

data from frames 80-119 are displayed for frame #80
data from frames 80-119 are displayed for frame #99
on page 1 (Cycle range is 0-99)

data from frames 80-119 are displayed for frame #100
data from frames 80-119 are displayed for frame #119
on page 2 (Cycle range is 100-199)

When this parameter is enabled, "#ofDecimations" is displayed.

#ofDecimations: x (y:z)

x  Number of decimations in the current list
y  Total number of decimations from the first to the current listing in the range specified by CycleR (including x)
z  Remaining number of decimations after subtracting y from the value of CycleR

This parameter is useful for large cycles, when the screen area is not enough to display all data within the range.

default: 0 (int)

---

I think the description of displayOpt is fine.
One point, I think this option is useful even when displayDecimation is not specified, so I thought it would be good to mention that it is not required in combination with displayDecimation, but can be used by itself.

[Ikotas]

Oh, I thought it would be a good idea to mention to Changelog that have fixed the list wrap bug.

[Ikotas]

Two requests about displayOpt.

1. about "*+"
   When I use TDecimate, I use AvsPmod to resize the image to 1280x720 for better displayability, but I noticed that it is difficult to distinguish between "**" and "*+" when resizing the image. (It's partly because of my poor eyesight...)
   How about "*!" instead of "*+"?
   Other symbols are acceptable if they are easier to see.

2. about "*X" at the beginning of the current list
   Is it possible to display the symbol at the beginning of the current list as the distance from the previous decimation?
   If it is difficult, can leave it as it is.

[pinterf]

1.) *! is not very intuitive. >> and << or -- and ++ probably. Anyway I'm gonna check them visually.
2.) I noticed too, that the first *+ or *- has no meaning, and needs to be replaced with something other.

[Ikotas]

1.) Those symbols looks good.
I have a few ideas to add as well.

<<,**,>>
--,**,++
-<,**,>+
<-,**,+>
<-,**,->
<=,**,=>
<*,**,*>
<_,**,_>

Use your judgment and adopt the one with the best visibility, please.

2.) I understood that it is not possible to display a symbol at a distance from the last decimation of the previous range of the current range.
If so, then simply "**" might be fine.

[pinterf]

Some other variants (x64 versions)
https://drive.google.com/uc?export=download&id=1dER9CuGBC8LmcCT13W-z_1EhDlP4eFgz

[Ikotas]

Wow! Thanks for the quick response!
I have checked variant1-4.
In my opinion, the order is as follows

4 >= 1 > 3 > 2

1 and 4 are hard to choose between, but I felt that 4 is easier to quickly understand than 1 because of the variation in the size of the letters.

[Ikotas]

I created an image with "**" also changed for a bit of inspiration, which is also easy to see.

<-,<>,->

However, I don't think it is possible to change the "**" because it would be annoying to those who are not involved in this change.

[Ikotas]

Ignore the <-,<>,-> in the post above.

Hmmm, I am confused.
I'm starting to think that 1 is better than 4 for visibility in normal cases, since it is rare to have many symbols mixed together in a single screen listing, as in the image above.
Normal case is the image of finding 1-2 symbols out of about 33-50 "**".
Please give me some time to try a few more and decide which one is best.
By the way, which one do you think is better?

[Ikotas]

I have compared them for a while and the visibility of 1 seems to be the best.
If it is OK with you, please adopt 1.

One thing I noticed is that in the first list, "#ofDecimations x(y:z)" seems to show "x" with one less number. "y" is correct.

The following is an additional explanation of displayOpt in "TDecimate - READ ME.txt".

This display starts from the first frame drop in the current list, and the first frame drop is always indicated by "**", no matter where it is located.
Also, the before and after of the currently displayed listing is not taken into account.

[pinterf]

O.K. Then let it be *- or ** or *+ ? I liked #4 (<- ** +>) for the first sight but didn't test them, just had a quick look at it.

One thing I noticed is that in the first list, "#ofDecimations x(y:z)" seems to show "x" with one less number. "y" is correct.

Yep, I missed it, done.

[Ikotas]

I should have stated it clearly.
1 is variant1(<<,**,>>)!
This combination is the most visible.
Adopt this one, please!

Yep, I missed it, done.

Thanks for the correction!

[Ikotas]

I also liked variant4 at first, but on a screen scaled down to 1280x720, it was easier to find one ">>" than to find one "+>" out of 33 "**".
">>" is more noticeable.
Perhaps the "+" and "*" look similar when reduced in size, and the repetition of the same symbols is easily noticeable.

[pinterf]

New build with << and >> and fixed diff-by-one
https://drive.google.com/uc?export=download&id=1bd7JGneKcWJxg2lcMrsSnRi3PDx0ILYE

[Ikotas]

Thank you, it is perfect for me.
Now just need to release it.
I hope these changes will be useful to someone else.
Let me know if there is anything else I can help you with, such as documentation checks.

[Ikotas]

I really like and use this modified version.
Thank you very much!

I noticed while using a large Cycle, the first frame drop symbol in the list you corrected last time, but I am reporting that in some cases, a symbol that is not "**" is displayed.

◆Cases I have confirmed

Case 1.
Set value
TDecimate(mode=0,cycleR=462,cycle=2786,ovr="xxx.txt",display=true,displayDecimation=199,displayOpt=6)

Where it started to occur
displayDecimation: 199 (1592-1790)

Case 2.
Set value
TDecimate(mode=0,cycleR=3102,cycle=18706,ovr="xxx.txt",display=true,displayDecimation=199,displayOpt=6)

Where it started to occur
displayDecimation: 199 (17512-17710)

Case 3.
Set value
TDecimate(mode=0,cycleR=330,cycle=1990,ovr="xxx.txt",display=true,displayDecimation=199,displayOpt=6)

Where it started to occur
displayDecimation: 199 (796-994)

The above is just one example I have confirmed.
It does not only occur in specific parts of a specific source, so you can confirm this by trying it in any source with the configuration values listed above that exclude the ovr specification.

If you have difficulty correcting this event, I would be happy to have you revert to the test3(TIVTC_dev_v1.0.28test3.7z) state.(I thought it was a matter of familiarity.)
(The symbols for the first frame drop in the list should remain as in test3, changing only "<<", "**",">>".)
Thank you for your time.

[pinterf]

Thanks for the feedback, I'm looking into it.
EDIT: Isn't it normal? The first symbol is always **, but only on the first page of the cycle where the 'from' value is zero in DisplayDecimations X (0-Y). If we are still within the cycle, but the 2nd, 3rd etc.. page is displayed, then we'd still have a valid decimation from the previous page and te proper << or >> marker could be displayed.
If this wasn't your intention then it's super easy to modify it to show always ** for the first decimation on each page, just tell me.

[Ikotas]

Ah, that's what you mean! I completely understand!
It was implemented so that the mark changes with the distance from the last frame drop of one of the previous ranges!
In fact, it was implemented as I had requested, but I was unaware of it myself.
And many times I had presented comments and document suggestions with the wrong understanding.
I am very sorry.
This specification is exactly as I requested, so it is perfectly fine as it is!
Thank you very much!

The only other thing is that there is only one space between "userframe:" and "displayDecimation:".
"#ofDecimations:" has been changed to two spaces, just in case. (It's such a small notice that it doesn't even need to be fixed.)
I will close this issue after reviewing the release, but if there are any differences from the rules for closing an issue, please let me know.

[Ikotas]

I have confirmed that it has been released and will close this issue.
Thank you very much for your patience with my request for over a month.
I appreciate it very much.
I hope these changes will be useful to someone.

[Ikotas]

Since you implemented my request, I was also motivated to create a guide for IVTC from 29.97fps to 25fps utilizing TDecimate.
I'm also actively utilizing the parameters you implemented in this Issue.
To begin with, I requested my request to make this IVTC process more efficient.
Once again, thank you for implementing my request.
I have just completed and published it, and I am here to let you know.

https://ikotas.github.io/25fpsIVTCGuide/

There is only a Japanese version, but I think the translation of Chrome is quite good these days, so I hope it won't be a problem to understand the meaning.

I was actually going to submit this guide to doom9, but since I just couldn't get the Activation email and had no way to contact the administrator, I gave up for now....
If any of you who subscribe to this Issue and receive updates via email find the information in this guide useful, I would appreciate it if you would share it on doom9.
I will only be watching....

In addition, I apologize if such a post is against the rules.
In that case, please delete this comment.

[pinterf]

Thank you for the contribution, I'm sure it will be useful to many.

[Ikotas]

Thank you for posting your share on doom9.
I am relieved that the guide seems useful.
I appreciate your kindness!

I will tell you what I noticed.
URL, but maybe "&_x_tr_hl=hu" is given extra.
I wondered because the Google Translate display at the top was in a language I could not read.