Decaying() Returns Unexpected Values

[SamuelLKane]

How to reproduce the behavior

Add the following code to any project which includes spaCy

from spacy import util

sizes = util.decaying(1., 10., 0.001)

size = next(sizes)
print (size)
assert size == 1.
size = next(sizes)
print (size)
assert size == 1. - 0.001
size = next(sizes)
print (size)
assert size == 1. - 0.001 - 0.001

This is a direct test of the example provided in the spaCy docs for util.decaying. It will fail on the first assertion.

Additionally the example shows an impossible sequence as this is a decaying series and 1 > 10. If you invert the start and end values you do get a sequence which never decays below the end.

Looking at the actual series you can see that it does not decay at a rate of 0.001 but some approximately close number lost to floating point math which eventually results in the ability to get nearly duplicate values in the series:

these values are next to each other when viewing the series defined by
decaying(1., 10., 0.001)

0.8257638315441783
0.8250825082508251

There is also a problem with how the decay factor is considered. If you use a larger factor the results are completely non-sensical:

dropout = decaying(10., 1., 0.45)
6.8965517241379315
5.2631578947368425
4.25531914893617
3.5714285714285716
3.076923076923077
2.7027027027027026
2.4096385542168672
2.173913043478261
1.9801980198019802
1.8181818181818181
1.680672268907563
1.5625
1.4598540145985401
1.36986301369863
1.2903225806451613
1.2195121951219514
1.1560693641618496
1.098901098901099
1.0471204188481675
1.0

I expressed this issue on twitter in this thread. This issue is mainly being opened so I can make the PR per the contribution guidelines.

Your Environment

• spaCy version: 2.0.18
• Platform: Darwin-18.2.0-x86_64-i386-64bit (macOS Mojave 10.14.3)
• Python version: 3.7.1
• Models: en

[honnibal]

Thanks for this!

I was sure I replied to that comment on Twitter, but I don't see it there. I guess I must not have. Maybe I lost connection after typing the tweet. Sorry!

Anyway, I would've encouraged you to open an issue, and noted that this type of thing is one of my weaker points.

[ines]

@honnibal Here it is btw – Twitter just makes it difficult to find nested replies: https://twitter.com/honnibal/status/1100848503759216640

[SamuelLKane]

I have pushed up a branch and made a PR - slk/issue#3447- you can see the change I've made to util.py. The change I've made brings the model for calculating the linear series more in line with how other methods calculate/return a series (i.e. compounding, stepping) as well as simplifies the logic so that it is a truly linear series.

I think this does address the core problem with how the method was originally defined but doesn't address the error I mentioned above about floating point inaccuracy. Here are the outputs for my version of decaying (new) and the current version of decaying (old).

decaying( 10., 1., .001)

'old'	'new'
9.990009990009991	10.0
9.980039920159681	9.999
9.970089730807578	9.998000000000001
9.9601593625498	9.997000000000002
9.950248756218906	9.996000000000002
9.940357852882704	9.995000000000003
9.9304865938431	9.994000000000003
9.920634920634921	9.993000000000004
9.910802775024777	9.992000000000004
9.900990099009901	9.991000000000005
9.891196834817015	9.990000000000006
9.881422924901186	9.989000000000006
9.87166831194472	9.988000000000007
9.861932938856016	9.987000000000007
9.852216748768473	9.986000000000008
9.84251968503937	9.985000000000008
9.832841691248772	9.984000000000009
9.823182711198427	9.98300000000001
9.813542688910697	9.98200000000001
9.803921568627452	9.98100000000001
9.794319294809013	9.980000000000011

[honnibal]

Merged, thanks!

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.