gh-108963: using random to generate unique string in sys.intern test #109491

aisk · 2023-09-16T12:50:40Z

Issue: test_sys.test_intern() fails if run more than once in the same process #108963

vstinner · 2023-09-18T11:49:46Z

This change hides the issue that the test actually leaks memory: intern a string which is then only released at Python exit. Can we add a private deintern() function in _testcapi? Or run the test in a subprocess?

Well, i don't know that it's a big deal, this change is ooookay-ish.

vstinner · 2023-09-18T11:51:35Z

Lib/test/test_sys.py

        self.assertRaises(TypeError, sys.intern)
-        s = "never interned before" + str(INTERN_NUMRUNS)
+        s = "never interned before" + str(random.random())


I would prefer using random.randint(), for example 10**9 to have 9 random digits. I prefer to avoid float when possible 😁 Same below.

But this increases the probability of random failure from 1/253 (about 1/1016) to 1/(9*10**8). If run the test every second, there will be a failure every 30 years in average.

On my machine, the random.random() returns something like 0.35934878814573135, which have 18 digits (with the first zero). So can we just increase the max range to 10 ** 18 to keep the random resolution almost like as the random.random()?

serhiy-storchaka · 2023-09-19T14:57:31Z

Lib/test/test_sys.py

@@ -693,7 +693,7 @@ def test_43581(self):

    def test_intern(self):
        self.assertRaises(TypeError, sys.intern)
-        s = "never interned before" + str(random.int(10**8, 10**9))
+        s = "never interned before" + str(random.randint(10**8, 10**9))


If the intention is to generate random 9-digit numbers, use randrange() instead of randint().

vstinner · 2023-09-19T15:00:51Z

Lib/test/test_sys.py

@@ -693,7 +693,7 @@ def test_43581(self):

    def test_intern(self):
        self.assertRaises(TypeError, sys.intern)
-        s = "never interned before" + str(random.random())
+        s = "never interned before" + str(random.int(10**8, 10**9))


random.int doesn't exist. What's the point of putting a minimum at 10**8? Why not 0?

There is a more recent commit in the PR fixed the bug. The minimal start is for ensure the random parts have 9 digits, I think it's more consistent.

i don't think that the string length (number of random int digits) matters here, does it?

vstinner · 2023-09-21T11:38:00Z

Lib/test/test_sys.py

        self.assertRaises(TypeError, sys.intern)
-        s = "never interned before" + str(INTERN_NUMRUNS)
+        s = "never interned before" + str(random.randrange(0, 10**18))


Oh, for me 9 digits was already way enough. 18 digits is just too much 😁 You can just pass the max value:

Suggested change

s = "never interned before" + str(random.randrange(0, 10**18))

s = "never interned before" + str(random.randrange(10**9))

vstinner

LGTM.

It makes me sad to leak memory. But I'm not sure about a private API to de-intern a string. I'm not sure of the consequences.

miss-islington · 2023-10-02T11:08:00Z

Thanks @aisk for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington · 2023-10-02T11:08:02Z

Sorry, @aisk and @vstinner, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 44b1e4ea4842c6cdc1bedba7aaeb93f236b3ec08 3.12

… test (python#109491) (cherry picked from commit 44b1e4e)

bedevere-app · 2023-10-02T11:17:33Z

GH-110216 is a backport of this pull request to the 3.12 branch.

…n test … (#110216) gh-108963: using random to generate unique string in sys.intern test (#109491) (cherry picked from commit 44b1e4e) Co-authored-by: AN Long <aisk@users.noreply.github.com>

… test (python#109491)

using random to generate unique string in sys.intern test

bdcae08

bedevere-app bot added awaiting review tests Tests in the Lib/test dir labels Sep 16, 2023

aisk changed the title ~~using random to generate unique string in sys.intern test~~ gh-108963: using random to generate unique string in sys.intern test Sep 16, 2023

bedevere-app bot mentioned this pull request Sep 16, 2023

test_sys.test_intern() fails if run more than once in the same process #108963

Closed

serhiy-storchaka requested a review from vstinner September 17, 2023 08:54

serhiy-storchaka approved these changes Sep 17, 2023

View reviewed changes

bedevere-app bot added awaiting core review and removed awaiting review labels Sep 17, 2023

vstinner reviewed Sep 18, 2023

View reviewed changes

aisk added 2 commits September 19, 2023 19:34

remove float in intern test

1653d05

fix test cases

a586b39

serhiy-storchaka reviewed Sep 19, 2023

View reviewed changes

vstinner reviewed Sep 19, 2023

View reviewed changes

using random.randrange instead of random.randint and increase range

4978df9

vstinner reviewed Sep 21, 2023

View reviewed changes

using 9 digits random range in intern test

785393d

vstinner approved these changes Sep 21, 2023

View reviewed changes

Merge branch 'main' into fix-gh108963

14008d9

vstinner added skip news needs backport to 3.12 bug and security fixes labels Oct 2, 2023

vstinner merged commit 44b1e4e into python:main Oct 2, 2023

bedevere-app bot removed the awaiting core review label Oct 2, 2023

miss-islington assigned vstinner Oct 2, 2023

vstinner pushed a commit to vstinner/cpython that referenced this pull request Oct 2, 2023

pythongh-108963: using random to generate unique string in sys.intern…

52c001d

… test (python#109491) (cherry picked from commit 44b1e4e)

vstinner mentioned this pull request Oct 2, 2023

[3.12] gh-108963: using random to generate unique string in sys.intern test … #110216

Merged

bedevere-app bot removed the needs backport to 3.12 bug and security fixes label Oct 2, 2023

aisk deleted the fix-gh108963 branch October 3, 2023 07:16

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024

pythongh-108963: using random to generate unique string in sys.intern…

34f3941

… test (python#109491)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-108963: using random to generate unique string in sys.intern test #109491

gh-108963: using random to generate unique string in sys.intern test #109491

aisk commented Sep 16, 2023 •

edited by bedevere-app bot

Loading

vstinner commented Sep 18, 2023

vstinner Sep 18, 2023

aisk Sep 19, 2023

serhiy-storchaka Sep 19, 2023 •

edited

Loading

aisk Sep 19, 2023 •

edited

Loading

serhiy-storchaka Sep 19, 2023

vstinner Sep 19, 2023

aisk Sep 19, 2023

vstinner Sep 19, 2023

vstinner Sep 21, 2023

vstinner left a comment

miss-islington commented Oct 2, 2023

miss-islington commented Oct 2, 2023

bedevere-app bot commented Oct 2, 2023

	s = "never interned before" + str(random.randrange(0, 10**18))
	s = "never interned before" + str(random.randrange(10**9))

gh-108963: using random to generate unique string in sys.intern test #109491

gh-108963: using random to generate unique string in sys.intern test #109491

Conversation

aisk commented Sep 16, 2023 • edited by bedevere-app bot Loading

vstinner commented Sep 18, 2023

vstinner Sep 18, 2023

Choose a reason for hiding this comment

aisk Sep 19, 2023

Choose a reason for hiding this comment

serhiy-storchaka Sep 19, 2023 • edited Loading

Choose a reason for hiding this comment

aisk Sep 19, 2023 • edited Loading

Choose a reason for hiding this comment

serhiy-storchaka Sep 19, 2023

Choose a reason for hiding this comment

vstinner Sep 19, 2023

Choose a reason for hiding this comment

aisk Sep 19, 2023

Choose a reason for hiding this comment

vstinner Sep 19, 2023

Choose a reason for hiding this comment

vstinner Sep 21, 2023

Choose a reason for hiding this comment

vstinner left a comment

Choose a reason for hiding this comment

miss-islington commented Oct 2, 2023

miss-islington commented Oct 2, 2023

bedevere-app bot commented Oct 2, 2023

aisk commented Sep 16, 2023 •

edited by bedevere-app bot

Loading

serhiy-storchaka Sep 19, 2023 •

edited

Loading

aisk Sep 19, 2023 •

edited

Loading