Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-109125: Run mypy on Tools/wasm #109126

Merged
merged 5 commits into from
Sep 19, 2023
Merged

gh-109125: Run mypy on Tools/wasm #109126

merged 5 commits into from
Sep 19, 2023

Conversation

sobolevn
Copy link
Member

@sobolevn sobolevn commented Sep 8, 2023

Some comments:

  1. type: ignore can be removed later, when we add these members to typeshed/
  2. I've change some PurePath to Path, because they were using .exists() method, which does not exist on PurePath and goes againts its purpose
  3. I've also change run_py to return int, because that's what similar methods do

Tools/wasm/mypy.ini Show resolved Hide resolved
python_version = 3.8

# Be strict...
strict = True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this mean all code in this directory must be fully typed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can ignore any code that is not typed:

  1. Via adding global disallow_incomplete_defs = false
  2. Via per-file ignores
  3. Via inline # type: ignore

For now, yes, all code must be typed. Do you want to add disallow_incomplete_defs = false?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to add disallow_incomplete_defs = false?

Yes please.

Copy link
Member

@brettcannon brettcannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing my review to make type checking opt-in per-file, but other LGTM.

@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@sobolevn
Copy link
Member Author

Done! I have made the requested changes; please review again

@bedevere-bot
Copy link

Thanks for making the requested changes!

@brettcannon: please review the changes made to this pull request.

@sobolevn
Copy link
Member Author

@brettcannon updated branch to fix freebsd unrelated failire

Copy link
Member

@hugovk hugovk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And backport to 3.12?

(We don't yet have .github/workflows/mypy.yml in 3.11.)

.github/workflows/mypy.yml Outdated Show resolved Hide resolved
@hugovk hugovk added the needs backport to 3.12 bug and security fixes label Sep 19, 2023
@hugovk hugovk merged commit f65497f into python:main Sep 19, 2023
25 checks passed
@miss-islington
Copy link
Contributor

Thanks @sobolevn for the PR, and @hugovk for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@miss-islington
Copy link
Contributor

Sorry, @sobolevn and @hugovk, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker f65497fd252a4a4df960da04d68e8316b58624c0 3.12

hugovk pushed a commit to hugovk/cpython that referenced this pull request Sep 19, 2023
(cherry picked from commit f65497f)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
@bedevere-app
Copy link

bedevere-app bot commented Sep 19, 2023

GH-109561 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 bug and security fixes label Sep 19, 2023
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian PGO 3.x has failed when building commit f65497f.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/249/builds/6370) and take a look at the build logs.
  4. Check if the failure is related to this commit (f65497f) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/249/builds/6370

Failed tests:

  • test.test_multiprocessing_fork.test_processes

Summary of the results of the build (if available):

==

Click to see traceback logs
remote: Enumerating objects: 20, done.        
remote: Counting objects:   5% (1/20)        
remote: Counting objects:  10% (2/20)        
remote: Counting objects:  15% (3/20)        
remote: Counting objects:  20% (4/20)        
remote: Counting objects:  25% (5/20)        
remote: Counting objects:  30% (6/20)        
remote: Counting objects:  35% (7/20)        
remote: Counting objects:  40% (8/20)        
remote: Counting objects:  45% (9/20)        
remote: Counting objects:  50% (10/20)        
remote: Counting objects:  55% (11/20)        
remote: Counting objects:  60% (12/20)        
remote: Counting objects:  65% (13/20)        
remote: Counting objects:  70% (14/20)        
remote: Counting objects:  75% (15/20)        
remote: Counting objects:  80% (16/20)        
remote: Counting objects:  85% (17/20)        
remote: Counting objects:  90% (18/20)        
remote: Counting objects:  95% (19/20)        
remote: Counting objects: 100% (20/20)        
remote: Counting objects: 100% (20/20), done.        
remote: Compressing objects:   9% (1/11)        
remote: Compressing objects:  18% (2/11)        
remote: Compressing objects:  27% (3/11)        
remote: Compressing objects:  36% (4/11)        
remote: Compressing objects:  45% (5/11)        
remote: Compressing objects:  54% (6/11)        
remote: Compressing objects:  63% (7/11)        
remote: Compressing objects:  72% (8/11)        
remote: Compressing objects:  81% (9/11)        
remote: Compressing objects:  90% (10/11)        
remote: Compressing objects: 100% (11/11)        
remote: Compressing objects: 100% (11/11), done.        
remote: Total 11 (delta 9), reused 0 (delta 0), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to 'f65497fd252a4a4df960da04d68e8316b58624c0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at f65497fd25 gh-109125: Run mypy on `Tools/wasm` (#109126)
Switched to and reset branch 'main'

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make[2]: [Makefile:2815: clean-retain-profile] Error 1 (ignored)
./Modules/readline.c: In function ‘setup_readline’:
./Modules/readline.c:1256:21: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types]
 1256 |     rl_startup_hook = on_startup_hook;
      |                     ^
./Modules/readline.c:1258:23: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types]
 1258 |     rl_pre_input_hook = on_pre_input_hook;
      |                       ^
./Modules/socketmodule.c: In function ‘getsockaddrarg’:
./Modules/socketmodule.c:2468:9: warning: ‘strncpy’ specified bound 64 equals destination size [-Wstringop-truncation]
 2468 |         strncpy((char *)sa->salg_name, name, sizeof(sa->salg_name));
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Modules/_decimal/libmpdec/context.c:56: warning: mpd_setminalloc: ignoring request to set MPD_MINALLOC a second time

./Modules/_decimal/libmpdec/context.c:56: warning: mpd_setminalloc: ignoring request to set MPD_MINALLOC a second time

./Modules/readline.c: In function ‘setup_readline’:
./Modules/readline.c:1256:21: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types]
 1256 |     rl_startup_hook = on_startup_hook;
      |                     ^
./Modules/readline.c:1258:23: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types]
 1258 |     rl_pre_input_hook = on_pre_input_hook;
      |                       ^
In function ‘utf8_toUtf8’,
    inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
    inlined from ‘doParseXmlDecl’ at ./Modules/expat/xmltok.c:1197:13:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
  390 |     memcpy(*toP, *fromP, bytesToCopy);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘utf8_toUtf8’,
    inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
    inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1075:9:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
  390 |     memcpy(*toP, *fromP, bytesToCopy);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘utf8_toUtf8’,
    inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
    inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1081:12:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
  390 |     memcpy(*toP, *fromP, bytesToCopy);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘utf8_toUtf8’,
    inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
    inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1088:9:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
  390 |     memcpy(*toP, *fromP, bytesToCopy);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
In function ‘utf8_toUtf8’,
    inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
    inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1115:7:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
  390 |     memcpy(*toP, *fromP, bytesToCopy);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘utf8_toUtf8’,
    inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3,
    inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1128:9:
./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=]
  390 |     memcpy(*toP, *fromP, bytesToCopy);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Kill <WorkerThread #2 running test=test_perf_profiler pid=1776311 time=201 ms> process group
make: *** [Makefile:2035: buildbottest] Error 5

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x has failed when building commit f65497f.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/345/builds/5863) and take a look at the build logs.
  4. Check if the failure is related to this commit (f65497f) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/345/builds/5863

Failed tests:

  • test.test_multiprocessing_forkserver.test_processes

Summary of the results of the build (if available):

==

Click to see traceback logs
remote: Enumerating objects: 20, done.        
remote: Counting objects:   5% (1/20)        
remote: Counting objects:  10% (2/20)        
remote: Counting objects:  15% (3/20)        
remote: Counting objects:  20% (4/20)        
remote: Counting objects:  25% (5/20)        
remote: Counting objects:  30% (6/20)        
remote: Counting objects:  35% (7/20)        
remote: Counting objects:  40% (8/20)        
remote: Counting objects:  45% (9/20)        
remote: Counting objects:  50% (10/20)        
remote: Counting objects:  55% (11/20)        
remote: Counting objects:  60% (12/20)        
remote: Counting objects:  65% (13/20)        
remote: Counting objects:  70% (14/20)        
remote: Counting objects:  75% (15/20)        
remote: Counting objects:  80% (16/20)        
remote: Counting objects:  85% (17/20)        
remote: Counting objects:  90% (18/20)        
remote: Counting objects:  95% (19/20)        
remote: Counting objects: 100% (20/20)        
remote: Counting objects: 100% (20/20), done.        
remote: Compressing objects:   9% (1/11)        
remote: Compressing objects:  18% (2/11)        
remote: Compressing objects:  27% (3/11)        
remote: Compressing objects:  36% (4/11)        
remote: Compressing objects:  45% (5/11)        
remote: Compressing objects:  54% (6/11)        
remote: Compressing objects:  63% (7/11)        
remote: Compressing objects:  72% (8/11)        
remote: Compressing objects:  81% (9/11)        
remote: Compressing objects:  90% (10/11)        
remote: Compressing objects: 100% (11/11)        
remote: Compressing objects: 100% (11/11), done.        
remote: Total 11 (delta 9), reused 1 (delta 0), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to 'f65497fd252a4a4df960da04d68e8316b58624c0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at f65497fd25 gh-109125: Run mypy on `Tools/wasm` (#109126)
Switched to and reset branch 'main'

configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

make: *** [Makefile:2029: buildbottest] Error 5

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable Clang 3.x has failed when building commit f65497f.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/435/builds/3628) and take a look at the build logs.
  4. Check if the failure is related to this commit (f65497f) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/435/builds/3628

Failed tests:

  • test.test_concurrent_futures.test_shutdown

Failed subtests:

  • test_interpreter_shutdown - test.test_concurrent_futures.test_shutdown.ProcessPoolSpawnProcessPoolShutdownTest.test_interpreter_shutdown

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/test/test_concurrent_futures/test_shutdown.py", line 49, in test_interpreter_shutdown
    self.assertFalse(err)
AssertionError: b'Exception in thread Thread-1:\nTraceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/threading.py", line 1059, in _bootstrap_inner\n    self.run()\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/concurrent/futures/process.py", line 339, in run\n    self.add_call_item_to_queue()\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/concurrent/futures/process.py", line 394, in add_call_item_to_queue\n    self.call_queue.put(_CallItem(work_id,\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/multiprocessing/queues.py", line 94, in put\n    self._start_thread()\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/multiprocessing/queues.py", line 177, in _start_thread\n    self._thread.start()\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/threading.py", line 978, in start\n    _start_new_thread(self._bootstrap, ())\nRuntimeError: can\'t create new thread at interpreter shutdown\nTraceback (most recent call last):\n  File "<string>", line 1, in <module>\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/multiprocessing/spawn.py", line 122, in spawn_main\n    exitcode = _main(fd, parent_sentinel)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/multiprocessing/spawn.py", line 132, in _main\n    self = reduction.pickle.load(from_parent)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/multiprocessing/synchronize.py", line 115, in __setstate__\n    self._semlock = _multiprocessing.SemLock._rebuild(*state)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n' is not false

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL8 LTO 3.x has failed when building commit f65497f.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/361/builds/4069) and take a look at the build logs.
  4. Check if the failure is related to this commit (f65497f) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/361/builds/4069

Failed tests:

  • test.test_multiprocessing_fork.test_misc

Failed subtests:

  • test_nested_startmethod - test.test_multiprocessing_fork.test_misc.TestStartMethod.test_nested_startmethod

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/test/_test_multiprocessing.py", line 5475, in test_nested_startmethod
    self.assertEqual(results, [2, 1])
AssertionError: Lists differ: [1, 2] != [2, 1]

csm10495 pushed a commit to csm10495/cpython that referenced this pull request Sep 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants