effortless use of threads

[stonebig]

Hi

Are there some simple links, examples or packages that would greatly help to program in threads for Python:

• with minimal headaches on how-to,
• with minimal (invisible?) programming effort ?

Typical usage:

• exploring threading effort and effect:
  • on some simple game theory algorithms (like sudoku)
  • on some simple operational situations
• exploring 'nogil' work required for simple packages

[colesbury]

I don't have good answers to these questions. One of the easiest ways to use threads in Python is with concurrent.futures.ThreadPoolExecutor. Some problems are easy to parallelize - they can be split into independent pieces, and ThreadPoolExecutor is often a good fit for those problems. Many problems are not easy to parallelize and require more thought and effort.

For something like Sudoku, it's pretty easy to parallelize over multiple games. Here's an example, based on Peter Norvig's Python sudoku solver:
https://github.com/colesbury/sudopy-python3/blob/main/sudopy.py#L165-L166

Parallelizing the search in individual games is much more difficult because it's a depth first search (DFS). There are algorithms for parallel DFS but they're not simple to implement. I'm not sure it would make much sense for Sudoku either because with a fast solver the puzzles are very quick to solve.

[stonebig]

I took solve_all([hard1], "easy", 0) as the measure of Python speed:

• it runs a visual 20-to-80 seconds,
• Cython or PyPy speed it up per 50% only,
• Numba fails on it,
• Python-3.11 didn't have a noticeable effect, on Windows.

... on the un-modifed (not threaded) code on a user PC windows10 , python-3.9.10nogil looks 50% slower than standard python-3.9.10 ... windows or windows defender is not a friend of python speed initiatives

I bloated my pagefile.sys at first try to move the thread split inside "def search(values)". definitely not an easy change to think in threads