MacOS support

.github/workflows/python-package.yml

@@ -15,7 +15,7 @@ jobs:
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        os: [ubuntu-20.04, windows-latest]
+        os: [ubuntu-20.04, windows-latest, macos-latest]
         python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
 
     steps:
@@ -31,6 +31,11 @@ jobs:
         pip install .[dashboard]
         pip install .[dill]
         pip install .[testing]
+    - name: Set ulimit for macOS
+      if: runner.os == 'macos-latest'
+      run: |
+        ulimit -a
+        ulimit -n 1024
     - name: Lint with flake8
       run: |
         # stop the build if there are Python syntax errors or undefined names

.gitignore

@@ -5,4 +5,7 @@ _build
 dist
 *.egg-info
 .eggs
+.pytest_cache
 
+# MacOS
+.DS_store

README.rst

@@ -49,7 +49,8 @@ Features
 - Optionally utilizes dill_ as serialization backend through multiprocess_, enabling parallelizing more exotic objects,
   lambdas, and functions in iPython and Jupyter notebooks.
 
-MPIRE is tested on Linux and Windows. For Windows and macOS users, there are a few minor known caveats, which are documented in the Troubleshooting_ chapter.
+MPIRE is tested on Linux, macOS, and Windows. For Windows and macOS users, there are a few minor known caveats, which 
+are documented in the Troubleshooting_ chapter.
 
 .. _benchmarks: https://towardsdatascience.com/mpire-for-python-multiprocessing-is-really-easy-d2ae7999a3e9
 .. _multiprocess: https://github.com/uqfoundation/multiprocess

docs/changelog.rst

@@ -2,7 +2,13 @@ Changelog
 =========
 
 Unreleased
-----------
+==========
+
+* Added support for macOS (`#27`_, `#79`_, `#91`_)
+
+  - Fixes memory leaks on macOS
+  - Reduced the amount of semaphores used
+  - Issues a warning when ``cpu_ids`` is used on macOS
 
 * Added :meth:`mpire.dashboard.set_stacklevel` to set the stack level in the dashboard. This influences what line to 
   display in the 'Invoked on line' section. (`#118`_)
@@ -12,9 +18,13 @@ Unreleased
   (`#117`_)
 * Make it possible to reuse the same `progress_bar_options` without raising warnings (`#117`_)
 
+.. _#27: https://github.com/sybrenjansen/mpire/issues/27
+.. _#79: https://github.com/sybrenjansen/mpire/issues/79
+.. _#91: https://github.com/sybrenjansen/mpire/issues/91
 .. _#117: https://github.com/sybrenjansen/mpire/pull/117
 .. _#118: https://github.com/sybrenjansen/mpire/pull/118
 
+
 2.9.0
 -----
 

docs/index.rst

@@ -32,8 +32,8 @@ Features
 - Optionally utilizes dill_ as serialization backend through multiprocess_, enabling parallelizing more exotic objects,
   lambdas, and functions in iPython and Jupyter notebooks.
 
-MPIRE has been tested on both Linux and Windows. There are a few minor known caveats for Windows users, which can be
-found at :ref:`troubleshooting_windows`.
+MPIRE has been tested on Linux, macOS, and Windows. There are a few minor known caveats for Windows and macOS users, 
+which can be found at :ref:`troubleshooting_windows`.
 
 .. _benchmarks: https://towardsdatascience.com/mpire-for-python-multiprocessing-is-really-easy-d2ae7999a3e9
 .. _dill: https://pypi.org/project/dill/

docs/troubleshooting.rst

@@ -149,8 +149,6 @@ define your function in a file that can be imported by the child process, or you
 Windows
 -------
 
-Windows support has some caveats:
-
 * When using ``dill`` and an exception occurs, or when the exception occurs in an exit function, it can print additional
   ``OSError`` messages in the terminal, but they can be safely ignored.
 * The ``mpire-dashboard`` script does not work on Windows.
@@ -161,8 +159,9 @@ Windows support has some caveats:
 macOS
 -----
 
-macOS is not yet officially supported. It seems to work mostly fine, with these caveats:
-
-* Use ``ulimit -n <number>`` to increase the limit of the number of open files.
-  This is required because mpire uses a lot of file-descriptor based synchronization primitives.
-  `<Number>` should be large enough to accomodate all of mpire's file descriptors, eg. `<number> = 65536`.
+* When encountering ``OSError: [Errno 24] Too many open files`` errors, use ``ulimit -n <number>`` to increase the 
+  limit of the number of open files. This is required because MPIRE uses file-descriptor based synchronization 
+  primitives and macOS has a very low default limit. For example, MPIRE uses about 190 file descriptors when using 10
+  workers. 
+* Pinning of processes to CPU cores is not supported on macOS. This is because macOS does not support the 
+  ``sched_setaffinity`` system call. A warning will be printed when trying to use this feature.

docs/usage/workerpool/cpu_pinning.rst

@@ -31,4 +31,4 @@ CPU IDs have to be positive integers, not exceeding the number of CPUs available
 
 .. note::
 
-    Pinning processes to CPU IDs doesn't work when using threading.
+    Pinning processes to CPU IDs doesn't work when using threading or when you're on macOS.