Skip to content

Commit

Permalink
Added set_stacklevel to set the stack level in the dashboard. This in…
Browse files Browse the repository at this point in the history
…fluences what line to display in the 'Invoked on line' section.
  • Loading branch information
sybrenjansen committed Feb 5, 2024
1 parent 08768d9 commit 6942eb8
Show file tree
Hide file tree
Showing 5 changed files with 84 additions and 8 deletions.
6 changes: 6 additions & 0 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
Changelog
=========

Unreleased
----------

* Added :meth:`mpire.dashboard.set_stacklevel` to set the stack level in the dashboard. This influences what line to
display in the 'Invoked on line' section.

2.9.0
-----

Expand Down
4 changes: 4 additions & 0 deletions docs/reference/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@ Dashboard

.. autofunction:: mpire.dashboard.shutdown_dashboard

.. autofunction:: mpire.dashboard.get_stacklevel

.. autofunction:: mpire.dashboard.set_stacklevel


Other
-----
Expand Down
40 changes: 40 additions & 0 deletions docs/usage/dashboard.rst
Original file line number Diff line number Diff line change
Expand Up @@ -139,3 +139,43 @@ In case you have enabled :ref:`worker insights` these insights will be shown rea
Click on the ``Insights (click to expand/collapse)`` to either expand or collapse the insight details.

The dashboard will refresh automatically every 0.5 seconds.


Stack level
-----------

By default, the dashboard will show information about the function that is called and where it is called from. However,
in some cases where you have wrapped the function in another function, you might be less interested in the wrapper
function and more interested in the function that is calling this wrapper. In such cases you can use
:meth:`mpire.dashboard.set_stacklevel` to set the stack level. This is the number of levels in the stack to go back to
find the function that is calling the wrapper function. For example:

.. code-block:: python
from mpire import WorkerPool
from mpire.dashboard import set_stacklevel, start_dashboard
class WorkerPoolWrapper:
def __init__(self, n_jobs, progress_bar=True):
self.n_jobs = n_jobs
self.progress_bar = progress_bar
def __call__(self, func, data):
with WorkerPool(self.n_jobs) as pool:
return pool.map(func, data, progress_bar=self.progress_bar)
def square(x):
return x * x
if __name__ == '__main__':
start_dashboard()
executor = WorkerPoolWrapper(4, progress_bar=True)
set_stacklevel(1) # default
results = executor(square, range(10000))
set_stacklevel(2)
results = executor(square, range(10000))
When you run this code you will see that the dashboard will show two progress bars. In both cases, the dashboard will
show the ``square`` function as the function that is called. However, in the first case, it will show
``return pool.map(func, data, progress_bar=self.progress_bar)`` as the line where it is called from. In the second case,
it will show the ``results = executor(square, range(10000))`` line.
9 changes: 5 additions & 4 deletions mpire/dashboard/__init__.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
try:
from mpire.dashboard.dashboard import connect_to_dashboard, shutdown_dashboard, start_dashboard
from mpire.dashboard.utils import get_stacklevel, set_stacklevel
except (ImportError, ModuleNotFoundError):
def connect_to_dashboard(*_, **__):
raise NotImplementedError("Install the dashboard dependencies to enable the dashboard")

def start_dashboard(*_, **__):
def _not_installed(*_, **__):
raise NotImplementedError("Install the dashboard dependencies to enable the dashboard")

connect_to_dashboard = shutdown_dashboard = start_dashboard = _not_installed
get_stacklevel = set_stacklevel = _not_installed
33 changes: 29 additions & 4 deletions mpire/dashboard/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
from functools import partial
from typing import Callable, Dict, List, Sequence, Tuple, Union

DASHBOARD_FUNCTION_STACKLEVEL = 1


def get_two_available_ports(port_range: Sequence) -> Tuple[int, int]:
"""
Expand Down Expand Up @@ -43,6 +45,25 @@ def _port_available(port_nr: int) -> bool:
raise OSError(f"Dashboard Manager Server: there are not enough ports available: {port_range}")

return tuple(sorted(available_ports))


def get_stacklevel() -> int:
"""
Gets the stack level to use when obtaining function details (used for the dashboard)
:return: Stack level
"""
return DASHBOARD_FUNCTION_STACKLEVEL


def set_stacklevel(stacklevel: int) -> None:
"""
Sets the stack level to use when obtaining function details (used for the dashboard)
:param stacklevel: Stack level
"""
global DASHBOARD_FUNCTION_STACKLEVEL
DASHBOARD_FUNCTION_STACKLEVEL = stacklevel


def get_function_details(func: Callable) -> Dict[str, Union[str, int]]:
Expand All @@ -62,13 +83,17 @@ def get_function_details(func: Callable) -> Dict[str, Union[str, int]]:
the next argument
:return: Function details dictionary
"""
# Get the frame in which the pool.map(...) was called. We obtain the current stack and skip all those which
# involve the current mpire module and stop right after that
# Get the frame in which the pool.map(...) was called. We obtain the current stack and skip all frames which
# involve the current mpire module. If the desired stack level is higher than 1, we continue until we've reached
# the desired stack level. We then obtain the code context of that frame.
invoked_frame = None
stacklevel = 0
for frame_info in inspect.stack():
if frame_info.frame.f_globals['__name__'].split('.')[0] != 'mpire':
if frame_info.frame.f_globals['__name__'].split('.')[0] != 'mpire' or stacklevel > 0:
invoked_frame = frame_info
break
stacklevel += 1
if stacklevel == DASHBOARD_FUNCTION_STACKLEVEL:
break

# Obtain proper code context. Usually the last line of the invoked code is returned, but we want the complete
# code snippet that called this function. That's why we increase the context size and need to find the start and
Expand Down

0 comments on commit 6942eb8

Please sign in to comment.