Skip to content

Commit

Permalink
test set_spawn_method and handle exceptions
Browse files Browse the repository at this point in the history
  • Loading branch information
Richard Kuo (Danswer) committed Jan 10, 2025
1 parent 2163a13 commit 384a384
Showing 1 changed file with 11 additions and 1 deletion.
12 changes: 11 additions & 1 deletion backend/onyx/background/celery/apps/heavy.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,17 @@ def on_worker_init(sender: Any, **kwargs: Any) -> None:
all_start_methods: list[str] = multiprocessing.get_all_start_methods()
logger.info(f"Multiprocessing all start methods: {all_start_methods}")

multiprocessing.set_start_method("spawn") # fork is unsafe, set to spawn
try:
multiprocessing.set_start_method("spawn") # fork is unsafe, set to spawn
except Exception:
logger.info("multiprocessing.set_start_method exceptioned.")
try:
multiprocessing.set_start_method(
"spawn", force=True
) # fork is unsafe, set to spawn
except Exception:
logger.info("multiprocessing.set_start_method force=True exceptioned.")

logger.info(
f"Multiprocessing selected start method: {multiprocessing.get_start_method()}"
)
Expand Down

0 comments on commit 384a384

Please sign in to comment.