Skip to content

Commit

Permalink
Problem: update_allocation stopped on unkown error
Browse files Browse the repository at this point in the history
While creating a VM if any unkown error was raised, update_allocation halted where it was, failing to process the rest of the list

Solution:
Catch all the possible errors

Continuation of ALEPH-337
  • Loading branch information
olethanh committed Jan 15, 2025
1 parent 45ac6c6 commit bb587e0
Showing 1 changed file with 10 additions and 2 deletions.
12 changes: 10 additions & 2 deletions src/aleph/vm/orchestrator/views/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -415,8 +415,12 @@ async def update_allocations(request: web.Request):
vm_hash = ItemHash(vm_hash)
await start_persistent_vm(vm_hash, pubsub, pool)
except vm_creation_exceptions as error:
logger.exception(error)
logger.exception("Error while starting VM '%s': %s", vm_hash, error)
scheduling_errors[vm_hash] = error
except Exception as error:
# Handle unknown exception separately, to avoid leaking data
logger.exception("Unhandled Error while starting VM '%s': %s", vm_hash, error)
scheduling_errors[vm_hash] = Exception("Unhandled Error")

# Schedule the start of instances:
for instance_hash in allocation.instances:
Expand All @@ -425,8 +429,12 @@ async def update_allocations(request: web.Request):
try:
await start_persistent_vm(instance_item_hash, pubsub, pool)
except vm_creation_exceptions as error:
logger.exception(error)
logger.exception("Error while starting VM '%s': %s", instance_hash, error)
scheduling_errors[instance_item_hash] = error
except Exception as error:
# Handle unknown exception separately, to avoid leaking data
logger.exception("Unhandled Error while starting VM '%s': %s", instance_hash, error)
scheduling_errors[vm_hash] = Exception("Unhandled Error")

# Log unsupported features
if allocation.on_demand_vms:
Expand Down

0 comments on commit bb587e0

Please sign in to comment.