Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release dependent modules only after the worker has finished for scheduled modules #39245

Merged
merged 2 commits into from
Sep 23, 2022

Commits on Sep 21, 2022

  1. Configuration menu
    Copy the full SHA
    291b1f4 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2022

  1. Use Worker's WaitingTaskList directly in PuttableProductResolver

    This change was triggered by a case where a PuttableProductResolver
    was filled by a Worker that produced many products, and one of the
    products (A) had a Ref to another product (B), and the product B was
    not consumed by any module (it was only accessed through the Ref).
    Since the putProduct() released the WaitingTaskList of the Resolver,
    that lead to the consumer of A to run, and that consumer dereferenced
    the Ref to see that B was not there.
    
    Besides scheduled modules another cases where PuttableProductResolver
    is used are Sources inheriting PuttableSourceBase, and TestProcessor.
    In these cases the products are put into the Resolvers (or left as
    non-produced) before launching the prefetching of the unscheduled
    system. Therefore in these use cases the consuming modules do not need
    to wait for the Resolver to be filled.
    
    After several fix attempts it seemed easiest to just use the Worker's
    WaitingTaskList directly in PuttableProductResolver. This approach
    fulfills the requirements of both Worker and Source(-like) use cases,
    and even simplifies the code.
    makortel committed Sep 22, 2022
    Configuration menu
    Copy the full SHA
    e8a6e6a View commit details
    Browse the repository at this point in the history