Context distiller puts out no output when one resource fails

[flowerncsu]

Two pinfiles tested:

---
pass_aws:
  topology: topology-0.yml

and

---
pass_aws:
  topology: topology-0.yml
fail_aws:
  topology: fail_aws.yml

topology-0 successfully creates an aws instance. fail_aws is:

---
topology_name: ec2-new
resource_groups:
  - resource_group_name: "aws"
    resource_group_type: "aws"
    resource_definitions:
      - name: "ci-testbox"
        flavor: "m1.small"
        role: "aws_ec2"

        region: "us-east-1"
        image: "not-an-image"
        security_group: "default"
        count: 1

When using the PinFile which includes fail_aws, the linchpin.distilled file is not created, regardless of whether there was a previous linchpin.distilled. When using the PinFile without fail_aws, linchpin.distilled is created if there is not one, and overwritten if there is (which is the desired behavior).

[herlo]

@flowerncsu After our discussion, I think there's a simple solution that could work for this scenario. One caveat being that I don't know how the beaker solution will be affected because of the tooling created to clean up resources if a topology fails.

The solution would be to modify the linchpin.conf and include a flag called something like 'ignore_return_code = True' (It will be set to false by default). This flag would then, for now, only be used with the Context Distiller, and would filter out any RunDB data which has a return_code != 0.

In this way, only successful instances (aws_ec2, os_server, specifically) would be distilled. Additional testing may be required for beaker, for reasons stated above.

What do you think? Would you be willing to test out the beaker bits once it's written and verify that it works the way you need?

[flowerncsu]

It seems like getting info for whatever context exists should really be the default behavior. If there's a flag needed, I would think the flag would be in the opposite direction (ie, a flag to not distill output if there are any failures) but I can't think of a reason someone would want to discard potential output just because there are pieces of it not available.

Also, I'd like to discuss more about the beaker issue. I definitely would not have expected that the system would tear down successful builds just because an unsuccessful build happened. If I encountered that behavior, I would very definitely think it was a bug, and if that behavior is needed, I think that should also be a flag that can be set in the configuration, rather than default.

[herlo]

@flowerncsu, In terms of the return_code discussion and whether we should output the data no matter if the other targets fail, I don't disagree with you. I do have some concerns on how and where the line is drawn.

One scenario I considered is, what if there are two or more of the same resources being provisioned and one of the two fails? LinchPin currently fails that job, even if one of the two is successful, because it's done within one ansible run.

Another scenario would be to have two different resources in a resource group within a single target. Or even more crazy, two separate resource groups each with a single resource within a single target. In each case, only one resource fails.

If we can address these scenarios by saying that if anything in a individual target fails, the whole target fails, this may be rectified simply enough. If not, and the desire is for any successful resource data within any target to be recorded, it's a bit more challenging and will require more research and time.

For beaker resources, it's very similar to the above issue. I found the issue requesting the fix. It's #394 filed by @jaypoulz. I didn't implement the code to enable the recall of instances, that credit goes to @seandst with #468. It may be useful to point out that testing of these features may reveal something useful, but I have not been able to test more than what is in the current bkr-new.yml topology file.

It may additionally be useful to have a discussion with them to see if there's a way to modify the behavior with options in the topology. I am not sure without some digging whether or not that exists.

I'd like to hear your opinion on this issue.

[flowerncsu]

It's sufficient for my current needs to simply include context for any successful target, and ignore (or mark as failed) any failed target. However, it may be worth further discussion to see if mixed-state targets (partially passing) might need to be tracked