Disribution files for the same version may differ api.pulumi.com

[ejiek]

What happened?

I'm sorry for bringing this issue here but it's the best place I've found to report it. It's not a problem with pulumi-aws itself but with a way it's distributed.

NixOS fails to build pulumi-bin v3.112.0 with pulumi-resource-aws v6.28.1

Nix package uses https://api.pulumi.com/releases/plugins/pulumi-resource-aws-v6.28.1-linux-amd64.tar.gz to download the plugin.
We've found that different archive is served for different locations. 2 versions are found so far. Both of them have the same content, but are packaged in a different way.
This leads to different hash sums for the archives which may lead to a nix package build failure due to a checksum mismatch.

Scope might be bigger than nixpkgs since it's not the only place with hash sum validation.

For more details see nixpkgs issue linked in Example Section.

Example

NixOS/nixpkgs#300994

Output of pulumi about

pulumi about is not available since pulumi isn't installed yet.

Affected version of pulumi-resource-aws is v6.28.1 available at https://api.pulumi.com/.

https://api.pulumi.com/releases/plugins/pulumi-resource-aws-v6.28.1-linux-amd64.tar.gz

Additional context

6.28.2 seems to be fine.

My goal with this issue is figuring out why this hash sum drift happened and preventing it in the future.

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[t0yv0]

Hi @ejiek thanks for filing this issue!

The v6.28.1 release was special as our release pipeline was affected by a temporary PyPI outage. As a quick workaround we restarted the failing GitHub action which also reran the provider publish and packaging steps. It would appear that these steps are not idempotent and have last-write-wins semantics, and additionally the packaging is byte-for-byte unstable.

The reference to the event is here (see Attempt pulumi/pulumi-aws#2):

https://github.com/pulumi/pulumi-aws/actions/runs/8461433970

I think this would be good to solve at some point but this is something our team needs to prioritize against other requests. I will move this to the repository that manages build and release infrastructure for pulumi-aws and other providers.

[ejiek]

@t0yv0 thanks for this clarification! It was quick and very detailed.
I do appreciate it =]

Current situation is more than reasonable. v6.28.1 hash mismatch looks like an exception. That means that nixpkgs and others are mostly safe and we can close nixpkgs issue.

Nevertheless, it's possible to build a pipeline that does not lead to this situation. Thanks for moving this issue to the ci repo and leaving it open.

Should we leave a broader problem scope well defined for a future evaluation? Reproducible builds, for example.

[t0yv0]

That's right. It's an exceptional situation where we should do better, but not an immediate blocker hopefully.