Commits on Nov 1, 2022

1. providercache: Ignore lock-mismatching global cache entries …

   When we originally introduced the trust-on-first-use checksum locking
   mechanism in v0.14, we had to make some tricky decisions about how it
   should interact with the pre-existing optional read-through global cache
   of provider packages:
   
   The global cache essentially conflicts with the checksum locking because
   if the needed provider is already in the cache then Terraform skips
   installing the provider from upstream and therefore misses the opportunity
   to capture the signed checksums published by the provider developer. We
   can't use the signed checksums to verify a cache entry because the origin
   registry protocol is still using the legacy ziphash scheme and that is
   only usable for the original zipped provider packages and not for the
   unpacked-layout cache directory. Therefore we decided to prioritize the
   existing cache directory behavior at the expense of the lock file behavior,
   making Terraform produce an incomplete lock file in that case.
   
   Now that we've had some real-world experience with the lock file mechanism,
   we can see that the chosen compromise was not ideal because it causes
   "terraform init" to behave significantly differently in its lock file
   update behavior depending on whether or not a particular provider is
   already cached. By robbing Terraform of its opportunity to fetch the
   official checksums, Terraform must generate a lock file that is inherently
   non-portable, which is problematic for any team which works with the same
   Terraform configuration on multiple different platforms.
   
   This change addresses that problem by essentially flipping the decision so
   that we'll prioritize the lock file behavior over the provider cache
   behavior. Now a global cache entry is eligible for use if and only if the
   lock file already contains a checksum that matches the cache entry. This
   means that the first time a particular configuration sees a new provider
   it will always be fetched from the configured installation source
   (typically the origin registry) and record the checksums from that source.
   
   On subsequent installs of the same provider version already locked,
   Terraform will then consider the cache entry to be eligible and skip
   re-downloading the same package.
   
   This intentionally makes the global cache mechanism subordinate to the
   lock file mechanism: the lock file must be populated in order for the
   global cache to be effective. For those who have many separate
   configurations which all refer to the same provider version, they will
   need to re-download the provider once for each configuration in order to
   gather the information needed to populate the lock file, whereas before
   they would have only downloaded it for the _first_ configuration using
   that provider.
   
   This should therefore remove the most significant cause of folks ending
   up with incomplete lock files that don't work for colleagues using other
   platforms, and the expense of bypassing the cache for the first use of
   each new package with each new configuration. This tradeoff seems
   reasonable because otherwise such users would inevitably need to run
   "terraform providers lock" separately anyway, and that command _always_
   bypasses the cache. Although this change does decrease the hit rate of the
   cache, if we subtract the never-cached downloads caused by
   "terraform providers lock" then this is a net benefit overall, and does
   the right thing by default without the need to run a separate command.

   

   apparentlymart committed Nov 1, 2022
   Configuration menu

   • View commit details
   • Copy full SHA for 581f0b9
   • Browse repository at this point

   Copy the full SHA

   581f0b9 View commit details

   Browse the repository at this point in the history