After updating with "clink update" you still see an update when you run "winget upgrade"

[MagicAndre1981]

After updating with clink update you still see an update when you run winget upgrade:

Microsoft Windows [Version 10.0.17763.3046]
(c) 2018 Microsoft Corporation. Alle Rechte vorbehalten.

Clink v1.3.25.080a5d
Copyright (c) 2012-2018 Martin Ridgers
Portions Copyright (c) 2020-2022 Christopher Antos
https://github.com/chrisant996/clink

C:\WINDOWS\system32>clink update
Checking latest version...
No update available; local version v1.3.25 is not older than latest release v1.3.25.

C:\WINDOWS\system32>winget upgrade
Name   ID                Version       Verfügbar     Quelle
-----------------------------------------------------------
Clink  chrisant996.Clink 1.3.23.5d5aed 1.3.25.080a5d winget

So maybe it is better to download the installer (not simply unzip the new files) and run the same silent mode like when doing an upgrade with winget.

Originally posted by @MagicAndre1981 in #303 (comment)

[chrisant996]

I do understand the problem.

But the suggested solution of always forcing install via the installer would create many problems. What about when people installed Clink from a zip file in the first place? What about when there are multiple installs of Clink on the same computer? And so on.

These are the kinds of reasons why I had not created an update command yet.

So, no, that's not a solution at all.

What I can potentially do is make the updater check if the setup exe has ever been used on the computer, and whether the copy that's being updated is the same location where the setup exe installed it, and if so then also update registry entries about what version was installed. If winget is using the setup exe's registry keys to figure out the version -- versus winget keeping track of the installed version and assuming that only winget installs updates.

May I ask, why would someone try to update Clink using multiple mechanisms? Why not turn off Clink's updater, if an external updater is preferred?

[chrisant996]

Anti-malware protection blocks launching the downloaded setup exe.

I think the best solution is this: If you're going to use an external updater program, then don't use Clink's built in updater.

[MagicAndre1981]

ok, I was a bit confused.

I run this winget upgrade from time to time to see if I missed an update to a software.

[chrisant996]

Furthermore, it seems that winget is looking at its own internal database of what it installed.

The most that Clink can do is overwrite the DisplayVersion regkey.

[MagicAndre1981]

The most that Clink can do is overwrite the DisplayVersion regkey.

will this prevent showing an clink update in winget after doing the intool update?

[chrisant996]

The most that Clink can do is overwrite the DisplayVersion regkey.

will this prevent showing an clink update in winget after doing the intool update?

Maybe. I can't predict one way or the other. It would depend on how winget determines what versions it thinks are installed.

[MagicAndre1981]

It would depend on how winget determines what versions it thinks are installed.

I found this statement:

We are depending on data available in Windows Apps & Features to match a manifest based on the "displayName" and "version".

[chrisant996]

That is non-specific and doesn't clarify where winget looks, what API calls it uses, what information source the data comes from, nor how that information source gets the relevant data stored into it.

But I should be clear:

I have no intention of trying to "support" winget if it uses a custom database or if it looks at anything other than regkeys directly set by the Clink setup program. If it does, then it'll work. If it doesn't, then Clink can't do anything about that.

And it doesn't really make sense to be trying to use multiple different updaters for the same program. Pick one. 🙃

[denelon]

@chrisant996

There is no custom database for determining which version of a package is installed. It's all driven by registry keys.

There is a database to record packages that have been installed, but it's purely for attempting to improve correlation between installed packages and manifests available in configured sources. The Windows Package Manager doesn't assume the package hasn't been modified with some other mechanism.

With a cursory glance at the versioning scheme used here, there may be challenges determining if an upgrade is available if the fourth position isn't necessarily incremental. I've seen several examples where commit hashes or some other non-sequential value is a part of a version number. I certainly didn't dig in and look, it's just a superficial observation.

[chrisant996]

@denelon Thanks for the confirmation!

I think you’re saying that winget list looks at the DisplayVersion value under the product's regkey under hklm\software\wow6432node\microsoft\windows\currentversion\uninstall, right? (It was hard to find where the data is collected from, using source inspection in the winget-cli repo, and I hadn't gotten around yet to building it and stepping thru in the debugger.)

The original author of Clink embedded a commit hash in the fourth value, and I haven’t removed it yet because I don’t know all the potential side effects. He’d been using it for several purposes, including attempting to work around a race condition in the NSIS installer. The first time I removed the hash from someplace, it caused a regression. I’m cautious about removing it from other places, now. 😉

For builds of Clink published on github, the hash is never a functional part of the version. I.e. there will never be two version numbers that differ only by hash.

So far, winget hasn’t seemed to have any problems comparing version numbers for Clink, probably because the hashes aren't a functional "identifier" part of the version number.

[denelon]

@chrisant996

My pleasure. Yes, you are correct. I totally get the commit hash as a part of the identifier and as long as the hashes aren't a functional part of the version number, there shouldn't be any conflict to resolve.

[chrisant996]

@denelon apparently as reported in #434, winget does not successfully compare version numbers for Clink, at least not anymore.

Clink publishes its version numbers as Major.Minor.Patch. In the filenames of the zip and exe installers, it also includes a fourth .CommitHash component so that it's easy to differentiate unreleased files versus released files (they'll have different commit hashes).

It seems something in winget is presumably parsing the file names, and it is inaccurately thinking that the published version number is Major.Minor.Patch.CommitHash. So when winget compares that interpretation of the version number versus the actual version number that Clink reports having installed, it apparently always thinks an update is available.

I'm not sure how to change the automated winget stuff to parse the version correctly. Any thoughts on how to fix how winget comes up with the available version number?

P.S. I'm pretty sure I didn't add Clink into winget, and it looks as though something is automatically generating manifests. Interestingly, it seems to have gotten the version number correct for 1.4.18, but all other versions seem to be wrong.

[denelon]

@vedantmgoyal2009 can you take a look at this?

@chrisant996 there are a few options for maintaining and publishing the manifest yourself as well.

PowerToys and Oh My Posh have GitHub actions using wingetcreate.

[Trenly]

It looks like part of the problem is that there were incorrect Product Code mappings in many of the manifests over on winget-pkgs. I've submitted PR's to fix that issue.

The other issue is that clink update and the clink installer write different versions to the DisplayVersion key. When trying to determine which packages are installed, the versions are compared as strings for equality. So, if a user installed clink v1.4.20 using the installer, they would see 1.4.20.236aba as the version in control panel. So, Winget looks for 1.4.20.236aba as the version to see if it was installed. But, this means that if a user installs v1.4.20 using clink upgrade, they would see 1.4.20 in control panel. This means that Winget can't map the version string into a specific version, and so it has to rely on other install data to figure it out, which leads to the third issue, which is actually the root cause.

Primary Issue: clink upgrade doesn't update the product code

Product code is considered to be one of the best heuristics for determining which application is installed. If there is a version that can't be mapped - product codes can determine at least which Package it is. If there is a DisplayName that doesn't quite match the metadata winget knows about - Product codes are used to create certainty.

When using the installer, the product code changes for each version, and so the metadata in winget reflects that the product code depends on the installer. But, clink upgrade just overwrites the existing registry entry instead of creating a new one and deleting the old one. Since product codes are such a distinctive factor, and the version isn't a perfect match (as described above), Winget sees that the product code is still (for example) clink_1.4.18.792517 - since there isn't any other package & version out there with that exact product code, winget believes that it is actually Clink 1.4.18 that is installed.

There are a few things that would need to be done in order for winget to be able to properly match in all cases. To avoid major changes to clink update, I think this can all be done with modifications just to the installer (NSI) side.

1. Change the NSI file to only write the 3-part version in the registry keys. This shouldn't break anything if you add a new variable like CLINK_VERSION_TAG, calculate it in the premake5.lua file, then pass it using /DCLINK_VERSION_TAG, and then only use it for Lines 153 and 160 in clink.nsi where the regkeys are actually written

2. Change the installer to always use the same product code. This is as simple as removing _${CLINK_VERSION} from line 152 of clink.nsi. This shouldn't break anything, as the install directory doesn't contain the version in the name.

---

It seems something in winget is presumably parsing the file names, and it is inaccurately thinking that the published version number is Major.Minor.Patch.CommitHash.

I believe that Vedant's automation includes the CommitHash as part of the version is because that is what is written to the DisplayVersion key of the registry when the installer is used

So when winget compares that interpretation of the version number versus the actual version number that Clink reports having installed, it apparently always thinks an update is available.

I don't think this is actually the case. Winget parses versions left to right in determining which version is greater than others, using . as the split character to break the version into "parts". If the first part of A is equal to the first part of B, it then compares the second, third, and so forth. In the case of Clink, since the third "part" is the one which is incremented for releases, the CommitHash part is essentially ignored for the purposes of determining which version is higher. See my description of the product code interaction above, which I think is the actual cause of the mismatch.

I'm not sure how to change the automated winget stuff to parse the version correctly. Any thoughts on how to fix how winget comes up with the available version number?

See recommendations above

[chrisant996]

Thanks, @Trenly, I'll independently confirm your observations and report back. Offhand they look both accurate and highly informative and helpful. I was unaware that a "product code" even existed.

I'm happy to modify clink update and/or the installer -- I want them to be doing the same thing, regardless what that thing is.

There is long-standing precedent for being able to have multiple different versions of Clink installed side-by-side. I certainly could remove that ability and use a single product code. But I worry that might break things for some people.

It seems that the SxS ability was introduced as a side effect of trying to work around a race condition in the installer due to a bug in the .nsi file. But regardless how it came into being, the SxS ability exists. Removing the ability opens up regression risk for some users.

EDIT: ...oh but winget doesn't really understand/support SxS does it? Interesting...

So, I will also investigate making changes to clink update.

Thanks again, this was very helpful and educational for me! ❤

[chrisant996]

@chrisant996 there are a few options for maintaining and publishing the manifest yourself as well.

PowerToys and Oh My Posh have GitHub actions using wingetcreate.

Thanks, @denelon -- I like that the PowerToys one uses .Trim to transform the release tag. I'll model mine after that version. Very helpful, thank you!

[Trenly]

@chrisant996 - I wasn't able to get SxS working on my end, either by using clink update or the installers themselves. Is there a guide on how to do this?

Using the installer, this section will always uninstall previous versions anyways -

clink/installer/clink.nsi

Lines 94 to 118 in 236abad

	Function cleanPreviousInstalls
	StrCpy $0 "Software\Microsoft\Windows\CurrentVersion\Uninstall"
	StrCpy $1 0
	EnumUninstallKeysLoop:
	EnumRegKey $2 HKLM $0 $1
	StrCmp $2 "" EnumUninstallKeysEnd
	; Skip installs of ourself over an existing installation.
	;
	StrCmp $2 "clink_${CLINK_VERSION}" EndIfClinkUninstallEntry 0
	; Check for uninstaller entries that start "clink_"
	;
	StrCpy $3 $2 6
	StrCmp $3 "clink_" 0 EndIfClinkUninstallEntry
	ReadRegStr $4 HKLM "$0\$2" "UninstallString"
	${GetParent} $4 $5
	ExecWait '"$4" /S _?=$5'
	Delete $4
	DeleteRegKey HKLM "$0\$2"
	EndIfClinkUninstallEntry:
	IntOp $1 $1 + 1
	Goto EnumUninstallKeysLoop
	EnumUninstallKeysEnd:
	FunctionEnd

[chrisant996]

@Trenly Yes, you're right -- the installer has never actually supported SxS.

When "installing" manually by extracting from a .zip file, it has always been possible to have multiple versions of Clink installed.
And clink update will update each of the separate copies. I will have to double check, but I think (and hope...) I made clink update smart enough to only update the registry when upgrading the same target directory as the version in the registry.

So maybe this will all be very easy and straightforward to clean up. 🤞

And denelon pointed me to the PowerToys workflow for updating winget, and I'll hook up a variation of that for Clink.

Do you happen to know whether I need to do anything additional to make the bot stop auto-creating updated packages for Clink?

[Trenly]

Do you happen to know whether I need to do anything additional to make the bot stop auto-creating updated packages for Clink?

I'll submit a PR over on Vedant's automation

[vedantmgoyal9]

I believe that Vedant's automation includes the CommitHash as part of the version is because that is what is written to the DisplayVersion key of the registry when the installer is used

Well, I guess I'm too late here, and Trenly already gave the answer 😅

@chrisant996 there are a few options for maintaining and publishing the manifest yourself as well.

(although, it is highly unlikely that one will prefer a community solution over an official solution, still posting it here 😶‍🌫️)

WinGet Releaser, a GitHub Action, also helps you to publish new releases to WinGet.

[denelon]

Options are always good @vedantmgoyal2009 😊

[chrisant996]

@Trenly How did you determine that clink update was the main problem?

If the current Clink being used was installed via the .exe installer, then clink update should be downloading a new .exe installer and running the .exe installer to perform the update. It relies on the .exe installer to do all the work.

Did you find a repro where Clink was installed by the .exe installer, but clink update got confused into trying to update by using a .zip file?

Line 711 gets the current installation type. When installed via the .exe installer, it returns "exe". In my testing, that seems to be working correctly.

Line 718 refuses to continue unless the downloaded update type (zip or exe) matches the current installation type (zip or exe).

Lines 722-723 run the .exe installer to perform an update if Clink was installed via the .exe installer.

(I'm actually not sure why the .zip updater is fiddling with registry keys at all... I think that's a carryover from before it knew how to download+run the .exe installer.)

clink/clink/app/scripts/update.lua

Lines 711 to 726 in 3563f62

	local install_type = get_installation_type()
	if not elevated and install_type == "zip" and this_install_type == "exe" then
	log_info("elevation required to update InstallDir in registry.")
	return -1
	end
	local ext = path.getextension(update_file):lower():match("^%.(.+)$")
	if ext ~= install_type then
	return nil, log_info(string.format("mismatched update type (%s) and update file (%s).", install_type, ext))
	elseif ext == "zip" then
	return apply_zip_update(update_file, true)
	elseif ext == "exe" then
	return run_exe_installer(update_file)
	else
	return nil, log_info("unable to determine update type from '" .. update_file .. "'.")
	end

[chrisant996]

@Trenly what should I expect winget show clink to report when the .exe installer has been used to freshly install v1.4.16 when no other version was installed yet?

C:\Users\chrisant>winget --version
v1.4.10173

This much is accurate:

C:\Users\chrisant>winget list | findstr clink
Clink v1.4.16.b8efc4                     clink_1.4.16.b8efc4                      1.4.16.b8efc4

But what nonsense is going on here? 🙃
It's citing v1.4.12, which is neither installed nor the latest version winget knows about.

C:\Users\chrisant>winget show clink
Failed in attempting to update the source: winget
Found Clink [chrisant996.Clink]
Version: 1.4.12.9ce7e8
Publisher: Christopher Antos
Publisher Url: https://github.com/chrisant996/
Publisher Support Url: https://github.com/chrisant996/clink/issues
...
Release Notes Url: https://github.com/chrisant996/clink/releases/tag/v1.4.12
...
Installer:
  Installer Type: nullsoft
  Installer Url: https://github.com/chrisant996/clink/releases/download/v1.4.12/clink.1.4.12.9ce7e8_setup.exe
  Installer SHA256: 3079cbc86df1a6273a1a7f2deb8195d165d9f3717fd9100ba64b0674e233ce74
  Release Date: 2023-01-20

I found some reports online about "Failed in attempting to update the source: winget". I have the latest released winget v1.4.10173. Is this an ignorable error caused by a server side cert problem as noted in microsoft/winget-cli#1656? Is my local cache of known versions stuck at somewhere around Jan 2023? winget show "Powertoys (Preview)" reports 0.66.0 from Jan 2023, but the latest that winget knows about is really 0.67.0. And Clink v1.4.12 is also from Jan 2023.

But I'm still confused why winget show clink reports about v1.4.12.

[Trenly]

@Trenly How did you determine that clink update was the main problem?

I installed v 1.4.18 via winget and noted the ARP Entries. Then I launched clink, and ran clink update. Checking the ARP Entries afterwards showed the DisplayName and DisplayVersion were updated from the 4-part 1.4.18. to 1.4.20 and the product code remained unchanged

I wiped the VM and re-installed 1.4.18, took note of the ARP entries. Then I ran winget upgrade clink which ran the 1.4.20 installer. Checking the ARP entries afterwards showed the DisplayName and DisplayVersion as the 4-part 1.4.20. and the product code was also updated.

I then looked at the metadata in the winget manifests and assessed how the matching logic would handle each scenario - comparing versions of different lengths and also how it interprets product codes, and described these in my original response.

If the current Clink being used was installed via the .exe installer, then clink update should be downloading a new .exe installer and running the .exe installer to perform the update. It relies on the .exe installer to do all the work.

That makes sense, but I'm not sure if that is actually happening

Did you find a repro where Clink was installed by the .exe installer, but clink update got confused into trying to update by using a .zip file?

I will check

Line 711 gets the current installation type. When installed via the .exe installer, it returns "exe". In my testing, that seems to be working correctly.

Line 718 refuses to continue unless the downloaded update type (zip or exe) matches the current installation type (zip or exe).

Lines 722-723 run the .exe installer to perform an update if Clink was installed via the .exe installer.

(I'm actually not sure why the .zip updater is fiddling with registry keys at all... I think that's a carryover from before it knew how to download+run the .exe installer.)

I haven’t taken a deep dive into how the zip updater works. I can understand why it may be updating the registry, but haven’t investigated yet

[Trenly]

Winget show seems to be working for me; Might just be a source error on your side? winget source reset --force may fix it

This is what I get after installing 1.4.16 using the exe -

[Trenly]

Did you find a repro where Clink was installed by the .exe installer, but clink update got confused into trying to update by using a .zip file?

Possibly? See screenshot below of install via winget in fresh sandbox, clink update, then refresh of ARP Table:

[chrisant996]

Aha. Apparently on my system winget is only able update its sources when run with elevation. It would be nice if it could explain that. I don't know why it requires admin access.

Now that it's been able to update its sources, winget show clink reports v1.4.20 as expected.

So now I started the test over:

• Uninstall Clink
• Install Clink v1.4.10
• Verify winget list clink:
  • Reports 1.4.10.45c041 is installed
  • Reports 1.4.20.236aba is available
• Run clink update
• It correctly identifies that it needs to download and run the .exe installer
• It elevates to do so
• KABOOM! Once elevated, it gets confused and performs a .zip update instead of a .exe update.

That's the problem: the .zip updater is accidentally getting run when it shouldn't be. And since the .zip updater has vestigial code to update regkeys (and in an incorrect manner), then things get weird, and since it attempted to update regkeys it can be easy to think it was working as intended, and easy to miss that it flew completely off the rails and down the embankment into the river.

😜

I'm debugging what's going on.

[chrisant996]

Oh. Of course... It's looking in HKCU. And when elevated, HKCU is administrator, not the user who installed Clink. So Clink isn't found in the installer regkeys. So it concludes it's a zip installation. Or at least that's my preliminary guess -- I'll know more soon.

UPDATE: Nope. Much simpler and much more embarrassing. Look at this line:

clink/clink/app/scripts/update.lua

Line 37 in 3563f62

local can_use_setup_exe = false

I had temporarily set it to false during a final phase of testing and validation. And then I failed to restore it to true before committing. 🤦‍♂️

UPDATE: I forgot -- it wasn't an embarrassing mistake. It was intentional, because during beta testing malware protection was getting excited too frequently for some users and blocking .exe installer from running. Always using the zip updater code was intentional to work around false positives in malware protection. 😕

So, it's indeed necessary to deal with the product code stuff.

[chrisant996]

Ok. I've tested updates mocking everything except the github site access, and doing multiple combinations of updating via .exe installers and .zip files. To the best of my knowledge and testing, it should work correctly in v1.4.21.

However, until a v1.4.21 or higher .exe installer runs, winget may still get confused and think there's an update when it's actually up to date. But that should ok, since if winget is confused it will run the latest .exe installer. Which will update to a constant product code, which should get things into agreement.

I hate making changes to established updater logic. It's nerve wracking. One tiny oversight and updating can become completely broken, which then prevents deploying a fix. However, as far as I can tell, everything will be good moving forward from v1.4.21. 🤞🍀

• The product code is "clink_chrisant996". I want to be cautious about the uniqueness of the product code name. Esp. since I found mention of some other product called "clink" that may exist.
• The display version is now Major.Minor.Patch in both the .exe installer and clink update.
• The .exe installer name still includes the .Commit part, and its title still shows the .Commit part.
• The Start Menu directory is now only one level deep (the versioned directory is gone).

[Trenly]

The product code is "clink_chrisant996". I want to be cautious about the uniqueness of the product code name. Esp. since I found mention of some other product called "clink" that may exist.

Great idea! Makes perfect sense

The display version is now Major.Minor.Patch in both the .exe installer and clink update.
The .exe installer name still includes the .Commit part, and its title still shows the .Commit part.

This should make things much easier to keep straight. I'm glad that the hash didn’t need to be "ripped out" everywhere, since I know it can be useful.

I'll keep an eye on the winget side, and once 1.4.21 can help test the matching and upgrade.

[MagicAndre1981]

I'll keep an eye on the winget side, and once 1.4.21 can help test the matching and upgrade.

works now as expected:

👍

[chrisant996]

works now as expected:
👍

Thanks very much for the confirmation, @MagicAndre1981 ! 🙂

[chrisant996]

Added "bug" label for tracking purposes, since there was something Clink wasn't doing quite properly from winget's perspective. Keeping "external" label as well, since it was mostly about how winget assumed things would be done by apps.

[MagicAndre1981]

ok, now it works as I expected this. Now it no longer matters how you update clink. Perfect 👍

[denelon]

I love the community cross collaboration!

And, for what it's worth, I also tend to keep issues open at winget-pkgs with an "Area-External" label to help identify when the issue requires something "outside" that repository to change in order to resolve issues.