CI can't download from sourceforge

[barbeque-squared]

The "Install Dependencies" of the Windows CI appears to be having issues. see for example https://github.com/UltraStar-Deluxe/USDX/actions/runs/10758034942/job/29832874545

It eventually fails with Unable to establish SSL connection.

relevant part of the job output:

--2024-09-08 07:31:49--  https://netcologne.dl.sourceforge.net/project/nsis/NSIS%203/3.05/nsis-3.05-setup.exe
Resolving netcologne.dl.sourceforge.net (netcologne.dl.sourceforge.net)... 78.35.24.122
Connecting to netcologne.dl.sourceforge.net (netcologne.dl.sourceforge.net)|78.35.24.122|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://downloads.sourceforge.net/project/nsis/NSIS%203/3.05/nsis-3.05-setup.exe?download&failedmirror=netcologne.dl.sourceforge.net [following]
--2024-09-08 07:31:50--  https://downloads.sourceforge.net/project/nsis/NSIS%203/3.05/nsis-3.05-setup.exe?download&failedmirror=netcologne.dl.sourceforge.net
Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 204.68.111.105
Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|204.68.111.105|:443... connected.
Unable to establish SSL connection.
./nsis-3.05-setup.exe: D:\a\_temp\c6c6e22e-1e7f-4d7d-9d32-ea30973937bf.ps1:6

It works on my machine so I don't know where the difference is coming from here. Maybe the choco wget has ancient CA's, maybe sourceforge is doing sourceforge things. For comparison, my machine outputs:

Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|204.68.111.105|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: <redirects to geographically nearby me so probably irrelevant>

Two other minor things I noticed:

• the newpascal download+extraction is completely silent, but takes about two full minutes. I don't need logspam but if these commands have not-logspamming modes that might be useful, otherwise a few echo's before/after a long-running command would also be better than this complete silence
• why isn't the CI exiting when the wget fails? why does it still attempt to execute the nsis setup?

[s09bQ5]

The wget in chocolatey uses OpenSSL 3.1.0 with a patch to take certificates from the Windows certificate store. Both 78.35.24.122 and 204.68.111.105 use Let's encrypt certificates with the same root CA. The date in the wget output is well within the validity range of the certificate. The server name matches the wildcard in the certificate. Maybe a temporary problem?

We could try to cache the downloads.

The silent installation of newpascal annoyed me as well. And there is no reason to use this ancient version of newpascal for our release builds.

The CI doesn't exit when wget fails because it is using Powershell by default on Windows. See PowerShell/PowerShell#3415. According to the Github documentation we could switch to bash.

[barbeque-squared]

Apparently it was temporary as I notice when s09 triggered it, it ran fine. But I don't like flakyness like this. I can live with a build that just crashes randomly some % of the time (can just retry and it will complete with usually a single retry), but this was consistently crashing over a period of at least a few hours, and no amount of retries was going to change that outcome.

It'll be near impossible to figure out what was going on, but if caching could prevent this (or at least make it have less impact), it's better than what it is now. The other suggestions (newpascal install, properly exit on error) would also be good improvements regardless of the sourceforge whims.

[s09bQ5]

The SSL handshake timeout is too short. Compare https://github.com/UltraStar-Deluxe/USDX/actions/runs/10767743743/job/29855612872 and https://github.com/UltraStar-Deluxe/USDX/actions/runs/10767894246/job/29856024612 where I enabled wget's debug output. The latter uses 60 seconds timeout.

[DeinAlptraum]

We also currently fix the mirror to netcologne, which I'm not sure makes any sense. I stopped experiencing any errors when I pointed it to https://sourceforge.net/projects/nsis/files/NSIS%203/3.05/nsis-3.05-setup.exe/download instead to let sourceforge choose the mirror automatically.

See for comparison PR #896

[DeinAlptraum]

Regarding:

why isn't the CI exiting when the wget fails? why does it still attempt to execute the nsis setup?

apparently this is supposed to happen by default in the Windows runners' Powershell scripts, but has been bugged for two years already, see here

[barbeque-squared]

PR has been merged, I'm not keeping this open for an eventual switch from PowerShell to bash at it doesn't really have anything to do with sourceforge. It would still be appreciated of course, but it's less important than the pipeline itself.