Windows agents failing to clone Git (LongPathsEnabled disabled) #4574
Comments
We may need to perform several reductions in order to reduce that path. Infra might be able to shorten "C:\Jenkins\agent" to "C:\J\A", but that is probably not enough to fit inside the 256 character limit. I think that we may need to change the temporary directory name to use fewer characters, instead of "j h13373974888129494271", maybe "j h1337". I believe that is a test configuration change, not something that the Jenkins infrastructure team can change. If the temporary directory name of the scripts directory could be changed from 5878204d05f31ba38b85152718c2798e20a41786a9ccbc2012fefa3fbdfb649c to a much shorter version, that would also help. Are those changes that you can explore @jglick or that you can point me to a general area in the build system where I could explore?
Unfortunately, |
|
FWIW I can reproduce a more descriptive failure in my own Windows 10 VM using mvn -pl pipeline-model-definition surefire:test -Dtest=AgentTest#agentNoneAfter git config --list --show-origin --show-scopethe test passes. |
|
The temporary directory name inside |
|
Using e.g. |
|
The temporary directory name is being created by Java: https://github.com/jenkinsci/jenkins-test-harness/blob/1e79b0dc94b7b05e695838d3209d8ffd88504e8b/src/main/java/org/jvnet/hudson/test/TemporaryDirectoryAllocator.java#L90 We could try to make it shorter (at the loss of atomicity and with the risk of occasional path conflicts). I could shave off three or four characters by using a shorter project name in the test, perhaps. You could set e.g. |
|
Note that PR builds pass; it is |
In these agents? After the switch to AWS? |
Actually 260, IIUC. (because Windows) |
has the setting been enabled on windows though? (I beleive it is still needed in the registry on server versions) |
On that job in the console log it shows: |
|
BTW there is a bug in |
The git client plugin uses |
|
|
|
the kernel has not enabled long paths, so it doesn't matter what the git config is :) |
FWIW works for me. my checkout is in |
jglick
changed the title
|
I've tested new images from jenkinsci/docker-agent#949 with
@jtnord does it work if you try it in a longer folder? |
check #4574 (comment) try with use init/fetch and checkout not clone. (and if that works you have a reproducable test case to file for git-for-windows!) |
|
Thanks, I forgot about that comment, retrying.
Yes, that's exactly what I wanted for jenkinsci/docker-agent#949, I'll look into their repo to propose something. |
git version 2.46.0.windows.1 |
|
git init still failing unfortunately: https://ci.jenkins.io/job/Packaging/job/docker-agent/job/PR-949/7/pipeline-console/log?nodeId=168 |
|
Failing with a short repo folder inside the long named one too, retried with hardcoded path like yours cf jenkinsci/docker-agent@f192d43 Some progress, https://ci.jenkins.io/job/Packaging/job/docker-agent/job/PR-949/10/pipeline-console/log?nodeId=166 |
|
New setting delivered: let's see how it behaves in |
The problem looks gone on the ci.jenkins.io agents. I'm closing the issue and defering to @lemeurherveCB for the jenkinsci agents (which issues are managed outside the jenkins-infra helpdesk). Thanks y'all for the help! |
|
Does not seem fixed to me: jenkinsci/pipeline-model-definition-plugin#774 (comment) |
|
Damn, I misunderstood the issue. I thought the initial agent cloning steps where the problem. The change we applied did fix the initial cloning, but it looks like it was jgit, while the issue here is |
|
I don't understand what the error is. When replaying the pipeline, I got the expected git and kernel setups: |
|
Seen also jenkinsci/pipeline-as-yaml-plugin#131 (comment) Linux work fine. No changes on code, simple parent bump |
|
Note: Was facing similar issues on Windows 10 nodes few years ago. I found the config (in my case was ansible) that was fixing all such git clone on windows - name: Git set longpaths
ansible.windows.win_shell: git config --system core.longpaths true
changed_when: false
tags: git
- name: Windows set longpaths
ansible.windows.win_regedit:
path: HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem
name: LongPathsEnabled
type: dword
data: 1
tags: git |
and
or am I missing something ? |
|
You're not missing anything, the error persists even with these settings set on the host and in the image.
The thing is, |
What image are you referring to @lemeurherveCB ? The mentioned build (https://ci.jenkins.io/job/Plugins/job/pipeline-model-definition-plugin/job/master/641/) utilizes a Windows VM agents (label
🤔 why would we need I feel like the commands at https://github.com/jenkins-infra/pipeline-library/blob/e01cbb988257ae3f308c82476a4792db4960365d/vars/infra.groovy#L144-L149 should be set with the |
|
I'm referring to the docker-agent image where I'm trying to reproduce a test and a fix. It's in this image that I can't call I'm aware this won't resolve this infra issue, but I figured it could help finding what's wrong, that may not be the case. |
Thanks for the explanation, as it wasn't clear for us before. It make sense to have a differential analysis like this one! => btw, if we downgrade git-windows to 2.47.x, is the problem present again? |
I might have missed a few messages, but https://github.com/git-for-windows/build-extra/blob/main/ReleaseNotes.md#git-for-windows-v2481-release-notes states
=> so we now know what is the problem: Git 2.48.x upgrade (which is the common denominator between the Windows Server Core 2019 VMs of ci.jio and the Nanoserver Docker images mentioned by @lemeurherveCB). I guess, it is worth checking the result with => on short term, I believe we may think to revert git to 2.47.x until we find a viable solution. WDYT @smerle33 @MarkEWaite ? |
|
Testing 2.47.1 with jenkinsci/docker-agent@c3ed713 |
I may be wrong but it seems that Checked with https://github.com/jenkinsci/docker-agent/blob/84ad70558949b7e718dfd6d127d32037686e021c/build.ps1#L51-L53 in build.ps1: # Sanity checks
Invoke-Expression 'git config --global --list'
Invoke-Expression 'reg.exe query HKLM\SYSTEM\CurrentControlSet\Control\FileSystem' |
|
PR ready, will be merged if previous version of git for windows is working correctly as confirmed by @lemeurherveCB |
the build stage run on 
with tag version 2.30.0 with the PR: jenkins-infra/packer-images#1779 weird |
|
This LongPaths is enabled, it seems that's the git config which is missing core.longPaths |
|
Which is weird as it should be, according to https://github.com/smerle33/packer-images/blob/c945b88cfc95db03b81fbbe439867da2c64c6212/provisioning/windows-provision.ps1#L146 EDIT: I was looking at |
|
Testing setting up core.longPaths directly in build.ps1 for now: jenkinsci/docker-agent@ab05d96 |
|
Are these tests using the git testing repo? I can not recall where this was but I fixed this a few years ago, and this may be causing the issues now if it is not picking up the system config (on purpose) jenkinsci/git-plugin@8975775 should not IIUC impact settings. |
|
I see. I was checking also my nodes And never saw any occurence of very long path So this might be related to |
It was a hypothesis, but if git is configured to use [1] I have both the windows setting and core.longpaths set (global) for my user. |
|
For diagnostic purposes I would suggest git config --show-origin --show-scope --list |
Service(s)
ci.jenkins.io
Summary
https://github.com/jenkinsci/pipeline-model-definition-plugin/runs/38184754576 (blocking deployment of jenkinsci/pipeline-model-definition-plugin#770)
The filename here is 263 characters long, which may exceed path length limitations.
Please
git config core.longpaths trueContext: jenkinsci/branch-api-plugin#129 (comment)
Reproduction steps
No response
The text was updated successfully, but these errors were encountered: