[SSHV0] Fix bash: No such file or directory issue

[alexander-smolyakov]

Task name: SSHV0

Description:
This PR fixes bash: No such file or directory problem when ssh task running on windows agent.

Changes:

• Fixed problem with the naming of script file on remote host.
• All logic for remove Windows CR LF from the script file was extracted to separate function
• Updated comments for source code
• Removed extra variables

Documentation changes required: No

Added unit tests: No

Attached related issue:

• https://github.com/microsoft/build-task-team/issues/286
• SSH - Run Inline Script from a Windows agent fails #13661

Checklist:

• Task version was bumped - please check instruction how to do it
• Checked that applied changes work as expected

[gamullen]

When will this be GA for customers? One of my customers is hitting this, and it's a showstopper for them.

[anatolybolshakov]

Hi @gamullen we are going to roll out the fix here soon, approximately today or tomorrow

[dglambert]

🙏

[dglambert]

Is there another place where we can monitor the progress of this deployment?

[JaredMacDonald]

This issue is currently blocking one of my clients pipeline as well. Is there some way to check whether this fix has been deployed or not?

[alexander-smolyakov]

@JaredMacDonald we are already rolled out a fix, could you please verify with a customer that the issue resolved now?

[gamullen]

My customer sent me an email this morning indicating that it works now. Thanks for the quick turnaround and transparency on timelines!

[DD202]

It appears that this is a breaking change. When using 'script file' from a windows machine, the copy and translate adds the ._unix suffix but the script execution is done against the original named file.

2020-10-07T20:47:12.4253710Z ==============================================================================
2020-10-07T20:47:12.4254232Z Task : SSH
2020-10-07T20:47:12.4254644Z Description : Run shell commands or a script on a remote machine using SSH
2020-10-07T20:47:12.4255853Z Version : 0.176.1
2020-10-07T20:47:12.4256205Z Author : Microsoft Corporation
2020-10-07T20:47:12.4256645Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/ssh
2020-10-07T20:47:12.4257245Z ==============================================================================
2020-10-07T20:47:13.7780316Z Trying to establish an SSH connection to ***@xxxxxxxx:50000
2020-10-07T20:47:15.5265595Z Successfully connected.
2020-10-07T20:47:19.7455791Z tr -d '\015' <./xxxx-xxxx_xxxx.sh> ./xxxx-xxxx_xxxx.sh._unix
2020-10-07T20:47:20.1804069Z ##[error]bash: ./xxxx-xxxx_xxxx.sh: No such file or directory

In the prior version it did the following:
2020-10-06T19:45:29.4908565Z Task : SSH
2020-10-06T19:45:29.4908928Z Description : Run shell commands or a script on a remote machine using SSH
2020-10-06T19:45:29.4909256Z Version : 0.175.0
2020-10-06T19:45:29.4909553Z Author : Microsoft Corporation
2020-10-06T19:45:29.4909956Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/ssh
2020-10-06T19:45:29.4910388Z ==============================================================================
2020-10-06T19:45:29.9348162Z Trying to establish an SSH connection to ***@XXXXXXXXX:50000
2020-10-06T19:45:31.3996584Z Successfully connected.
2020-10-06T19:45:35.1297514Z tr -d '\015' <"./XXX_XXX-XXX.sh"> "./XXX_XXX-XXX.sh._unix"
2020-10-06T19:45:35.4913788Z chmod +x "./XXX_XXX-XXX.sh._unix"
2020-10-06T19:45:35.8528401Z "./XXX_XXX-XXX.sh._unix"

[ronniemacapobre]

We are also encountering the same issue described by @DD202. It appears that we are still not getting the SSH 0.177.0 version which has the fix.

[JaredMacDonald]

@JaredMacDonald we are already rolled out a fix, could you please verify with a customer that the issue resolved now?

Yes, this is fixed for my customer. Thanks!

[anatolybolshakov]

Hi @ronniemacapobre, we still have some issues with rolling out these changes for all customer, we are currently working on this and let you know once we complete it - sorry for inconvenience here

[ronniemacapobre]

Hi @ronniemacapobre, we still have some issues with rolling out these changes for all customer, we are currently working on this and let you know once we complete it - sorry for inconvenience here

Thanks for the update @anatolybolshakov. Should we update the build agent to pull the fix once the roll out is complete?

[devnetkc]

This has caused new troubles

List of runs yesterday when it failed. This pipeline runs every 4hrs to fetch changes. Nothing changed on our end on either side -- remote or local.

Last successful run time

First failed run time

Actual log output:

Starting: pullChanges
==============================================================================
Task         : SSH
Description  : Run shell commands or a script on a remote machine using SSH
Version      : 0.176.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/ssh
==============================================================================
Trying to establish an SSH connection to ***@**
Successfully connected.
tr -d '\015' <./wp-git-sync.sh> ./wp-git-sync.sh._unix
##[error]bash: ./wp-git-sync.sh: No such file or directory

##[error]Command failed with errors on remote machine.
Finishing: pullChanges

Last completed successful log

Starting: pullChanges
==============================================================================
Task         : SSH
Description  : Run shell commands or a script on a remote machine using SSH
Version      : 0.175.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/ssh
==============================================================================
Trying to establish an SSH connection to ***@**
Successfully connected.
tr -d '\015' <"./wp-git-sync.sh"> "./wp-git-sync.sh._unix"
chmod +x "./wp-git-sync.sh._unix"
"./wp-git-sync.sh._unix" -fo "https://***:***@***rc" -pd "/home/***/public_html/rc" -fe
Remote DevOps not found

Fetching devops

This was ran a few minutes ago, will report back once we see 0.177 installed. This is the full debug log of a failed run.

2020-10-08T14:59:21.7046113Z ##[debug]Evaluating condition for step: 'pullChanges'
2020-10-08T14:59:21.7048123Z ##[debug]Evaluating: succeeded()
2020-10-08T14:59:21.7048886Z ##[debug]Evaluating succeeded:
2020-10-08T14:59:21.7050343Z ##[debug]=> True
2020-10-08T14:59:21.7051115Z ##[debug]Result: True
2020-10-08T14:59:21.7051899Z ##[section]Starting: pullChanges
2020-10-08T14:59:21.7337811Z ==============================================================================
2020-10-08T14:59:21.7338202Z Task         : SSH
2020-10-08T14:59:21.7338550Z Description  : Run shell commands or a script on a remote machine using SSH
2020-10-08T14:59:21.7339003Z Version      : 0.176.1
2020-10-08T14:59:21.7339364Z Author       : Microsoft Corporation
2020-10-08T14:59:21.7339753Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/ssh
2020-10-08T14:59:21.7340204Z ==============================================================================
2020-10-08T14:59:21.9317780Z ##[debug]agent.TempDirectory=C:\Agent\_work\_temp
2020-10-08T14:59:21.9349561Z ##[debug]loading inputs and endpoints
2020-10-08T14:59:21.9366419Z ##[debug]loading ENDPOINT_AUTH_
2020-10-08T14:59:21.9387835Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_
2020-10-08T14:59:21.9391930Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2020-10-08T14:59:21.9394854Z ##[debug]loading ENDPOINT_AUTH_SCHEME_
2020-10-08T14:59:21.9397547Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2020-10-08T14:59:21.9400403Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2020-10-08T14:59:21.9403482Z ##[debug]loading INPUT_ARGS
2020-10-08T14:59:21.9406020Z ##[debug]loading INPUT_COMMANDS
2020-10-08T14:59:21.9408400Z ##[debug]loading INPUT_FAILONSTDERR
2020-10-08T14:59:21.9411250Z ##[debug]loading INPUT_INTERACTIVESESSION
2020-10-08T14:59:21.9413751Z ##[debug]loading INPUT_INTERPRETERCOMMAND
2020-10-08T14:59:21.9416115Z ##[debug]loading INPUT_READYTIMEOUT
2020-10-08T14:59:21.9418819Z ##[debug]loading INPUT_RUNOPTIONS
2020-10-08T14:59:21.9420980Z ##[debug]loading INPUT_SCRIPTPATH
2020-10-08T14:59:21.9423496Z ##[debug]loading INPUT_SSHENDPOINT
2020-10-08T14:59:21.9428391Z ##[debug]loading SECRET_DEVOPSHOST
2020-10-08T14:59:21.9431181Z ##[debug]loading SECRET_DEVOPSTOKEN
2020-10-08T14:59:21.9433972Z ##[debug]loading SECRET_DEVOPSTOKENUSER
2020-10-08T14:59:21.9441457Z ##[debug]loaded 18
2020-10-08T14:59:21.9446937Z ##[debug]Agent.ProxyUrl=undefined
2020-10-08T14:59:21.9448320Z ##[debug]Agent.CAInfo=undefined
2020-10-08T14:59:21.9448762Z ##[debug]Agent.ClientCert=undefined
2020-10-08T14:59:21.9449174Z ##[debug]Agent.SkipCertValidation=undefined
2020-10-08T14:59:22.0939000Z ##[debug]check path : C:\Agent\_work\_tasks\SSH_91443475-df55-4874-944b-39253b558790\0.176.1\task.json
2020-10-08T14:59:22.0943277Z ##[debug]adding resource file: C:\Agent\_work\_tasks\SSH_91443475-df55-4874-944b-39253b558790\0.176.1\task.json
2020-10-08T14:59:22.0944020Z ##[debug]system.culture=en-US
2020-10-08T14:59:22.0971063Z ##[debug]sshEndpoint=8aa67ad1-493e-4a8c-9325-0d985eda6a21
2020-10-08T14:59:22.0974678Z ##[debug]8aa67ad1-493e-4a8c-9325-0d985eda6a21 auth param username = ***
2020-10-08T14:59:22.0975401Z ##[debug]8aa67ad1-493e-4a8c-9325-0d985eda6a21 auth param password = undefined
2020-10-08T14:59:22.0976125Z ##[debug]8aa67ad1-493e-4a8c-9325-0d985eda6a21 data host = undefined
2020-10-08T14:59:22.0976857Z ##[debug]8aa67ad1-493e-4a8c-9325-0d985eda6a21 data port = undefined
2020-10-08T14:59:22.0979767Z ##[debug]interactiveSession=false
2020-10-08T14:59:22.0982428Z ##[debug]readyTimeout=20000
2020-10-08T14:59:22.0982974Z ##[debug]Using private key for ssh connection.
2020-10-08T14:59:22.0985325Z ##[debug]runOptions=script
2020-10-08T14:59:22.0988044Z ##[debug]scriptPath=C:\Agent\_work\10\s\wp-git-sync.sh
2020-10-08T14:59:22.0988599Z ##[debug]check path : C:\Agent\_work\10\s\wp-git-sync.sh
2020-10-08T14:59:22.0992237Z ##[debug]args=-fo "https://***:***@***rc" -pd "/home/***/public_html/rc" -fe
2020-10-08T14:59:22.0995435Z ##[debug]failOnStdErr=true
2020-10-08T14:59:22.1015428Z Trying to establish an SSH connection to ***@***
2020-10-08T14:59:22.5600113Z Successfully connected.
2020-10-08T14:59:22.5604273Z ##[debug]remoteScriptPath = ./wp-git-sync.sh._unix
2020-10-08T14:59:22.5605797Z ##[debug]Copying script to remote machine.
2020-10-08T14:59:23.5724579Z ##[debug]Copied script file to remote machine at: ./wp-git-sync.sh._unix
2020-10-08T14:59:23.5747213Z ##[debug]Fixing the line endings in case the file was created in Windows
2020-10-08T14:59:23.5748455Z tr -d '\015' <./wp-git-sync.sh> ./wp-git-sync.sh._unix
2020-10-08T14:59:23.5752215Z ##[debug]command = tr -d '\015' <./wp-git-sync.sh> ./wp-git-sync.sh._unix
2020-10-08T14:59:23.7317203Z ##[debug]stderr = bash: ./wp-git-sync.sh: No such file or directory

2020-10-08T14:59:23.7379923Z ##[error]bash: ./wp-git-sync.sh: No such file or directory

2020-10-08T14:59:23.7400235Z ##[debug]Processed: ##vso[task.issue type=error;]bash: ./wp-git-sync.sh: No such file or directory%0A
2020-10-08T14:59:23.7402068Z ##[debug]code = 1, signal = undefined
2020-10-08T14:59:23.7403166Z ##[debug]task result: Failed
2020-10-08T14:59:23.7404825Z ##[error]Command failed with errors on remote machine.
2020-10-08T14:59:23.7406610Z ##[debug]Processed: ##vso[task.issue type=error;]Command failed with errors on remote machine.
2020-10-08T14:59:23.7409684Z ##[debug]Processed: ##vso[task.complete result=Failed;]Command failed with errors on remote machine.
2020-10-08T14:59:23.7411109Z ##[debug]Closing the SSH client connection.
2020-10-08T14:59:23.8092982Z ##[section]Finishing: pullChanges

[alexander-smolyakov]

Hi @ronniemacapobre, we still have some issues with rolling out these changes for all customer, we are currently working on this and let you know once we complete it - sorry for inconvenience here

The fix should be available for all Azure DevOps users now, please verify that the issue is resolved now and let us know if there are still any issues.

[ronniemacapobre]

Hi @ronniemacapobre, we still have some issues with rolling out these changes for all customer, we are currently working on this and let you know once we complete it - sorry for inconvenience here

The fix should be available for all Azure DevOps users now, please verify that the issue is resolved now and let us know if there are still any issues.

Hi @alexander-smolyakov, I have confirmed that the issue is now resolved from our end. Thank you very much.