Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remote-SSH connection stalls and never continues #84

Closed
joeygibson opened this issue May 3, 2019 · 42 comments
Closed

Remote-SSH connection stalls and never continues #84

joeygibson opened this issue May 3, 2019 · 42 comments
Labels
info-needed Issue requires more information from poster ssh Issue in vscode-remote SSH

Comments

@joeygibson
Copy link

joeygibson commented May 3, 2019

  • VSCode Version: Version 1.34.0-insider (1.34.0-insider) 473af338e1bd9ad4d9853933da1cd9d5d9e07dc9
  • Local OS Version: MacOS Mojave 10.14.4 (18E226)
  • Remote OS Version: PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
    NAME="Debian GNU/Linux"
    VERSION_ID="9"
    VERSION="9 (stretch)"
    ID=debian
  • Remote Extension/Connection Type: SSH

Steps to Reproduce:

  1. Cmd-Shift-P, Remote-SSH: Connect current window to host...

  2. Connection is successful. Remote tools are installed and server is running on remote:

    $ ps -ef | grep code
    root 281660 1 0 16:36 ? 00:00:00 sh /root/.vscode-remote/bin/473af338e1bd9ad4d9853933da1cd9d5d9e07dc9/server.sh --port=0
    root 281669 281660 5 16:36 ? 00:00:00 /root/.vscode-remote/bin/473af338e1bd9ad4d9853933da1cd9d5d9e07dc9/node /root/.vscode-remote/bin/473af338e1bd9ad4d9853933da1cd9d5d9e07dc9/out/remoteExtensionHostAgent.js --port=0

  3. This shows up:
    image
    and it never goes beyond that.

  4. Here's the full log:

    SSH Resolver called for "ssh-remote+redacted.machinename.com"
    SSH Resolver called for host: redacted.machinename.com
    Setting up SSH remote "redacted.machinename.com"
    Using commit id "473af338e1bd9ad4d9853933da1cd9d5d9e07dc9" and quality "insider" for server
    Install and start server if needed
    > Linux redacted.machinename.com 4.14.80 #4 SMP Tue Nov 13 16:27:02 EST 2018 x86_64
    > 
    > mesg: ttyname failed: 
    > Inappropriate ioctl for device
    > bash: cannot set terminal process group (281625): Inappropriate ioctl for device
    > bash: no job control in this shell
    > mesg: ttyname failed: Inappropriate ioctl for device
    > Found existing installation...
    > Printing the current remote environment...
    > PYTHON_EGG_CACHE=/var/tmp/
    > LC_ALL=en_US.utf8
    > LD_LIBRARY_PATH=/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext:/lib:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext
    > SSH_CONNECTION=10.0.7.10 60630 10.0.29.202 22
    > VERTICAINI=/etc/vertica.ini
    > LANG=en_US.UTF-8
    > HISTTIMEFORMAT=%D %T 
    > JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
    > S_COLORS=auto
    > USER=root
    > PWD=/root
    > HOME=/root
    > SSH_CLIENT=10.0.7.10 60630 22
    > https_proxy=http://proxy.redacteddomain.com:3128
    > http_proxy=http://proxy.redacteddomain.com:3128
    > 
    > no_proxy=localhost,127.0.0.1,10.*,172.16.*,192.168.*,redacteddomain.com
    > MAIL=/var/mail/root
    > SHELL=/bin/bash
    > LS_OPTIONS=--color=auto
    > TMOUT=3600
    > SHLVL=3
    > LANGUAGE=en_US
    > TERMINFO=/lib/terminfo
    > LOGNAME=root
    > XDG_RUNTIME_DIR=/run/user/0
    > PATH=/opt/vertica/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    > _=/usr/bin/printenv
    > Starting agent...
    > Waiting for server log...
    >  
    > *
    > * Reminder: You may only use this software with Visual Studio family products,
    > * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057)
    > *
    >  
    > 1bab7624-76d1-4d65-9542-c072562363dd==42089==
    "install" terminal command done
    Received install output: 1bab7624-76d1-4d65-9542-c072562363dd==42089==
    Server is listening on port 42089
    Spawning tunnel with: ssh  redacted.machinename.com -N -L localhost:9456:localhost:42089
    Spawned SSH tunnel between local port 9456 and remote port 42089
    Waiting for ssh tunnel to be ready
    Tunneling remote port 42089 to local port 9456
    Resolving "ssh-remote+redacted.machinename.com" to "localhost:9456", attempt: 1

The machine can be gotten to through ssh, obviously, since VSCode was able install and start the server. The server has internet access through a proxy, but there's a no_proxy that covers the IP space my laptop is in.

I've let it sit for over half an hour, and it never progresses any further.

If I can provide any further details, please let me know.

Does this issue occur when you try this locally?: Yes
Does this issue occur when you try this locally and all extensions are disabled?: Yes

Update: The solution to my specific problem is explained here #84 (comment)

@kieferrm kieferrm added the ssh Issue in vscode-remote SSH label May 3, 2019
@davidwin
Copy link

davidwin commented May 3, 2019

Is TCP forwarding enabled on the ssh server? I see the exact same issue where I am testing this and we have forwarding disabled for security reasons.

@joeygibson
Copy link
Author

Is TCP forwarding enabled on the ssh server? I see the exact same issue where I am testing this and we have forwarding disabled for security reasons.

There was nothing in /etc/ssh/sshd_config enabling or disabling it, so I added AllowTcpForwarding yes and restarted sshd, but it made no difference.

@davidwin
Copy link

davidwin commented May 3, 2019

Hm, have you tried the same ssh command with a different client port (i.e. ssh redacted.machinename.com -N -L localhost:9457:localhost:42089) and then tried accessing the local port? When I do that, the ssh client reports that the operation is "administratively prohibited" or something like that. (Can't test now.)

@wqfan
Copy link

wqfan commented May 3, 2019

@davidwin I have the same issue as @joeygibson. The tunnel was established:

"install" terminal command done
Received install output: 0e53b95e-f0b8-4c7f-85c6-a21823a0c689==45539==
Server is listening on port 45539
Spawning tunnel with: ssh  dev-server -N -L localhost:53777:localhost:45539
Spawned SSH tunnel between local port 53777 and remote port 45539
Waiting for ssh tunnel to be ready
Tunneling remote port 45539 to local port 53777
Resolving "ssh-remote+dev-server" to "localhost:53777", attempt: 1

When I tried to access localhost:53777 from browser, I got "Not Found" on the page.

Also, VS Code shows an error at right-bottom:

Failed to connect to the remote extension host server (Error: Connection error: Unauthorized client refused.)

@joeygibson
Copy link
Author

I ran ssh redacted.machinename.com -N -L localhost:9457:localhost:42089 and then this

curl localhost:9457
curl: (56) Recv failure: Connection reset by peer

@roblourens
Copy link
Member

@wqfan Do you know whether AllowTcpForwarding is disabled?

Also can you share the logs from the "Remote - SSH" output channel that might explain Unauthorized client refused?

@roblourens
Copy link
Member

@joeygibson I don't understand, did you try to run a curl command or is that an error that was written?

@roblourens roblourens added the info-needed Issue requires more information from poster label May 4, 2019
@wqfan
Copy link

wqfan commented May 4, 2019

@roblourens I have AllowTcpForwarding enabled. I believe my server is able to do TCP forwarding since I have been using SSH channel for other purposes.

Here is the full output from "Remote - SSH" channel:

SSH Resolver called for "ssh-remote+dev-server"
SSH Resolver called for host: dev-server
Setting up SSH remote "dev-server"
Using commit id "473af338e1bd9ad4d9853933da1cd9d5d9e07dc9" and quality "insider" for server
Install and start server if needed
> bash: no job control in this shell
> Found existing installation...
> Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ
> Found running server...
>  
> *
> * Reminder: You may only use this software with Visual Studio family products,
> * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057)
> *
>  
> bb80525e-2122-4fa5-8e4a-9d91821eeb83==45539==
"install" terminal command done
Received install output: bb80525e-2122-4fa5-8e4a-9d91821eeb83==45539==
Server is listening on port 45539
Spawning tunnel with: ssh  dev-server -N -L localhost:32546:localhost:45539
Spawned SSH tunnel between local port 32546 and remote port 45539
Waiting for ssh tunnel to be ready
Tunneling remote port 45539 to local port 32546
Resolving "ssh-remote+dev-server" to "localhost:32546", attempt: 1

Logs from "Log (Window)" channel:

[2019-05-03 18:02:25.843] [renderer1] [error] Connection error: Unauthorized client refused.: Error: Connection error: Unauthorized client refused.
    at v (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:1544:9)
    at t.onControlMessage.o (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:1542:930)
    at d.fire (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:77:818)
    at a (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:321:48)
    at e (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:321:94)
    at d.fire (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:77:818)
    at S._receiveMessage (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:330:65)
    at S._socketDisposables.push._socketReader.onMessage.e (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:327:243)
    at d.fire (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:77:818)
    at g.acceptChunk (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:323:801)
    at _register._socket.onData.e (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:323:153)
    at Socket.t (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:332:235)
    at Socket.emit (events.js:182:13)
    at addChunk (_stream_readable.js:279:12)
    at readableAddChunk (_stream_readable.js:264:11)
    at Socket.Readable.push (_stream_readable.js:219:10)
    at TCP.onread (net.js:636:20)
[2019-05-03 18:02:25.844] [renderer1] [error] Cannot read property 'extensions' of null: TypeError: Cannot read property 'extensions' of null
    at e.<anonymous> (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:3850:751)
    at Generator.next (<anonymous>)
    at r (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:34:454)
    at process._tickCallback (internal/process/next_tick.js:68:7)
[2019-05-03 18:02:25.844] [renderer1] [error] Connection error: Unauthorized client refused.: Error: Connection error: Unauthorized client refused.
    at v (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:1544:9)
    at t.onControlMessage.o (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:1542:930)
    at d.fire (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:77:818)
    at a (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:321:48)
    at e (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:321:94)
    at d.fire (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:77:818)
    at S._receiveMessage (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:330:65)
    at S._socketDisposables.push._socketReader.onMessage.e (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:327:243)
    at d.fire (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:77:818)
    at g.acceptChunk (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:323:801)
    at _register._socket.onData.e (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:323:153)
    at Socket.t (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:332:235)
    at Socket.emit (events.js:182:13)
    at addChunk (_stream_readable.js:279:12)
    at readableAddChunk (_stream_readable.js:264:11)
    at Socket.Readable.push (_stream_readable.js:219:10)
    at TCP.onread (net.js:636:20)
[2019-05-03 18:02:28.207] [renderer1] [error] Could not fetch environment: Error: Could not fetch environment
    at d.getEnvironment.then.e (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:3813:413)

@roblourens
Copy link
Member

Warning: bad syntax, perhaps a bogus '-'

This warning sounds relevant, can you tell me about your remote OS? And possibly the output of ps --version?

@wqfan
Copy link

wqfan commented May 4, 2019

@roblourens Sure.

$ cat /etc/system-release
Amazon Linux Bare Metal release 2012.03
$ ps --version
procps version 3.2.8

@roblourens
Copy link
Member

@wqfan please open another issue, I realized that you got through the SSH part successfully and Unauthorized client refused is a different sort of issue. We can continue debugging in another issue.

@wqfan
Copy link

wqfan commented May 4, 2019

@roblourens Sounds good. I opened: #103

@lars18th
Copy link

lars18th commented May 5, 2019

Hi,

I have the same problem. And after some debug in the server, I see this behaviour:

  • The Remote-SSH starts the server, and gets the listening port. However, after some seconds the server is closed. I don't know the reason, but the client can't connect because the listening port is closed. However, in the logs is not printed anything about the cause of the server exit.

So, I suggest:

  1. Print in the server log the cause for the exit.
  2. Add some forwarding control to inform to the client that the server is not running.

Regards.

@zhouxiaowei1120
Copy link

zhouxiaowei1120 commented May 6, 2019

I have met the similar problem. I followed #75 and solved it.

Please find details in #75, as follows are the main operations.

  1. set the sshd config AllowTcpForwarding yes on server.
  2. I just add these
Host *
    ControlMaster auto
    ControlPath  ~/.ssh/sockets/%r@%h-%p
    ControlPersist  600

in ~/.ssh/config on local.

Hope it will help for you.

@davidwin
Copy link

davidwin commented May 6, 2019

After reading through the comments, it's obvious that there could be multiple reasons for a stalled connection, without much logging to tell the difference. What I did to conclude that TCP forwarding was disabled was to:

  • Test that I could access the remote server port from the remote server, with e.g. curl or plain telnet. If it works, the server is running and you are able to resolve localhost at least.
  • Run the ssh -N -L ... command manually, specifying a different local port. Make sure that LogLevel isn't set to QUIET or ERROR.
  • Now access the local port on the desktop. If you use telnet, you will notice that the connection is closed immediately, and ssh logs this error:
    You are connected to: example.com
    channel 2: open failed: administratively prohibited: open failed

@CoderOverflow
Copy link

I met the same problem of connecting the ubuntu 16

"install" terminal command done
Received install output: 7a739fc7-a8c1-43d8-b543-58ba23365f29==40471==
Server is listening on port 40471
Spawning tunnel with: ssh 111-N -L localhost:57831:localhost:40471
Spawned SSH tunnel between local port 57831 and remote port 40471
Waiting for ssh tunnel to be ready
Tunnel(40471) stderr: Permission denied, please try again.

Tunnel(40471) stderr: Permission denied, please try again.

Tunnel(40471) stderr: xxxx@192.168.111.111: Permission denied (publickey,password).

Could not establish tunnel to "111" (Max wait time exhausted).
Could not establish tunnel to "111" (Max wait time exhausted).

@joeygibson
Copy link
Author

After adding the Control* settings to ~/.ssh/config, as zhouxiaowei1120 suggested, two things happened: I got lots of debug messages showing up in my terminal, and VSCode got slightly farther before hanging. Here's the VSCode log:

"install" terminal command done
Received install output: a78a1c82-c267-4020-ad6b-df7c1558c344==35031==
Server is listening on port 35031
Spawning tunnel with: ssh  pod-fc -N -L localhost:34661:localhost:35031
Spawned SSH tunnel between local port 34661 and remote port 35031
Waiting for ssh tunnel to be ready
Tunneling remote port 35031 to local port 34661
Resolving "ssh-remote+pod-fc" to "localhost:34661", attempt: 1
Tunnel(35031) stderr: mesg: ttyname failed: Inappropriate ioctl for device

And here's what showed up in my terminal:

debug1: multiplexing control connection
debug1: channel 1: new [mux-control]
debug1: channel 6: new [client-session]
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
debug1: client_input_channel_req: channel 6 rtype exit-status reply 0
debug1: channel 6: free: client-session, nchannels 9
debug1: channel 1: free: mux-control, nchannels 8
debug1: multiplexing control connection
debug1: channel 1: new [mux-control]
debug1: channel 6: new [client-session]
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
debug1: client_input_channel_req: channel 6 rtype exit-status reply 0
debug1: channel 6: free: client-session, nchannels 9
debug1: channel 1: free: mux-control, nchannels 8
debug1: multiplexing control connection
debug1: channel 1: new [mux-control]
debug1: Local forwarding listening on ::1 port 62371.
debug1: channel 6: new [port listener]
debug1: Local forwarding listening on 127.0.0.1 port 62371.
debug1: channel 7: new [port listener]
debug1: channel 10: new [client-session]
debug1: Sending environment.
debug1: Sending env LC_CTYPE =
debug1: Connection to port 62371 forwarding to localhost port 35031 requested.
debug1: channel 11: new [direct-tcpip]
debug1: Connection to port 62371 forwarding to localhost port 35031 requested.
debug1: channel 12: new [direct-tcpip]
debug1: Connection to port 62371 forwarding to localhost port 35031 requested.
debug1: channel 13: new [direct-tcpip]
channel 11: open failed: administratively prohibited: open failed
channel 12: open failed: administratively prohibited: open failed
debug1: channel 11: free: direct-tcpip: listening port 62371 for localhost port 35031, connect from 127.0.0.1 port 60333 to 127.0.0.1 port 62371, nchannels 14
debug1: channel 12: free: direct-tcpip: listening port 62371 for localhost port 35031, connect from 127.0.0.1 port 60334 to 127.0.0.1 port 62371, nchannels 13
channel 13: open failed: administratively prohibited: open failed
debug1: channel 13: free: direct-tcpip: listening port 62371 for localhost port 35031, connect from 127.0.0.1 port 60335 to 127.0.0.1 port 62371, nchannels 12

In both cases, pod-fc is the remote hostname (same machine as my initial report). I was able to get VSCode's remote editing working from my laptop with an Ubuntu box I spun up this morning in AWS, so the client side is setup correctly.

@roblourens
Copy link
Member

It sounds like tcp forwarding may be disabled for you @joeygibson

channel 11: open failed: administratively prohibited: open failed

Do you know whether that's the case?

@joeygibson
Copy link
Author

It sounds like tcp forwarding may be disabled for you @joeygibson

channel 11: open failed: administratively prohibited: open failed

Do you know whether that's the case?

As I said here #84 (comment) I didn't see that it was disabled, so I explicitly enabled it, but to no avail.

@lars18th
Copy link

lars18th commented May 6, 2019

After reading through the comments, it's obvious that there could be multiple reasons for a stalled connection, without much logging to tell the difference. What I did to conclude that TCP forwarding was disabled was to:

Hi,

From my experience, the main problem isn't the tunnel. If you can connect with SSH then the tunnel only fails if the server has disabled tunnels. And this is easy to check it. The real problem is that the end of the tunnel is offline. That's the server starts and after it closes. And for this reason you can't connect. It's quite simple, but it's hard (at time) to determine the cause of the fail.

So I suggest to print more logs regading the close of the server.
I hope it helps.
Regards.

@roblourens
Copy link
Member

@joeygibson can you give the contents of ~/.vscode-remote/.473af338e1bd9ad4d9853933da1cd9d5d9e07dc9.log to see whether it's the same as #103 (comment)? It sounds like you are seeing a different error message @lars18th so it's not obvious that it's the same issue.

And please check for errors in the developer tools (Help > Toggle developer tools)

Another problem that came up in another issue is that localhost was not bound to 127.0.0.1 on the server, is that possible?

@joeygibson
Copy link
Author

@roblourens Here's the contents of the log file:

Remote configuration data at /root/.vscode-remote


*
* Visual Studio Code Server
*
* Reminder: You may only use this software with Visual Studio family products,
* as described in the license https://aka.ms/vscode-remote/license
*


IP Address: <my ip address>
Extension host agent listening on 38301

And yes, my /etc/hosts does have

127.0.0.1 localhost

in it

@lars18th
Copy link

lars18th commented May 6, 2019

Hi @joeygibson ,

Think on this:

  • The SSH connection works.
  • The server is started.
  • The script gets the listening socket.
  • The SSH forwarding can be stablished, as localhost, SSH and the target port are ok.

... But the connection can fail if the server CLOSES. Then all connections from the client to the server fails. Please, improve the handle of this case. I belive a lot of problems are related to this cause.

Regards.

@thernstig
Copy link

I also had the issue of channel 2: open failed: administratively prohibited: open failed. The fix in my case was to set the sshd config AllowTcpForwarding yes.

@joeygibson
Copy link
Author

I noticed that both processes that get started on my machine have --port=0. Is that normal? That seems like it's wrong. Unless that's a way to tell the process to pick a random port.

@lars18th
Copy link

lars18th commented May 7, 2019

I also had the issue of channel 2: open failed: administratively prohibited: open failed. The fix in my case was to set the sshd config AllowTcpForwarding yes.

Yes, this is the first cause of this message. However, when this isn't then the most likely cause is that the target port is closed (because the server has terminated).

I noticed that both processes that get started on my machine have --port=0. Is that normal? That seems like it's wrong. Unless that's a way to tell the process to pick a random port.

In this case the most likely cause is that the server is not started or the script has not received (readed) the listening port.

@joeygibson
Copy link
Author

In this case the most likely cause is that the server is not started or the script has not received (readed) the listening port.

The script is running. I found the --port=0 in a ps listing on my server.

@achhabr1
Copy link

achhabr1 commented May 8, 2019

@joeygibson Are you able to resolve this? I am facing similar issue

Remote configuration data at /home/ac043s/.vscode-remote

Error: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by /home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/node_modules/spdlog/build/Release/spdlog.node)
at Object.Module._extensions..node (internal/modules/cjs/loader.js:731:18)
at Module.load (internal/modules/cjs/loader.js:612:32)
at tryModuleLoad (internal/modules/cjs/loader.js:551:12)
at Function.Module._load (internal/modules/cjs/loader.js:543:3)
at Module.require (internal/modules/cjs/loader.js:650:17)
at Module.patchedRequire [as require] (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/node_modules/diagnostic-channel/dist/src/patchRequire.js:14:46)
at require (internal/modules/cjs/helpers.js:20:18)
at bindings (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/node_modules/bindings/bindings.js:81:44)
at Object. (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/node_modules/spdlog/index.js:3:35)
at Module._compile (internal/modules/cjs/loader.js:702:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:713:10)
at Module.load (internal/modules/cjs/loader.js:612:32)
at tryModuleLoad (internal/modules/cjs/loader.js:551:12)
at Function.Module._load (internal/modules/cjs/loader.js:543:3)
at Module.require (internal/modules/cjs/loader.js:650:17)
at Module.patchedRequire [as require] (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/node_modules/diagnostic-channel/dist/src/patchRequire.js:14:46)
at require (internal/modules/cjs/helpers.js:20:18)
at Function.i [as __$__nodeRequire] (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:31:542)
at Object.t.createSpdLogService (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.js:213:301)
at G. (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.js:363:833)
at Generator.next ()
at /home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.js:16:388
at new Promise ()
at r (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.js:16:165)
at G._createServices (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.js:363:624)
at Function.create (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.js:363:554)
at T. (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.js:370:597)
at Generator.next ()
at /home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.js:16:388
at new Promise ()
at r (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.js:16:165)
at T.start (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.js:370:516)
at Object. (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.js:379:52)
at Function.t._safeInvokeFunction (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:20:173)
at Function.t._invokeFactory (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:20:420)
at t.complete (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:20:657)
at s._onModuleComplete (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:30:796)
at s._onModuleComplete (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:31:41)
at s._onModuleComplete (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:31:41)
at s._onModuleComplete (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:31:41)
at s._onModuleComplete (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:31:41)
at s._onModuleComplete (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:31:41)
at s._resolve (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:30:356)
at s.defineModule (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:24:768)
at r (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:28:919)
at Object.l (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:36:405)
at Function.t._safeInvokeFunction (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:20:173)
at Function.t._invokeFactory (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:20:420)
at t.complete (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:20:657)
at s._onModuleComplete (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:30:796)
at s._onModuleComplete (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:31:41)
at s._resolve (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:30:356)
at s.defineModule (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:24:768)
at r (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:31:950)
at f (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:17:524)
at Object. (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/agent/remoteExtensionHostAgent.nls.js:4:1)
at t._loadAndEvalScript (/home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:17:568)
at /home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/out/vs/loader.js:17:285
at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:440:3)
IP Address: 10.10.57.179
IP Address: 192.168.53.4
IP Address: 192.168.54.4
IP Address: 192.168.55.4
IP Address: 192.168.59.4
IP Address: 192.168.56.4
IP Address: 192.168.132.4
IP Address: 192.168.72.4
IP Address: 192.168.71.4
IP Address: 192.168.64.4
IP Address: 172.30.7.212
IP Address: 192.168.165.4
Extension host agent listening on 40105
Unauthorized client refused.
Unauthorized client refused.
{ [Error: ENOENT: no such file or directory, scandir '/home/ac043s/.vscode-remote/data/logs']
errno: -2,
code: 'ENOENT',
syscall: 'scandir',
path: '/home/ac043s/.vscode-remote/data/logs' }
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.
Unauthorized client refused.

@achhabr1
Copy link

achhabr1 commented May 9, 2019

Getting this from a ubuntu server

`> LESSOPEN=| /usr/bin/lesspipe %s

XDG_RUNTIME_DIR=/run/user/1000
J2REDIR=/usr/lib/jvm/java-8-oracle/jre
LESSCLOSE=/usr/bin/lesspipe %s %s
_=/usr/bin/printenv
OLDPWD=/home/danush/.vscode-remote/bin/693a13cd32c5be798051edc0cb43e1e39fc456d9
Starting agent...
Waiting for server log...

a91dbb54-b7d9-4bdc-8e30-e1b50c1c334b==44649==
"install" terminal command done
Received install output: a91dbb54-b7d9-4bdc-8e30-e1b50c1c334b==44649==
Server is listening on port 44649
Spawning tunnel with: ssh 135.16.39.199 -L localhost:27325:localhost:44649 "echo -e 'Connected to SSH Host - Please do not close this terminal' && sleep infinity"
Spawned SSH tunnel between local port 27325 and remote port 44649
Waiting for ssh tunnel to be ready
danush@135.16.39.199's password:
Could not establish tunnel to "135.16.39.199" (Max wait time exhausted).
Could not establish tunnel to "135.16.39.199" (Max wait time exhausted).`

Has anyone been able to resolve this?

@lars18th
Copy link

Hi,

The relevant part is this bug (log copied from the first post) is this:

    > mesg: ttyname failed: 
    > Inappropriate ioctl for device
    > bash: cannot set terminal process group (281625): Inappropriate ioctl for device
    > bash: no job control in this shell

See my comment at #103 (comment)

@zbage
Copy link

zbage commented May 16, 2019

server: centos
Failed to connect to the remote extension host server (Error: Connection error: Unauthorized client refused.)

Remote configuration data at /root/.vscode-remote

*
* Visual Studio Code Server
*
* Reminder: You may only use this software with Visual Studio family products,
* as described in the license https://aka.ms/vscode-remote/license
*

IP Address: 192.168.1.6
Extension host agent listening on 53753
Unauthorized client refused.
Unauthorized client refused.

@Anudorannador
Copy link

server: centos
Failed to connect to the remote extension host server (Error: Connection error: Unauthorized client refused.)

Remote configuration data at /root/.vscode-remote

*
* Visual Studio Code Server
*
* Reminder: You may only use this software with Visual Studio family products,
* as described in the license https://aka.ms/vscode-remote/license
*

IP Address: 192.168.1.6
Extension host agent listening on 53753
Unauthorized client refused.
Unauthorized client refused.

Exactly the same for me : (

@syockit
Copy link

syockit commented May 18, 2019

Somehow these two values never match.

a.signedData="186P+yv1gtYPNvobZ2zQw+6cFam2RVgkmkP3v+HZ0AlzVk="
k="7f19a0be-6c9d-4890-a219-19b42faf0ce5"
Unauthorized client refused.
a.signedData="186P+yv1gtYPNvobZ2zQw+6cFam2RVgkmkP3v+HZ0AlzVk="
k="7f19a0be-6c9d-4890-a219-19b42faf0ce5"
Unauthorized client refused.

Forcing the signed data to be the same as the UUID made it work for me.


@achhabr1

Error: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by /home/ac043s/.vscode-remote/bin/daf71423252a707b8e396e8afa8102b717f8213b/node_modules/spdlog/build/Release/spdlog.node)

Please install newer libstdc++ and point LD_LIBRARY_PATH to there from your login script (.bashrc, but somehow in my workplace it was .bash_login)

P/S: While that solves running the node module, I now have problem starting the CPP extension because it requires a new libc, which cannot be easily overriden using LD_LIBRARY_PATH (due to relocation problem), but that is of course off-topic for this ticket.
PP/S: Well, managed to make the CPP extension work by unpacking newer glibc to somewhere local, renaming the extension binaries to something else and replace it with a script that runs the new glibc's ld.

@roblourens
Copy link
Member

The "unauthorized client refused" error is being tracked here: #103

I don't believe the OP in this issue is having the same issue, this issue should focus on whatever @joeygibson is seeing which we haven't gotten to the bottom of.

Sorry for neglecting this issue, my best guess is still just that tcp forwarding is somehow disabled based on seeing

channel 11: open failed: administratively prohibited: open failed

@vscodebot
Copy link

vscodebot bot commented May 26, 2019

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.

Happy Coding!

@joeygibson
Copy link
Author

As multiple people suggested, the problem was, in fact, that TCP forwarding was blocked. I tried setting AllowTcpForwarding=yes in /etc/ssh/sshd_config and restarting ssh, but that didn't work. It turned out that -o AllowTcpForwarding=no was present in /etc/default/ssh, which I didn't know to check. Once I removed that option and restarted ssh, everything worked.

@lars18th
Copy link

As multiple people suggested, the problem was, in fact, that TCP forwarding was blocked.

No, quite the opposite in 99%!
The forwarding works but the listening port on the server is not running. That's why the connection fails. The real problem is just that: the logs don't print that the server is closed, and the connection through the forwarding fails because the end point is closed.

Cases like yours only represent 1%. But I'm glad you could work it out. 😉

@RyanEwen
Copy link

As multiple people suggested, the problem was, in fact, that TCP forwarding was blocked. I tried setting AllowTcpForwarding=yes in /etc/ssh/sshd_config and restarting ssh, but that didn't work. It turned out that -o AllowTcpForwarding=no was present in /etc/default/ssh, which I didn't know to check. Once I removed that option and restarted ssh, everything worked.

Tried this on Centos 6 (WHM/cPanel server) without any luck.

@Chuxel
Copy link
Member

Chuxel commented May 29, 2019

@RyanEwen CentOS 6 is due to outdated glibc and libstdc++ libraries in the distribution. That's being discussed in #103.

@LiamMahoney
Copy link

LiamMahoney commented Jun 20, 2019

Wondering if anyone has any suggestions/best practices concerning allowing TCP Forwarding? The environment I'm working in has set AllowTcpForwarding to NO, and everyone I've reached out to seems hesitant on changing that setting. I've been looking into using the Match User <username> within sshd_config to limit TCP forwarding to my specific user which worked fine in testing. However, it was also suggested to me that I should be using the PermitOpen rule to limit the IPs and ports that TCPForwarding would work on. I haven't been able to figure out how to properly configure that setting to get my remote connection to work. Any suggestions?

Here's an example of my sshd_config:

...
AllowTcpForwarding no
Match User <my_username>
     AllowTcpForwarding yes
     PermitOpen <ip_addr:port> #haven't been able to figure out what ip/port combo will work

@davidwin
Copy link

See if you can get acceptance for setting AllowTcpForwarding to local, which adds very little risk since you have shell access already. At least for now it is sufficient to get VS Code working.
If you're lucky, your IT department will see the value in developers using VS Code this way and will roll out a configuration change. It worked for me. :)

@mrseanryan
Copy link

in case it helps anyone:

I add similar issue, with error like this:

Waiting for ssh tunnel to be ready
    Tunneling remote port 42089 to local port 9456
    Resolving "ssh-remote+redacted.machinename.com" to "localhost:9456", attempt: 1

I ran this command in Visual Code: F1 -> Remote-SSH: Kill VS Code Server on Host...

Then re-connecting in Visual Code worked OK.

It looks like the Visual Code host on server either got in a hung state OR could not be updated ....

@vscodebot vscodebot bot locked and limited conversation to collaborators Jul 10, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
info-needed Issue requires more information from poster ssh Issue in vscode-remote SSH
Projects
None yet
Development

No branches or pull requests