SSH Agent Forwarding

[dt]

Would be great to be able to forward local SSH agents. Seems to ignore ForwardAgent yes in ~/.ssh/config

[jlynn]

👍 I can't use mosh for this very reason. I need to be able to connect to other machines in the environment I'm accessing plus use my ssh key to push/pull code from github.

[peterjeremy]

This requires tunnelling the agent request from the server back to the client.

[phmarek]

I'd be interested in this feature, too.

[dt]

If #41 ends up being resolved be adding tunnels for arbitrary octet streams, agent forwarding should be trivial to add on top of that.

[thrillcall]

+1 for being able to ForwardAgent. In our case also critical for getting code to/from GitHub on remote servers.

[dbanck]

+1 I need the agent for sudo authentication.

[clneagu]

+1 mostly for github

[avleen]

+1 also!

[stianeikeland]

👍 .. this would be very useful

[nostra]

+1

[plathrop]

+1

[plathrop]

Hard to resist the urge to say nothing but '+1' to @Mkaysi's comment...

[keithw]

All right, folks, the message has been heard loud and clear! We will put this on the roadmap for a release, probably the one immediately after 1.2. No need to keep +1ing - we will do it.

[Hidendra]

Any word on if this is still planned for 1.2 or near/after 1.3?

[chekhovs1]

+1 from me. I naively thought that this was implemented and after building mosh at the server systems and running it, realized that this is not quite ready yet. Any updates on the timeline?

[kergoth]

+1 for me as well, this is a blocking issue for me, as I do remote development, using local keys to communicate with upstream SCM repositories.

[ogrisel]

+1 mostly for being able to work with github when doing remote development from EC2 machines.

[tsuna]

Sorry for spamming everyone, GitHub doesn't appear to have a feature to get notifications of changes on an issue without +1'ing it. So +1 for me too.

[andreypopp]

@tsuna see at the bottom of any issue page

[Gonzih]

+1 for agent forwarding

[ghost]

Came to report another issue, saw this. Key forwarding is absolutely required to my workflow and I am seriously concerned about the way the mosh devs handle ssh keys if it wasn't baked in from the beginning.

[keithw]

Mosh works fine with SSH keys (or any SSH login method), since we just use SSH itself to handle authentication.

We don't allow the user to forward ssh-agent requests because, frankly, nobody has contributed the code and we haven't gotten around to it.

But in general, ssh-agent forwarding is inappropriate for high-security environments. There is little to stop a compromised remote host from requesting your credentials to do 'rm -rf /' (or modify authorized_keys) on a bunch of remote hosts. The user doesn't find out what the remote host wants to do with your credentials; the ssh-agent protocol just lets a process use your key to sign any request. If you forward ssh-agent, then it lets a remote host use your key to sign any request. It's almost the same security characteristics as just copying your credentials to the remote host in the first place -- but not quite, since the remote host loses access to your credentials after you log out (unless it uses them in the meantime to insert its evil key in your authorized_hosts file elsewhere).

I am surprised by all the people saying we need this because they are security freaks. You don't forward ssh-agent if you are a security freak. ssh-agent forwarding is a convenience feature that should not be on by default and should only be used when connecting to semi-trusted hosts.

@apeiron, no need for concern. A process that does ssh-agent forwarding just takes bytes from the remote host and ferries them to the local host's ssh-agent and similarly for the reply. It doesn't touch secret data. All it does is make remote requests to the ssh-agent look like they are coming from the local host. For the reasons above, that's a risky thing to allow, but actually making it happen is easy.

[maxcom]

@keithw ssh-agent has confirmation feature that can protect you from that scenario when remote host does 'rm -rf /' on other remotes.

ssh agent forwarding is really good for accessing git repository from development and testing servers. Also it is good for proxying ssh connections through other hosts when there is no direct connection.

[keithw]

Tell me more about this confirmation feature. My understanding is that there is no way for the agent (and therefore the local user) to learn what command the remote host intends on executing once it is able to authenticate. All the protocol allows is for ferrying public-key challenges and the associated responses.

[maxcom]

Yes, you can not controll what command is executed. But you can control
what hosts are involved.
31.10.2012 9:54 пользователь "Keith Winstein" notifications@github.com
написал:

Tell me more about this confirmation feature. My understanding is that
there is no way for the agent (and therefore the local user) to learn what
command the remote host intends on executing once it is able to
authenticate. All the protocol allows is for ferrying public-key challenges
and the associated responses.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/120#issuecomment-9933956.

[keithw]

I don't think that's right. The ssh-agent only learns which identity (key pair) the requesting process is trying to use. It doesn't learn which host the request came from (remember, forwarding makes the request appear to be coming from the local machine) and it doesn't learn which remote server that process is trying to connect to, or what command the process is trying to execute on the remote server. Happy to be corrected though if my understanding is off.

[maxcom]

Yes, it seems that you are right. So the better way to improve ssh-agent security is using different keys for different groups of servers. Anyway, that is generic weakness of single sing on systems. SSH has agent forwarding feature and it will be nice to have it in mosh.

[tsuna]

I think for most people waiting on this issue, this isn't as much about conveniency as it is about security.

[rinne]

There already is a fork that loosely follows the master branch of upstream mosh + agent patch. I won't commit myself to maintain it for the rest of my life though.

https://github.com/rinne/mosh

[bukzor]

@rinne Yes, this is about motivating upstream to take this seriously. There's actually two bounties now. A smaller one for this pull request in particular, and a larger for the original feature request.

[Apreche]

Just discovered Mosh today. Was excited to try it out. Discovered that Agent Forwarding didn't work. Connected to my server, and everything was great. Then I tried to git clone and realized there was no agent forwarding. Without that feature Mosh is completely useless to me. Saw that this issue is from 2012. That is very disappointing. Guess it's back to trusty old SSH for me.

[HaleTom]

If we have a working patch, what exactly is preventing the merge?

1. Enough time
2. Code quality
3. Design decisions
4. Other

What is blocking this so we can be clear about resolving it or working around it?

[rinne]

@HaleTom Lack of interest from the upstream developers' side.

[keithw]

Please see mailing list discussion on this (https://www.mail-archive.com/mosh-devel@mit.edu/msg00980.html).

Excerpt:

The issue is basically the same since the original pull request in 2013 -- whatever change we make to the Mosh protocol to support ssh-agent forwarding is one we have to live with forever, and the limitations of the Mosh protocol make us not want to commit ourselves to these changes. Mosh does not handle big Instructions well; our fragmentation system is very simple, so adding reliable transport of not-exactly bounded OOB data in the synchronized SSP object makes me nervous.

There is some good news: we think we've figured out a way to do secure ssh-agent forwarding (something that could be enabled all the time, for any server, without the stern warning that accompanies ssh -A) and with a prototype that doesn't require changing the core Mosh protocol. Please stay tuned...

[rinne]

@keithw

we think we've figured out a way to do secure ssh-agent forwarding

Would you mind sharing?

[HaleTom]

I read the mailing list discussion on this and would appreciate anyone subscribed to that list posting here when the following tool is made available:

We have developed something internally (at Stanford) that you might
like that also does "secure" ssh-agent forwarding, by allowing the agent to
authenticate and limit (1) the host making the request, (2) the remote host
that host is trying to authenticate to, and (3) the command the host wants
to execute on the remote host. (With normal ssh-agent forwarding, the agent
can't learn any of these things and is basically signing a blank check.)
This works alongside SSH and Mosh. We hope to have a public beta soon and
will look forward to reports from anybody who wants to test it.

[HaleTom]

Hmm, is there any work-around for this issue until @keithw 's good news gets implemented (fingers crossed).

[rinne]

@HaleTom

We have developed something internally (at Stanford) ...

In the dawn of time when there was a standardisation effort for ssh protocol, there was also some new proposals for agent protocol. However, the protocol has remained much the same and there is no real standard for that. The pretty obvious extension that would allow policy based decisions in agent, would be a "forward notification" packet that all clients would add to the forwarded agent connection (towards the agent) and include the information of the endpoint where that particular client is connected. That would mean that agent receiving an agent connection would get 0..n forward notification packets before anything else and could reconstruct the forwarding path before deciding whether to allow signatures and which keys or certificates to show etc.

Basically the user could then limit the agent forwarding to the systems he trusts. And the first level of forward notifications, would only need support in the client the user is using first. Server support is never needed.

Anyways, this is not mosh specific.

[keithw]

Hi folks,

Here is our prototype for secure agent forwarding that works with Mosh (or SSH): Guardian Agent.

Compared with traditional ssh-agent forwarding, Guardian Agent provides more-constrained agent forwarding that we think could safely be enabled on any connection. Like Mosh, this tool has to be installed on your local machine and on any host you want to Mosh to. It does not require anything to be installed (or any changes at all) on remote servers (e.g. GitHub) that you want that host to be able to SSH to using forwarded credentials.

This approach also does not require changes to the underlying Mosh protocol to carry ssh-agent request data within the interactive terminal, and we don't have to commit to (and support) a protocol for carrying reliable agent traffic between mosh-clients and mosh-servers. Right now, Guardian Agent works alongside any version of Mosh or SSH. Users run sga-guard (the agent) on their local machine, in a separate window alongside the interactive session. sga-guard prompts the user to approve forwarded ssh requests from the intermediary host, either with an X11 popup or in that second terminal window. Based on feedback to this beta/prototype, we may incorporate the techniques behind Guardian Agent more deeply into Mosh.

There is a more detailed writeup in the README. Please let us know what you think, both about the usability of the tool or about the underlying mechanism. Please file any issues with the tool as GitHub issues in the Guardian Agent repository. We're looking forward to everybody's feedback.

[sbz]

+1
Same for me it doesn't work on version mosh 1.3.2 [build mosh 1.3.2]. I tried the following:

mosh --ssh="ssh -A" <server>
mosh --ssh="ssh -o ForwardAgent=yes" <server>

I can see the SSH_AUTH_SOCK variable is exported in the environment, but the agent forwarding doesn't seems to work.

SSH agent forwarding is a very nice feature, I would be happy to see that feature supported by mosh.

Description of the agent protocol can be found

• IETF RFC draft from the secsh group draft-ietf-secsh-agent-02
• OpenSSH PROTOCOL.agent from djm@ draft-miller-ssh-agent-02

[keithw]

@sbz Something like that is not going to work -- Mosh only uses SSH to authenticate and remotely invoke the mosh-server program. The SSH connection ends and is replaced with Mosh end-to-end. There's no SSH around to forward the agent requests.

If you want ssh-agent forwarding, use Guardian Agent! It works (and the design is more secure than normal ssh-agent forwarding, keeping in mind that Guardian Agent is beta software and needs the experience of people trying to break it and criticize the design).

[rinne]

I am very skeptic about a new agent implementation that requires a script wrapper around ssh is going to see any wide audience adoption. And an agent is already integrated to default sessions in most Linux distros and MacOS.
To that note, I've once more merged the current mosh master branch to my agent forwarding enabled version of mosh at https://github.com/rinne/mosh

[sbz]

@keithw Ok thank for your reply and clarification. I will try Guardian Agent!

@rinne I will checkout your branch, looks like the good commit to watch is rinne/mosh@dfa183d

[tnguyen14]

@keithw appreciate all the work you and your team put into mosh and Guardian Agent. I think it's great that there's a more secure option for agent forwarding. However, I agree with @rinne that wide adoption is rather unlikely, at least in the short term.

Now that there's a supposedly better solution for agent forwarding, I think mosh should also add support for the builtin agent forwarding, and let the users decide which one they want to use. I think it's detrimental to mosh to not merge #696. Clearly there is a lot of interest and support for that feature. Let the users decide which alternatives they prefer.

[webbp]

I would like this feature, too, but a <30 second workaround for connecting to a remote server and then accessing github is to just generate a new ssh key and add it to github, e.g.,
ssh-keygen -t rsa -b 4096 -C "your@email.com"
hit return several times
cat ~/.ssh/id_rsa.pub
copy it, go here: https://github.com/settings/ssh/new, paste

[rdnetto]

From the README:

Q: Is Guardian Agent secure?

A: Guardian Agent is a beta that was first released for beta testing in October 2017. It has not accumulated enough testing and scrutiny to make claims that the implementation is bulletproof.

IMO, this is a pretty strong signal that we should not use Guardian Agent for security-conscious environments, which are precisely the ones where we'd prefer agent forwarding over simply storing a key on the remote host.

I appreciate the desire to avoid the issues associated with agent forwarding, but until Guardian Agent is considered stable and secure, I don't think it's a viable alternative.

[keithw]

:-) We had a pretty similar disclaimer for the first few years of Mosh, and the results speak for themselves. There's no point in boasting about security before we have a track record. But I think the idea is that if you all try Guardian Agent and the feedback is good, we'll integrate it more tightly into Mosh. That sort of solves the problem of adoption.

[keithw]

In general, I know it's frustrating to have a PR sitting around and not get merged, but it's not just a question of the implementation -- we really don't want to complicate the protocol in a way that we then have to support forever going forward (especially by bolting it into the object that gets synchronized by SSP, which is unfortunately a bit of a kludge -- not Timo's fault, it's just that SSP is not very good at this use case).

If you look at where the security holes come from in OpenSSH and OpenSSL, it's largely from added complexity that wasn't really necessary. Mosh has never had a significant security hole (yet -- knock on wood) and I don't think that's entirely luck -- it also comes from saying no to things that add complexity to the protocol in a way that makes it harder to reason about.

[rinne]

@keithw, just please mark the protobuf message numbers used in agent forwarding patch as reserved in upstream master, so they are not accidentally recycled later. Apart from that, I'm pretty much beyond caring.

[keithw]

Sounds pretty reasonable -- you want Mosh to keep tag 8 (your oob) reserved in TransportBuffers::Instruction?

[rinne]

@keithw "Sounds pretty reasonable -- you want Mosh to keep tag 8 (your oob) reserved in TransportBuffers::Instruction?"

Yes, please. As far as I understand, that should be enough to never break interoperability with agent enabled client vs non-enabled server and vice versa.

[xhalo32]

+1 add agent forwarding

[zviratko]

This discussion is a bit ridiculous.
It's true that enabling ssh-agent (without confirmation) is a security risk and could be considered bad practice.
But tons of people are doing it, using it and without it, mosh is usable just for their IRC screen hosts (like me).
To install it in an enterprise and use it to manage your servers, you usually need ssh-agent support because sooner or later you'll hit a script that needs it (for git, automation, whatever).

mosh is either a supposed SSH replacement intended for use instead of SSH everywhere possible (in which case it will need to support all the silly stuff as well) or it is just a wrapper that makes roaming easier for few hosts (in which case you might argue improving security is a thing).
I say let the users decide.
I believe mosh security is on-par if not better than SSH with all it's legacy bloat, which should be one more reason to make it possible for people to actually use it as SSH replacement = ssh-agent is a prerequisite of that (and for the record I believe port forwarding as well)

[keithw]

Hi folks -- I'm going to lock the conversation here because it doesn't seem to be productive anymore. We are not making the protocol changes proposed in Timo Rinne's pull request, for the reasons we've explained earlier here: we really don't want to complicate the protocol in a way that we then have to support forever going forward (especially by bolting it into the object that gets synchronized by SSP, which is unfortunately a bit of a kludge -- not Timo's fault, it's just that SSP is not very good at this use case).

If you look at where the security holes come from in OpenSSH and OpenSSL, it's largely from added complexity that wasn't really necessary. Mosh has never had a significant security hole (yet -- knock on wood) and I don't think that's entirely luck -- it also comes from saying no to things that add complexity to the protocol in a way that makes it harder to reason about.

If you want ssh-agent forwarding with Mosh, please use Guardian Agent and provide feedback if you have it: https://github.com/StanfordSNR/guardian-agent

It works and I use it every day! If feedback is good, we may consider integrating Guardian Agent more closely into a future version of Mosh. It's true that Guardian Agent is newer and less-tested software, but, so was Mosh when it was younger. The only way it will be more-tested is if more people use it. And, we think it is a lot more secure than "uncontrolled" ssh-agent forwarding.