HPN-README

Notes:

LibreSSL Support:
Changes in LibreSSL version 3.5 and 3.6 prevent the use of the threaded AES CTR cipher.
In those cases HPNSSH will fallback to the serial version of the AES CTR cipher. A warning
is printed to stderr.

Automatic Port Fallback (in version 17v3)
The hpnssh client now uses TCP port 2222 to connect automatically as this is the
default hpnsshd port. However, we understand that many users will be end up connecting
standard SSH servers on port 22. To make the easier for users the client will fall back to
port 22 in the event that there is no hpnssh server running on port 2222. The behaviour can
be modifed as follows:
-oFallback=[yes|no] will enable or disable port fallback. Default is yes.
-oFallbackPort=[N] where N is the port number that should be used for fall back.
    Default is 22.

TCP_INFO Metrics
This features allows the client to request tcp networking information from the
TCP_INFO struct. This includes data on retransmits, round trip time, lost packets,
data transferred, and the like. The metrics are polled periodically through the
life of the connection. By default this is every 5 seconds but users can pick different
polling periods. The resulting data is stored in two distinct files; one for local
metrics and one for remote metrics. Remote metrics are only available if the remote
supports this feature. This feature will *not* diagnose a poorly performing connection
but may provide insight into what is happening during the connection.

Usage:
-oMetrics=[yes|no] will enable metrics polling. Default: No.
-oMetricsInterval=[N] where N is the polling period in seconds. Default: 5 seconds.
-oMetricsPath=[/filepath/filename] is the name of the file where the remote and
    local data will be stored. Default: ./ssh_stack_metrics.[local|remote].
    Any other option chosen by the user will have a .local or .remote suffix appended to it.

The number of instruments polled by this features is dependent on the kernel of the host.
This means that a remote host with an older kernel may report fewer instruments than a client
host running a current kernel or vice versa. If there is a discrepency in the number of instruments
in the ssh_stack_metrics.local and .remote file this is the most likely reason.

Additionally, determining which file represents sender side versus receiver side is dependent
on the nature of the connection. Therfore, it's up to the user to make that determination.

Linux Note: Currently this is only supported on Linux kernel versions 3.7 and greater. Newer kernels
may have more instruments available to poll than older kernels.

FreeBSD Note: This is supported on Release 6 and higher. However, FreeBSD has fewer available
instruments than new Linux kernels.

Multiplexing Note: The metrics are reported from the TCP socket which means that gathering
metrics from multiplexed sessions will report on the activity of all sessions on that socket.
This will likely result in less clear results and, as such, we suggest only gathering metrics
from non-multiplexed session.

HPNSCP with Resume functionality
This feature allows hpnscp to resume failed transfers. In the event of a failed transfer
issues the same scp command with the '-R' option. For example - if you issued:
'hpnscp myhugefile me@host:~'
and it dies halfway through the transfer issuing
'hpnscp -Z myhugefile me@host:~'
will resume the transfer at the point where it left off.

This is implemented by having the source host send a hash (blake2b512) of the file to the
target host. The target host then computes it's own hash of the target file. If the hashes match
then the file is skipped as this indicates a successful transfer. However, if the hashes do not
match then the target sends the source its hash along with the size of the file. The source then
computes the hash of the file *up to* the size of the target file. If those hashes match then
the source only send the necessary bytes to complete the transfer. If the hashes do not match then
the entire file is resent. If the target file is larger then the source file then the entire
source file is sent and any existing target file is overwritten.


MULTI-THREADED AES CIPHER:
The AES cipher in CTR mode has been multithreaded (MTR-AES-CTR). This will allow ssh installations
on hosts with multiple cores to use more than one processing core during encryption.
Tests have show significant throughput performance increases when using MTR-AES-CTR up
to and including a full gigabit per second on quad core systems. It should be possible to
achieve full line rate on dual core systems but OS and data management overhead makes this
more difficult to achieve. The cipher stream from MTR-AES-CTR is entirely compatible with single
thread AES-CTR (ST-AES-CTR) implementations and should be 100% backward compatible. Optimal
performance requires the MTR-AES-CTR mode be enabled on both ends of the connection.
The MTR-AES-CTR replaces ST-AES-CTR and is used in exactly the same way with the same
nomenclature.
Usage examples:
		ssh -caes128-ctr you@host.com
		scp -oCipher=aes256-ctr file you@host.com:~/file

MULTI-THREADED ChaCha20-Poly1305 CIPHER:
The default cipher used by HPN-SSH is now a threaded implementation of the
ChaCha20-Poly1305 cipher. In tests this cipher results in an approximately 90% gain in
throughput performance in comparison to the serial implementation found in OpenSSH.
This cipher is fully compatible with all known existing implementations of ChaCha20-Poly1305.

Internally this cipher is referred to as chacha20-poly1305-mt@hpnssh.org (CC20-MT)
similar to how the OpenSSH implementation is known as chacha20-poly1305@openssh.com
(CC20-S (for serial)). No changes are required on the part of the user to make use of CC20-MT.
However, if the users doesn't want to make use of this cipher they can explicitly load CC20-S using
'-cchach20-poly1305@openssh.com` on the command line.

NONE CIPHER:
To use the NONE option you must have the NoneEnabled switch set on the server and
you *must* have *both* NoneEnabled and NoneSwitch set to yes on the client. The NONE
feature works with ALL ssh subsystems (as far as we can tell) *AS LONG AS* a tty is not
spawned. If a user uses the -T switch to prevent a tty being created the NONE cipher will
be disabled.

The performance increase will only be as good as the network and TCP stack tuning
on the reciever side of the connection allows. As a rule of thumb a user will need
at least 10Mb/s connection with a 100ms RTT to see a doubling of performance. The
HPN-SSH home page describes this in greater detail.

http://www.psc.edu/networking/projects/hpn-ssh

NONE MAC:
Starting with HPN 15v1 users will have the option to disable HMAC (message
authentication ciphers) when using the NONE cipher. You must enable the following:
NoneEnabled, NoneSwitch, and NoneMacEnabled. If all three are not enabled the None MAC
will be automatically disabled. In tests the use of the None MAC improved throuput by
more than 30%.

ex: scp -oNoneSwitch=yes -oNoneEnabled=yes -oNoneMacEnabled=yes file host:~

HPN Specific Configuration options

TcpRcvBufPoll=[yes/no] client/server
      Enable of disable the polling of the tcp receive buffer through the life
of the connection. You would want to make sure that this option is enabled
for systems making use of autotuning kernels (linux 2.4.24+, 2.6, MS Vista)
default is yes.

NoneEnabled=[yes/no] client/server
      Enable or disable the use of the None cipher. Care must always be used
when enabling this as it will allow users to send data in the clear. However,
it is important to note that authentication information remains encrypted
even if this option is enabled. Set to no by default.

NoneMacEnabled=[yes/no] client/server
      Enable or disable the use of the None MAC. When this is enabled ssh
will *not* provide data integrity of any data being transmitted between hosts. Use
with caution as it, unlike just using NoneEnabled, doesn't provide data integrity and
protection against man-in-the-middle attacks. As with NoneEnabled all authentication
remains encrypted and integrity is ensured. Default is no.

NoneSwitch=[yes/no] client
     Switch the encryption cipher being used to the None cipher after
authentication takes place. NoneEnabled must be enabled on both the client
and server side of the connection. When the connection switches to the NONE
cipher a warning is sent to STDERR. The connection attempt will fail with an
error if a client requests a NoneSwitch from the server that does not explicitly
have NoneEnabled set to yes. Note: The NONE cipher cannot be used in
interactive (shell) sessions and it will fail silently. Set to no by default.

HPNDisabled=[yes/no] client/server
     In some situations, such as transfers on a local area network, the impact
of the HPN code produces a net decrease in performance. In these cases it is
helpful to disable the HPN functionality. By default HPNDisabled is set to no.

Credits: This patch was conceived, designed, and led by Chris Rapier (rapier@psc.edu)
         The majority of the actual coding for versions up to HPN12v1 was performed
         by Michael Stevens (mstevens@andrew.cmu.edu). The MT-AES-CTR cipher was
         implemented by Ben Bennet (ben@psc.edu) and improved by Mike Tasota
         (tasota@gmail.com) an NSF REU grant recipient for 2013.
	 Allan Jude provided the code for the NoneMac and buffer normalization.
         This work was financed, in part, by Cisco System, Inc., the National
         Library of Medicine, and the National Science Foundation.

Sponsors: Thanks to Niklas Hambuchen for being the first sponsor of HPN-SSH
	  via github's sponsor program!


Edited: October 11, 2023