How to use SSH with 2FA

How to log in to the Maths servers from "outside" machines, with 2FA two-factor authentication.
Some ideas here may be useful without 2FA, e.g. for laptops or office machines within the School.
Internal users connect to enna e.g. ssh -C -X MATHSNAME@enna, not to maths.usyd.edu.au; see also the network setup needed for internal laptops.
To log in to office Windows PCs see the RDP HowTo, into office Macs see the VNC HowTo also (instead?).

Contents

Set up 2FA for your account

2FA two-factor authentication provides a cheap, effective, additional layer of security, similar to popular online services such as Google accounts, or banks.

Choose between Web-OTP, TOTP or skeys.
You can use Web-OTP anytime, without any prior setup;
while TOTP or skeys need to be set up while in the School.
Note that our software prompts for the code first, and the password after that.

To decide whether to use Web-OTP, TOTP or skeys: Our software prompts for the second-factor code first, only then for the fixed password. This is more secure against password guessers: let them have a go at the ephemeral (so useless?) code, before even getting a look at the password. Most other implementations do it wrongly, the other way around.

Recommendations in a nutshell

Follow these recommendations to use 2FA words or codes, and password, just once per day.
For details, rationale, other ways or other things you can do, see later in this document.

Essentials

You will need an ssh client (the ssh command or program); you should also have an X server. After running your ssh command, just follow the prompts: type the words from your skey sheet for the line number shown, or the authenticator code, then your normal enna password. You will be logged in to enna.
The very first time you use ssh, you will be prompted about the as-yet unknown authenticity fingerprint: say yes.

You will have full X-windows access, so could xfrom to other machines, or use other X-windows software like nedit or tuteroll, or licenced software like matlab, mathematica or maple, or even firefox or thunderbird (with full access to scnews and other internal pages or mail files, though these may be too slow over the network); or for a slow but authentic login experience, use xnest or nxagent and in there run /usr/sms/share/ldm/Xsession.

File transfer with scp

As well as ssh, you can use scp (or sftp or sshfs) to copy files between enna and your machine.

Tempting to try some GUI clients ... but mostly they do not understand the 2FA prompt (expecting to send just a plain password) so they fail; they work fine once you save 2FA prompts ... see below under made easy.
Some GUI clients for SFTP:

It may be best to use a tool integrated with the native "file manager":

File transfer made easy

Save 2FA prompts
... (and the "complexity" of using them), use tunneling or port forwarding so you can connect direct to enna (in effect from enna itself, so no 2FA is needed).
Start your enna login session with port forwarding: With your enna login session running, use sftp (or scp or ssh) or rsync or your favourite
GUI client for SFTP, to connect to localhost on port 12022.
See commands to use below, under easy access.
Some (Mac?) machines do not know about localhost, then use 127.0.0.1 instead of the name localhost.
Your initial ssh session needs 2FA, it is subsequent connections that may avoid it.

Save typing your password
... use public keys. Generate some keys and copy things around so you end up having the private key on the laptop and the public key as authorized on enna.
At the initial login with 2FA you still get a password prompt, as configured for security.
Easy for Linux or Mac, not so easy to do on Windows.
You may also use this to ssh/scp between enna and the research servers.
Do not inadvertently publish your private key, e.g. when uploading to web or Git servers.

On enna run commands

    ssh-keygen     (press ENTER twice: to accept filename, and to use empty passphrase)
    ln -s id_rsa.pub ~/.ssh/authorized_keys
then copy the private key file ~/.ssh/id_rsa to your laptop. Easy access
... to your files on enna, while your enna login session is running, with port forwarding in place:
You could use command-line scp:
    scp -P 12022 MATHSNAME@localhost:file-on-enna place-on-local-machine
    scp -P 12022 file-on-local-machine MATHSNAME@localhost:place-on-enna
but there are better ways.
"Look Ma, no hands": no pesky 2FA or password prompts!

Using the ssh-with-2fa script

Using the ssh-with-2fa script
www.maths.usyd.edu.au/u/psz/ssh-with-2fa.txt
is not required, but using it may make things a little easier, as it chooses the "right" options for ssh or scp or sftp. In particular, you could start your enna login session just by typing
    ssh MATHSNAME@enna
or use
    scp file1 MATHSNAME@enna:
    scp MATHSNAME@enna:file2 .
and the "right" things would happen: connect to the right host and port (maybe with 2FA, or 2FA-less via the port forwarding while your enna login session is running), with the "correct" options (all those mentioned above and below), whether your laptop is "outside" or connected to the School's internal network; it will also do the right thing for any other uses, connecting to any other places, not just for "School related".

To use the script:

Then software that uses ssh (e.g. unison or rsync) will also work just fine (without 2FA or even passwords).
Beware unison: version numbers, maybe also OCaml versions, must match for it to work.

Please let Paul know if you find anything that is any less than magical and perfect.
If you have any problems with ssh-with-2fa then run it as ssh --debug MATHSNAME@enna to see more verbose messages.

Port forwardings: IMAP SMTP

Most people now have @sydney email addresses, stored on Office365: see www.maths.usyd.edu.au/u/psz/davmail.html for settings and instructions on how to use.

There may be some use for "local" Maths email services. You may want to (and ssh-with-2fa will) use the port forwarding options

    -L 12143:enna:143 -L 12025:rome:25
to make our internal IMAP server (enna) and SMTP server (rome) accessible, while the enna login session is running. Set your mail client (e.g. mutt, alpine, thunderbird) to use:
   proto   server   port
   IMAP   localhost   12143
   SMTP   localhost   12025
This also solves the issue of finding an SMTP server from home networks like Optusnet or Bigpond that disallow port 25 access.
Seems that first-time IMAP users need to do mkdir -p ~/Mail/.imap for IMAP login to succeed.

You may also want to (and ssh-with-2fa will) set the same port forwardings for "internal" laptop clients connecting ssh to enna, so the mail client configuration does not need to change between internal and external uses.

No POP to rome anymore: was unused anyway.

Port forwardings: CUPS and no LPD

For printing, you may want to (and ssh-with-2fa will) use the port forwarding option
    -L 12631:siv:631
to make our internal CUPS server (siv) accessible, while the enna login session is running. To use, set up CUPS printing as described in
www.maths.usyd.edu.au/u/psz/visitors.html#printing but replacing any mention of siv or siv:631 by localhost:12631.
Printing from outside, when not around to pick up printout, is not much use.

You may also want to (and ssh-with-2fa will) set the same port forwarding for "internal" laptop clients connecting ssh to enna, so the printing client configuration does not need to change between internal and external uses.

Seems tempting to use -L 515:siv:515 for LPD printing. But we cannot use low ports on Linux: maybe not on the laptop for listening, and certainly not on enna for connecting to the LPD server. Setting up local LPD printing would not be trivial; copying the file to be printed with scp and then printing directly from enna may be simpler. Use CUPS printing instead, as above.

Port forwarding: other machines (e.g. savona)

Sometimes it is desired to connect to another server e.g. savona. Use a forwarding option something like
    -L 14022:savona:22
and then (while the enna login session is running) separately connect to localhost, port 14022 (with ssh, scp, FileZilla, WinSCP etc).

The ssh-with-2fa script will not (by itself) use the above forwarding, but you would need to get things started with a command like

    ssh -L 14022:savona:22 MATHSNAME@enna
(or on Windows similar command but with ssh2).

Generally users have passwords set on enna only, not on other servers like savona. If you use public keys as suggested above, that is shared by savona so will work there also; otherwise, if needed, a password could be set on savona.

The BioInformatics people have a page about using RStudio Server though with restricted access.

ProxyJump for other machines (e.g. savona)

Sometimes all you want is to log in to another server e.g. savona, and would like to do in a single command. (The supported way is ssh to enna and then "xfrom savona"; or you can ssh to enna with port forwarding set and then separately ssh to savona, as suggested above.)

Using the ssh ProxyJump option, via ~/.ssh/config (on your laptop) containing:

    Host savona
      HostName savona
      User MATHSNAME
      ProxyJump enna

    Host enna
      HostName maths.usyd.edu.au
      User MATHSNAME
(or some complicated command line) might work. The current Windows putty has a similar Proxy panel feature.

Port forwarding seems more generally useful (e.g. for other servers or for file transfer).

Dynamic forwarding

From outside, you may want to (and ssh-with-2fa will) use the dynamic forwarding option
    -D 13080
so on your laptop (in a terminal, not the ssh-ed one running on enna) you can use e.g. proxychains to access any "internal" services.
See
en.wikipedia.org/wiki/Comparison_of_proxifiers about other similar utilities e.g. for Windows.

To use proxychains (Linux, Mac): on the laptop, edit ~/.proxychains/proxychains.conf (or /etc/proxychains.conf) to contain the lines

    quiet_mode
    [ProxyList]
    socks5 127.0.0.1 13080
(do not use proxy_dns, and with just one line in ProxyList).

On your laptop (in a terminal, not the ssh-ed one running on enna) use commands like

    proxychains command args...
Examples: With dynamic forwarding, name lookups (DNS services) are done locally not via the forwarding, so need to use FQDN names e.g. enna.maths.usyd.edu.au instead of plain enna, on all "local" commands.

Port forwarding: comments

We use ports above 12000 because low ports are root-access-only on Linux, and to avoid clashes with locally running services.

Very tempting to use -L 139:rome:139 then use connect-to-server smb://localhost to access the Samba server on rome (for file access or even printing), but that might not work: on Linux/Mac it requires root access e.g. sudo on the laptop; on Windows, port 139 may be "in use" already.
I could not get it to work on Windows, not even with the tricks in support.blue.net.au/support/tunneling-smb-over-ssh-secure-file-sharing/

Performance

The performance (responsiveness etc) of the connection should depend on the speed of the "home" end of the network: on a fast cable or ADSL connection, should be similar to the speed of a Y-terminal within the School.

Known issues:

NOTE: file:/ links (as above) do not work in Firefox. Copy link location then paste to Firefox URL bar, see kb.mozillazine.org/Links_to_local_pages_don't_work .

See also the section on xpra.

No VPN

We do not even try incoming VPN, as that would require complex and intrusive settings on both enna and on the laptop. For references see:

Idle timeouts

If you allow your computer to disconnect from the network e.g. to go to sleep/hibernate, as most laptops do with the lid closed, then your ssh session will be terminated.

There are timeouts set in several network "appliances":

The above timeouts may all be non-issues, as we have the lines
    TCPKeepAlive yes
    ClientAliveInterval 10
    ClientAliveCountMax 60
in the Maths
/etc/ssh/sskd_config file: we are never much idle, and survive network dropouts of 10 minutes.
BEWARE: this consumes data, about half a kB each 10 secs, 200kB per hour, 4MB per day.
If your computer goes to sleep or hibernate then it will not communicate, and the above settings will cause the session to be terminated in 10 minutes. Without the above settings, the ICT firewall would drop the connection in 5 minutes, instead.

If affected still, you may want to add the -oServerAliveInterval=10 option to ssh (for Linux or Mac, putty has keepalive settings but not for command line); or leave

    while :; do date; sleep 60; done
(rpt 60 date) running.

Random ramblings

For internal laptops etc, please see
www.maths.usyd.edu.au/s/scnitm/psz-Laptops-WirelessAndWiredC
about wired and wireless connections, and
www.maths.usyd.edu.au/u/psz/visitors.html#laptops
about settings and printing.

For incoming connections, ssh or putty talks to the firewall, and only the firewall knows which internal machine the connection is sent to: currently enna.

For outgoing connections we have transparent proxy on the firewall, see
www.maths.usyd.edu.au/s/scnitm/psz-TransparentProxy-noProxyS
You may still want to use the ssh-with-2fa script, to choose sensible options.

When using Cygwin (its ssh and its X server), or maybe from MacOSX, you need the (unsafe) -Y option instead of -X: I guess needed whenever xdpyinfo does not show the SECURITY extension.

Dire warnings (words of Jim Richardson):
Note that skeys are only for use of the person to whom the sheet was allocated, and no forwardings or tunnels other than the above should be used without prior arrangement with the School Computing Manager.

Note for Maths (129.78.68.*, e.g. Magma) users:
You do not need 2FA from such "trusted" hosts.

Our 2FA software is available in directory /usr/sms/etc/2fa (on enna).
NOTE: file:/ links (as above) do not work in Firefox. Copy link location then paste to Firefox URL bar, see kb.mozillazine.org/Links_to_local_pages_don't_work .

This "SSH HowTo" page is referenced from (or referred to in):
www.maths.usyd.edu.au/local.html     (click "Incoming ssh")
www.maths.usyd.edu.au/loc/comp/alpha/net-security.html#incoming
www.maths.usyd.edu.au/u/psz/ssh-howto.html    (here).

Rate limits

On occasions, while trying to log in, you may receive errors like
    Connection closed by remote host
    server unexpectedly closed network connection
    Connection refused
or our Web-OTP service may show "Too Many Requests".

When that happens, try again in a little while. Long story below.

That is our protection against password guessing attacks, in action: we have rate limiting on ssh connections. When that happens, try again in a little while; or maybe wait until the next wall-clock hour, then try; maybe use "ssh -v ..." (or "putty -v ...") to see the "error" message; try soon after the restriction is lifted, before the "bad guys" use up all permitted tries.

For some background, see:
https://isc.sans.edu/diary/Guess+what+SSH+again/6214
https://isc.sans.edu/diary/Dealing+With+Unwanted+SSH+Bruteforcing/7855/
and example log lines from 2011:

    Aug 25 22:17:33 bari sskd: Failed for invalid user aaa
    Aug 25 22:17:48 bari sskd: Failed for invalid user aaron
    Aug 25 22:17:51 bari sskd: Failed for invalid user abacus
    Aug 25 22:17:56 bari sskd: Failed for invalid user abby
Our ssh service is handled by: The limits on connections count both successful and failed logins.

Our Web-OTP service also has limits: 2 per minute for each connecting machine, and a limit of 2 per second or 20 per minute for all connections.

We are pretty safe against any breakins with 2FA; in fact I have never noticed them trying 2FA at all, they just try single passwords. Many try root only as the login name (and root does not have 2FA).

We limit connections to protect against attackers wasting resources, hoping to make the attacker "go away" and try another victim. Our protections have stopped many ssh password guessing runs/attacks, significantly lowering the CPU load on our machines.

Any limits (in xinetd, sshind or sskd) will affect legitimate users also: hopefully our rates and back-off times are not too annoying.

Use xpra

Use xpra to Xpra is independent of (so you would not need) VcXsrv or XQuartz.
Using xpra may speed things up on slow networks like access from "outside",
but may instead slow things down on fast networks like internal ones,
and may introduce its own oddities and bugs.

Instructions:

Enjoy the magic of indirection and subterfuge, one on top of another!

Comments about xpra (and other ways to use), for completeness:

Further reading, random references

en.wikipedia.org/wiki/Perl#Availability
en.wikipedia.org/wiki/Comparison_of_SSH_clients
en.wikipedia.org/wiki/X_Windows#Implementations
https://www.perl.org/get.html
www.openbsd.org/cgi-bin/man.cgi?query=ssh
www.openbsd.org/cgi-bin/man.cgi?query=ssh_config
puttytray.goeswhere.com/
cygwin.com/
x.cygwin.com/
www.starnet.com/products/xwin32/
connectivity.opentext.com/products/exceed.aspx
en.wikipedia.org/wiki/Port_forwarding
help.ubuntu.com/community/SSH/OpenSSH/PortForwarding
expect.sourceforge.net/
www.ora.com/catalog/expect/
blog.ropnop.com/extracting-ssh-private-keys-from-windows-10-ssh-agent/
arstechnica.com/features/2018/09/macos-10-14-mojave-the-ars-technica-review/12/
discussions.apple.com/thread/251226509
www.unix-ninja.com/p/attacking_google_authenticator


Paul Szabo psz@maths.usyd.edu.au 15 Mar 24