sshd on Raspberry Pi

Contents

  1. Background
  2. First step - sshd configuration
  3. Next step - denyhosts
  4. Sample auth.log entries
  5. Solution - Change Port Number

Background

Having enabled sshd on our pi, and forwarded port 22 (ssh) connections to our internet firewall to the pi, the pi fell victim to a significant number of brute force ssh attacks (mainly from IP addresses in china).

Being aware that a ssh attack was a possibility I had already changed the pi password from it's default raspberry to an obscure password containing apparently random characters. Nether the less, I didn't want

  1. the outside risk that a brute force attack might eventually hit on my password string, and
  2. the load associated with the attack heating our pi CPU up, and filling the SD card space associated with the /var/log/auth.log log file.

First step - sshd configuration

My first attempt at defense was to temporarily disable the port 22 forwarding on the firewall, and then make the following additions to the /etc/ssh/sshd_config configuration file

  1. Changed PermitRootLogin from yes to no
  2. Added "AllowUsers pi" to limit ssh connection to use of the pi account
  3. Added "MaxStartups 3:30:10" to limit the load that brute force attacks could put on the pi

After restarting sshd (with "sudo /etc/init.d/ssh restart") and putting the firewall port forwarding back in place, the IP addresses which had been attacking had given up and gone away, so it was a couple of hours before the pi fell under ssh attack again.

Next step - denyhosts

When ssh attacks returned, the pi was again receiving hundreds or thousands of attempts from individual ip addresses, and again putting a significant load on the pi (using CPU which I wanted for other purposes). So my next step was to install denyhosts to greatly limit how much any given IP address could load our pi.

As the manual page says, denyhosts is a python program that automatically blocks ssh attacks by adding entries to /etc/hosts.deny.

So I again temporarily disabled the port 22 forwarding on the firewall and installed denyhosts with the command "sudo apt-get install denyhosts" (after a "sudo apt-get update" of course).

I was impressed that on installing and starting up denyhosts, initial entries were made to /etc/hosts.deny listing the IP addresses which had attacked prior to the installation of denyhosts.

I took the precaution of adding an internal IP address to /etc/hosts.allow to ensure that I could always get to the pi internally, in the event that any of the external IP addresses which I might us might get denied for me by denyhosts.

Sample auth.log entries

The records below are taken from /var/log/auth.log. I grep'ed out the sshd records from the log, and then (manually) extracted just those relating to a single IP address (from China of course). I have added blank lines to the extract below to aid navigation from the discussion below.

May 10 20:32:00 pi sshd[20503]: User root from 116.10.191.206 not allowed because not listed in AllowUsers
May 10 20:32:00 pi sshd[20503]: input_userauth_request: invalid user root [preauth]
May 10 20:32:01 pi sshd[20503]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=116.10.191.206  user=root
May 10 20:32:03 pi sshd[20503]: Failed password for invalid user root from 116.10.191.206 port 15402 ssh2
May 10 20:32:05 pi sshd[20503]: Failed password for invalid user root from 116.10.191.206 port 15402 ssh2
May 10 20:32:08 pi sshd[20503]: Failed password for invalid user root from 116.10.191.206 port 15402 ssh2
May 10 20:32:10 pi sshd[20503]: Failed password for invalid user root from 116.10.191.206 port 15402 ssh2
May 10 20:32:12 pi sshd[20503]: Failed password for invalid user root from 116.10.191.206 port 15402 ssh2
May 10 20:32:15 pi sshd[20503]: Failed password for invalid user root from 116.10.191.206 port 15402 ssh2
May 10 20:32:15 pi sshd[20503]: Disconnecting: Too many authentication failures for root [preauth]
May 10 20:32:15 pi sshd[20503]: PAM 5 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=116.10.191.206  user=root
May 10 20:32:15 pi sshd[20503]: PAM service(sshd) ignoring max retries; 6 > 3

May 10 20:32:26 pi sshd[20509]: User root from 116.10.191.206 not allowed because not listed in AllowUsers
May 10 20:32:26 pi sshd[20509]: input_userauth_request: invalid user root [preauth]
May 10 20:32:26 pi sshd[20509]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=116.10.191.206  user=root
May 10 20:32:28 pi sshd[20509]: Failed password for invalid user root from 116.10.191.206 port 22883 ssh2

May 10 20:32:30 pi sshd[20513]: User root from 116.10.191.206 not allowed because not listed in AllowUsers
May 10 20:32:30 pi sshd[20513]: input_userauth_request: invalid user root [preauth]

May 10 20:32:30 pi sshd[20509]: Failed password for invalid user root from 116.10.191.206 port 22883 ssh2

May 10 20:32:30 pi sshd[20513]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=116.10.191.206  user=root
May 10 20:32:32 pi sshd[20513]: Failed password for invalid user root from 116.10.191.206 port 24539 ssh2

May 10 20:32:33 pi sshd[20509]: Failed password for invalid user root from 116.10.191.206 port 22883 ssh2

May 10 20:32:33 pi sshd[20517]: refused connect from 116.10.191.206 (116.10.191.206)

May 10 20:32:36 pi sshd[20513]: Failed password for invalid user root from 116.10.191.206 port 24539 ssh2

May 10 20:32:36 pi sshd[20509]: Failed password for invalid user root from 116.10.191.206 port 22883 ssh2

May 10 20:32:39 pi sshd[20513]: Failed password for invalid user root from 116.10.191.206 port 24539 ssh2

May 10 20:32:39 pi sshd[20509]: Failed password for invalid user root from 116.10.191.206 port 22883 ssh2

May 10 20:32:41 pi sshd[20513]: Failed password for invalid user root from 116.10.191.206 port 24539 ssh2

May 10 20:32:41 pi sshd[20509]: Failed password for invalid user root from 116.10.191.206 port 22883 ssh2
May 10 20:32:41 pi sshd[20509]: Disconnecting: Too many authentication failures for root [preauth]
May 10 20:32:41 pi sshd[20509]: PAM 5 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=116.10.191.206  user=root
May 10 20:32:41 pi sshd[20509]: PAM service(sshd) ignoring max retries; 6 > 3

May 10 20:32:42 pi sshd[20524]: refused connect from 116.10.191.206 (116.10.191.206)

May 10 20:32:45 pi sshd[20513]: Failed password for invalid user root from 116.10.191.206 port 24539 ssh2
May 10 20:32:47 pi sshd[20513]: Failed password for invalid user root from 116.10.191.206 port 24539 ssh2
May 10 20:32:47 pi sshd[20513]: Disconnecting: Too many authentication failures for root [preauth]
May 10 20:32:47 pi sshd[20513]: PAM 5 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=116.10.191.206  user=root
May 10 20:32:47 pi sshd[20513]: PAM service(sshd) ignoring max retries; 6 > 3

May 10 20:32:49 pi sshd[20527]: refused connect from 116.10.191.206 (116.10.191.206)

May 10 20:32:55 pi sshd[20530]: refused connect from 116.10.191.206 (116.10.191.206)

These log records show the effect of a number of the changes made.

  1. The very first record appears to indicate that the presence of the /etc/hosts.allow might allow ssh connections from hosts listed in /etc/hosts.allow to override the "PermitRootLogin no" directive in /etc/ssh/sshd_config. The second line in the log suggests that rejection would occur due to the attempt to connect to the root account.

    To confirm that "PermitRootLogin no" had not been overridden I attempted a connection from a host listed in /etc/hosts.allow, but, as one might hope, it failed, with the following auth.log entry:

    May 10 22:46:36 pi sshd[20717]: User root from 192.x.x.x not allowed because not listed in AllowUsers
    May 10 22:46:36 pi sshd[20717]: input_userauth_request: invalid user root [preauth]
    May 10 22:46:45 pi sshd[20717]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.x.x.x  user=root
    May 10 22:47:17 pi sshd[20717]: Connection closed by 192.x.x.x [preauth]
    Although the first line above implies the rejection was based on the "AllowUsers pi" directive rather than the "PermitRootLogin no", but either way, the job was done.

  2. The first 12 records, all those for sshd process id 20503 (the records containing "sshd[20503]:") show 5 attempts at connecting to the pi's root account, all rejected by sshd on the grounds that root is not an account that is allowed to connect.

    Then, significantly, the whole TCP connection is closed for "Too many authentication failures for root [preauth]". I think this is due to the "MaxStartups 3:30:10" directive, after the third attempt (the 3 before the first colon), the probability of the connection being closed was 30% (the 30 between the two colons). IP 116.10.191.206 got lucky and had a fourth attempt. The probability of the connection being closed then increased to 40% (linear probability increase up to 100% at the 10th attempt, where 10 is the third colon separated number in the MaxStartups directive), and IP 116.10.191.206 got lucky again and had a fifth attempt. The probability of the connection being closed then increased to 50%, and IP 116.10.191.206 got disconnected.

  3. We then see IP 116.10.191.206 pause for 10 seconds, before attempting multiple connections concurrently. Noteably, after denyhosts has started refusing connections, the two TCP connections already in place at that time continue to be used by IP 116.10.191.206 to exhaust the number of password attempts which each process allows.

Solution - Change Port Number

After three weeks with the above configuration, I decided I'd had enough of folk knocking on my sshd door. Over the period I had an average of just over ten IP addresses added to /etc/hosts.deny each day, and the repeated "refused connect from" records in /var/log/auth.log from certain IP addresses was wasting space in my /var/log filesystem.

So I

  1. added a second port number to /etc/ssh/sshd_config and restarted sshd
  2. replaced port 22 in our firewall Port Forwarding with the new port number

So I can still use port 22 (or the new port number) for ssh within the household, but connections from the internet require the use of the new port number. With just six weeks experience so far, I have received just one sshd attack on the new port number.

Given the "experience" gained with a few weeks of port 22 being exposed to the internet, I am now happy that the measures used with sshd on the non-intuitive port number offer a longer term solution.