Setup autossh script on Raspbian for Raspberry Pi to connect the Raspberry Pi from the Internet via reverse ssh tunnel

0. Introduction

This is not my first time using reverse ssh tunnel to expose the NAT server (Ubuntu and Centos) to the Internet. But when I tried to use my previous experience on Raspberry Pi (Pi3), I was frustrated that I failed so many times.

Now that I have solved all the problems I have encountered and would like to write it down for the future me or someone else who gonna use the same function next time.

1. Prerequisites

  1. A vps accessible from the Internet
  2. The Pi3 (My Pi3 is headless, using Raspbian)

2. Reverse ssh tunnel

2.1. Preparation on VPS

If you have root access to the vps, the best practice is to add a new user for the Pi3 and prevent the new user from using the bash.

$ su
$ adduser vps_user
$ mkdir /home/vps_user/.ssh
$ chown -R vps_user:vps_user /home/vps_user/.ssh

Change vps_user.

2.2. Operations on Pi3 - upload key

To make sure the Pi3 can reverse ssh to the vps without password, a key should be generated. Besides, there are two kinds of reverse ssh tunnel, one is that you need to login to the vps first then you can login to the Pi3, the other is that you can directly login to the Pi3 but you will expose the ssh port of Pi3 to the Internet

pi $ sudo ssh-keygen
pi $ sudo vim ~/.ssh/config
host    #vps_ip
port xxx    #vps_ssh_port
pi $ scp /home/pi/.ssh/ [email protected]:/home/vps_user/.ssh/

Change, xxx, vps_user.

2.3. Operations on Pi3 - Login to the vps as vps_user

[email protected] $ ssh -p xxx [email protected]
[email protected] $ sudo cat >> /home/vps_user/.ssh/authorized_keys

#(Optional) If you want to expose the port of Pi3 to the Internet, then do the following:
[email protected] $ vim /etc/ssh/sshd_conf
GatewayPorts clientspecified    #add this line
[email protected] $ service ssh restart

#Allow the port for reverse ssh
[email protected] $ iptables -A INPUT -p tcp -m tcp --dport vps_port -j ACCEPT

#Prevent the vps_user from using the bash on vps
[email protected] $ usermod -s /bin/false vps_user

Change, xxx, vps_user, vps_port.

2.4. Operations on Pi3 - initiate reverse ssh tunnel

There are two options as mentioned before:
option 1: the Pi3 can only be connected locally from the vps
option 2: the Pi3 can be connected from any other host from the Internet

#option 1
pi $ ssh -fN -R vps_port:localhost:pi3_port [email protected]_ip

pi $ ssh -fN -R vps_ip:vps_port:localhost:pi3_port [email protected]_ip

#see if the Pi3 can be connected
#option 1
[email protected] $ ssh -p vps_port [email protected]
#opthon 2
[email protected] $ ssh -p vps_port [email protected]_ip

Change vps_ip, vps_user, vps_port.

3. Startup script for autossh on Raspbian

The above mentioned reverse ssh connection is not stable and cannot restart after disconnection or reboot. So it is necessary to use autossh to make sure that the reverse ssh tunnel is alive and to reconnect the vps if not alive.

Notes: I tried to write the following command under /etc/rc.local, but it doesn't work after reboot:

autossh -M pi3_checking_port -fN -o "PubkeyAuthentication=yes" -o "StrictHostKeyChecking=false" -o "PasswordAuthentication=no" -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -R vps_ip:vps_port:localhost:pi_port -i /home/pi/.ssh/id_rsa [email protected]_port

Here is the log from /var/log/syslog:

Aug 25 10:28:23 raspberrypi autossh[723]: starting ssh (count 1)
Aug 25 10:28:23 raspberrypi autossh[723]: ssh child pid is 725
Aug 25 10:28:28 raspberrypi autossh[723]: received signal to exit (15)

The signal (15) means terminate the process. I don't understand why it works perfectly on my Ubuntu server but doesn't work here. I tried adding sudo, or use the key from the root -i /root/.ssh/id_rsa, or disable the iptables, but none of these worked. I know there's something different from debian and Ubuntu, and I guess I am currently facing one of the differences. So I am writing a startup script instead of using rc.local.

3.1. Create the startup script

$ sudo apt-get insall autossh
$ sudo vim /etc/init.d/    #Create the new file with a name you like
#Insert the following scripts

# Provides:          autossh
# Required-Start:    $local_fs $remote_fs $network $syslog
# Required-Stop:     $local_fs $remote_fs $network $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: starts the autossh
# Description:       starts the autossh

case "$1" in
    echo "start autossh"
    sudo /usr/bin/autossh -M pi3_checking_port -fN -o "PubkeyAuthentication=yes" -o "StrictHostKeyChecking=false" -o "PasswordAuthentication=no" -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -R vps_ip:vps_port:localhost:pi_port -i /root/.ssh/id_rsa [email protected]_port
    sudo killall autossh
    sudo killall autossh
    sudo /usr/bin/autossh -M pi3_checking_port -fN -o "PubkeyAuthentication=yes" -o "StrictHostKeyChecking=false" -o "PasswordAuthentication=no" -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -R vps_ip:vps_port:localhost:pi3_port -i /root/.ssh/id_rsa [email protected]_port
    echo "Usage: $0 (start|stop)"
exit 0

#Then save the file

Just need to change the following parameters: pi3_checking_port, vps_ip, vps_port, pi3_port, and vps_user.

3.2. Enable the script

$ sudo chmod a+x /etc/init.d/    #make the script executable
$ sudo update-rc.d defaults    #create a link under "rc0.d" to the script
$ sudo update-rc.d enable    #execute when startup

4. Troubleshooting

4.1. To see if the script work:

$ sudo /etc/init.d/ start
$ sudo /etc/init.d/ stop

4.2. To see if there is a link to the script:

$ ls /etc/rc0.d
$ ls /etc/rc1.d
$ ls /etc/rc2.d
$ ls /etc/rc3.d
$ ls /etc/rc4.d
$ ls /etc/rc5.d
$ ls /etc/rc6.d

4.3. To disable it:

#Remove the script's link under /etc/rc0.d
sudo update-rc.d /etc/init.d/ remove
#Enable of disable the script
sudo update-rc.d /etc/init.d/ enable|disable

4.4. The ssh hangs when the Pi3 is using wifi

debug2: channel 0: open confirm rwindow 0 rmax 32768
#after a long time...
Write failed: Broken pipe

Currently no way to solve it. If you know how, please feel free to let me know.

Change to LAN instead of wifi. After all the wifi on the raspberry pi shares the I/O with the USB and is much slower than LAN.