Creating an internet firewall/router with a Banana Pi

February 8, 2015 · by · in Security

Background

I’ve had a home-made Internet firewall for a long time, but it’s getting old and one day it’s just going to fail, so I decided that it’s time to update it. The old firewall was just a PC running Linux and using iptables to manage the rules. Actually, when I say “old”, I’m not really kidding. The specs of the machine are:

  • Pentium processor running at 150MHz
  • 64M RAM
  • 630MB hard disk
  • Running RedHat 7.1 (released late 2001)
  • Installed and running since April 2002

So, it’s coming up for 13 years old which I think means it’s done pretty well for itself!

Hardware

So I started looking around for some hardware options for a replacement. I looked at some dedicated firewall router boxes, but didn’t really like the functionality (or price). I’m also somewhat concerned about poor security, the possibility of “maintenance” back-doors, and the inability to know exactly what rules are being implemented. The alternative is to do what I did before, which was to implement my own firewall on a general purpose computer. There are some small form factor computers around (for example, Alix 2-3), but then someone at work pointed me at the Banana Pi BPI-R1 router board.  It’s based on the rather well-known Raspberry Pi, but with the following features:

  • dual core CPU
  • 1G RAM
  • 4 LAN, 1 WAN Ethernet connections
  • on-board space for a SATA drive

And all for $145 including a case. That’s cheap enough to buy a second unit to act as a hot spare (I can be a bit anal about things like that…).

So I ordered one and started working out the requirements.

Design thoughts

I could have just installed a dedicated firewall solution like OpenWRT, but in the end I decided to roll my own solution.

The OS resides on a micro SD card, so you need to take some precautions to avoid corrupting it, otherwise a poorly-timed power outage could cause some writes to be lost and the server rendered unbootable. I decided to mount the entire SD card read-only to avoid this scenario. Plus there are obvious security advantages in this – any attack that requires writing something to the filesystem is automatically blocked. It makes maintenance and updates a bit harder, but you can always remount the filesystem in read-write mode temporarily to make the required changes.

The firewall will block all connections by default, with only explicitly-allowed connections allowed through. This is even for outgoing connections, which can be a bit annoying when someone wants to connect to some unusual port outside, but I can live with that.

Access to the firewall needs to be strictly controlled, obviously. The ssh settings are locked down (no root logins, only listen to the internal network interface, only public key authentication is allowed). The only time a password is needed is for sudo access to root from the admin account. System logs are sent off-host to another server for security.

Apart managing connections between the outside and the inside (and DMZ), there are a few basic services that the firewall will need to support:

  • Provision of a caching forwarding DNS server for use by the firewall and internal computers
  • Connection to an external NTP server for time synchronisation
  • A DHCP client to obtain network information from the cable modem
  • An SSH server for access from the inside
  • Relaying of IMAPS connections from the outside to an internal mail server (so I can access my email from outside)

My internal network is 192.168.13.0/24, and the DMZ network is 192.168.14.0/24. The address of the firewall on these networks is 192.168.13.5 and 192.168.14.1 respectively. The external interface is connected to a cable modem which provides a dynamic external address via DHCP.

I don’t currently have any machines to put on the DMZ, but that is likely to change in the future.

Installation process

These are the steps I followed to deploy the firewall.

Basic OS installation and configuration

I connected a monitor to the HDMI port, a USB keyboard/trackpad to the USB port, and a network cable to one of the LAN ports). I downloaded a suitable OS image (mattrix-bananian-14.11-BPI-R1.img) and wrote it to an 8G micro SD card. Note that it is important to get an image that is specific to the BPI-R1, as it includes tools and drivers for the special Ethernet controller. I stuck the SD card in the slot and powered up the board. The installer allows you to do some basic configuration which should be pretty straightforward. The only important thing for me was to make sure it had the right locale installed for me (en_AU.UTF-8).

VLAN setup

Once I could log in using the default password, I set up the network VLANs. Unlike some other machines, all the network interfaces belong to the same device (eth0), with their roles being defined by VLAN tags. Each socket has a port number, viewed from the front they are ordered as follows:

LAN WAN
2 1 0 4 3

I wanted to use the first port for inside, the 4th port for the DMZ and obviously last on is for the outside. The command to manage VLANs is swconfig, and the easiest way to set things is with the configuration file, /etc/network/if-pre-up.d/swconfig:

#!/bin/sh
# port ordering (view from front):
#   [ 2 1 0 4 ] [ 3 ]

ifconfig eth0 up

swconfig dev eth0 set reset 1
swconfig dev eth0 set enable_vlan 1
swconfig dev eth0 vlan 101 set ports '2 8t'
swconfig dev eth0 vlan 102 set ports '1 8t'
swconfig dev eth0 vlan 103 set ports '0 8t'
swconfig dev eth0 vlan 104 set ports '4 8t'

swconfig dev eth0 vlan 201 set ports '3 8t'
swconfig dev eth0 set apply 1

This will create 5 network interfaces: eth.101 (inside), eth.102, eth.103 (both unused), eth.104 (DMZ) and eth.201 (outside).

Next I need to set up the networks for each of the interfaces. This is my configuration file, /etc/network/interfaces:

# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback

# WAN - external
auto eth0.201
iface eth0.201 inet dhcp

# VLAN 101 - internal
auto eth0.101
iface eth0.101 inet static 
    address 192.168.13.5 
    netmask 255.255.255.0

# VLAN 102 - unused
auto eth0.102
iface eth0.102 inet manual

# VLAN 103 - unused
auto eth0.103
iface eth0.103 inet manual

# VLAN 104 - DMZ
auto eth0.104
iface eth0.104 inet static
    address 192.168.14.1
    netmask 255.255.255.0

Now is a good time to reboot and make sure the networking stuff is set up the way I want.

There are a few packages that I  needed to install that aren’t provided. We don’t have a connection to the outside world at this point, so I added a temporary default route via my current firewall and installed the packages:

# route add default gw 192.168.13.6
# apt-get install sudo
# apt-get install tcpdump
# apt-get install unionfs-fuse
# apt-get install bind9

Setting up users and authentication

Next I set up an “admin” user:

# useradd -m -u 510 -s /bin/bash admin
# passwd admin
      (choosing a very secure password)
# su - admin
$ mkdir .ssh
$ chmod 700 .ssh
$ vi .ssh/authorized_keys
      (add public key from a secure internal host)
$ chmod 600 .ssh/authorized_keys

I put generated a dedicated key-pair for access to the firewall, and gave it a passphrase, so that only I could use it.

As I will always be logging in as the admin user, I need to add admin to the sudoers file, using visudo, by adding the following line:

admin     ALL = (ALL:ALL) ALL

Now is a good time to check that you can log into the firewall using the admin account and your private key, and that you can the use sudo to become root (e.g. run “sudo id”) using the secure admin password I  created earlier.

The final step is to lock down ssh. I added /etc/ssh/sshd_config and ensured that it contains the following:

ListenAddress 192.168.13.5
PermitRootLogin no
PasswordAuthentication no

I then told sshd to reload its configuration:

# kill -HUP <pid-of-sshd>

I gave it a bit of a test to  make sure it was all working.

Configure for read-only filesystem

Now I am ready to configure the filesystems for read-only operation.

First, I need to turn off swapping:

# swapoff /swapfile1
# vi /etc/fstab    (removed the swap entry)

Next I  want to set up /var as a memory-resident filesystem using unionfs (I used this pi3g blog entry as the basis for these instructions).

First, I created a unionfs mount script:

# cat > /usr/local/bin/mount_unionfs <<\HERE
#!/bin/sh
DIR=$1
ROOT_MOUNT=$(awk '$2=="/" {print substr($4,1,2)}' < /etc/fstab)

if [ $ROOT_MOUNT = "rw" ]
then
    /bin/mount --bind ${DIR}_org ${DIR}
else
    /bin/mount -t tmpfs ramdisk ${DIR}_rw
    /usr/bin/unionfs-fuse -o cow,allow_other,suid,dev,nonempty ${DIR}_rw=RW:${DIR}_org=RO ${DIR}
fi
HERE
# chmod 755 /usr/local/bin/mount_unionfs

Next, prepare the directories:

# mv /var /var_org
# mkdir /var /var_rw

This configuration moves our current /var to /var_org, and uses this as the static contents of our /var filesystem. It then creates a new empty ramdisk filesystem on /var_rw and then a unionfs which stacks the two and mounts the result on /var. Pretty neat!

Now, I want to make the root and /boot filesystems read-only, and to add a new mount point for /var. The /etc/fstab should look something like this:

/dev/mmcblk0p1 /boot auto ro,noatime 0 2
/dev/mmcblk0p2 / auto ro,noatime 0 1 
none /tmp tmpfs defaults 0 0 
mount_unionfs /var fuse defaults 0 0

I reboot the firewall and verified that everything is read-only, except for /var. Now is a good time to clean up any unwanted stuff from the static /var_org filesystem.

From now on, whenever I want to make any changes I will need to temporarily remount the root filesystem in read-write mode:

$ sudo mount -o remount,rw /

and to go back to read-only mode:

$ sudo mount -o remount,ro /

I remount the filesystem as read-write, as I still have some configuration to do.

Off-host logging

I wanted syslog to redirect all messages to another machine, which I’ll call “loghost”. The logging machine uses rsyslogd, so I do the following on the logging server:

Created a directory where the remote logs will be written:

# mkdir /var/log/firewall

Added the following to /etc/rsyslog.conf:

#### RULES ####

#
# Process logs from firewall
#
$Ruleset remote
kern.error  /var/log/firewall/iptables
user.*      /var/log/firewall/user
*.*         /var/log/firewall/messages

$InputTCPServerBindRuleset remote
$InputTCPServerRun 514

#
# Process local logs
#
$Ruleset RSYSLOG_DefaultRuleset

# [existing rules follow...]

I also want these logfiles to be rotated, so I added a new logrotate script, /etc/logrotate.d/syslog-firewall:

/var/log/firewall/messages
/var/log/firewall/iptables
/var/log/firewall/user
{
    sharedscripts
    postrotate
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

I then told rsyslog to reload its configuration:

# kill -HUP `cat /var/run/syslogd.pid`

Now I am ready to redirect logging on the firewall itself. I added the following entry to my /etc/hosts files for the logging host:

192.168.13.22    loghost

I then edited /etc/rsyslog.conf, and replaced everything below the “RULES” comment with:

*.*    @@loghost

Next I told rsyslogd on this host to reload its configuration. I checked that log entries are now being written on the remote logging host (a good tool for this is the “logger” command).

Setting up the name server

I want the firewall to provide a name server for the use of all internal hosts. My ISP provides IP addresses that I can forward my DNS queries to, so I will use them. This makes this a caching, forwarding server configuration.

I rewrote the /etc/bind/named.conf.options file:

acl trusted {
        192.168.13.0/24;
        192.168.14.0/24;
        localhost;
};
options {
        directory "/var/cache/bind";

        recursion yes;
        allow-query { trusted; };

        forwarders {
                61.9.211.34;
                61.9.211.2;
        };
        forward only;

        //========================================================================
        // If BIND logs error messages about the root key being expired,
        // you will need to update your keys.  See https://www.isc.org/bind-keys
        //========================================================================
        dnssec-enable yes;
        dnssec-validation yes;

        auth-nxdomain no;    # conform to RFC1035
        listen-on-v6 { any; };
};

Finally, I created a directory for the name server cache:

# mkdir /var_rw/cache/bind

Firewall rules

The final major step was to create the firewall rules. A full description of setting up iptables is more than I want to cover here, but as an example, I have made my scripts available on GitHub:

https://github.com/rolfeb/system-firewall

Feel free to have a look and make whatever use of them you like. Note the script in the init.d folder; you will need to install that into the system init.d directory:

$ sudo cp ~admin/firewall/init.d/firewall.sh /etc/init.d
$ sudo update-rc.d firewall.sh defaults 12345 06

At this point I rebooted your firewall once again and verified that all was well. Note that the name server won’t work until it is connected it to the internet connection.

Deploying the firewall

At this point I’m pretty much ready to deploy your firewall. This is always a bit of an exciting task 🙂 , but hopefully nothing will go wrong…

Here is a checklist I went through make sure everything was ok:

  • The firewall rules are in place: sudo iptables -L
  • I can reach the outside: ping 8.8.8.8
  • Name lookup is working: ping www.google.com
  • The time server is syncing: ntpq -p
  • I  can reach the outside from your web browser
  • Entries are appearing in your syslogs (expect to see dropped packets arriving at your external interface)

I will probably want to keep monitoring your firewall logs for a few weeks, looking for anything unusual. If I  find something not able to connect to the outside world, it may be trying to connect to some port that you haven’t enabled – I can check the iptables logs, and if necessary update the firewall scripts and redeploy on the firewall.

Now I just need to back up my SD card and keep it around in case I ever need to recover it. One thing I did post-install was to add an SSD disk, mounted as “/data”. I am using this to collect traffic logs using tcpdump, in case I want  to retrospectively investigate any problems. This filesystem is purely used for non-essential logging.

Links