Sunday, March 5, 2017

Raspberry Pi for your home network

The Raspberry Pi can provide a great deal of functionality to your home network.  Here are some uses I recommend:
  1. Encrypted DNS
    So your ISP can't log what sites you visit and sell this data to marketers
  2. Ad Blocking
    Significantly speed up browsing for all your home devices
  3. Encrypted FTP
    Securely send and receive large files with friends and business partners
  4. Route 53 Dynamic DNS
    So your FTP address (ftp.domain.tld) always points to your public IP
The Pi also seems powerful enough to run a web server, web application framework and a decent database such as Postgres, but you might be better off hosting that on a t2.micro instance on Amazon EC2.

First, follow my Raspberry Pi 3 Initial Configuration guidelines except I recommend you connect this Pi to your home network using an Ethernet cable rather than use Wi-Fi.  You'll also want to assign it a static IP - outside of the DHCP range of your router.

Encrypted DNS

This consists of two services - DNSMASQ and DNSCRYPT.  DNSMASQ is fully featured DNS server and is required for the Ad Blocking feature as well.  DNSMASQ caches DNS lookups it does through DNSCRYPT so you are not using DNSCRYPT for every lookup.  DNSCRYPT proxies lookups through certain DNS servers on the internet that support DNSCRYPT and claim they will not log your lookups.

The Pi needs to be configured so that its own lookups go through your regular DNS servers - otherwise DNSCRYPT can't resolve the name servers it needs to establish encrypted connections to.

DNSMASQ will be used by other clients on your network that try to use the Pi as a name server. 

Read the Network Filter article on how to set up DNSMASQ and DNSCRYPT on your Pi.  Pay careful attention to the configuration of the resolv.conf and resolv-dnsmasq.conf files. 

The upshot of this is the standard /etc/resolv.conf file used by your Pi (and DNSCRYPT) should only contain nameserver lines pointing to your ISP's nameservers.  I would even remove the 127 address as an option.  There is no way when DNSCRYPT is starting that it will be able to resolve its nameservers when DNSMASQ running on 127 is trying to use DNSCRYPT.  We're not too worried about encrypting the DNS lookups the Pi is making.  We really just want to encrypt the lookups that DNSMASQ is making for your other network clients.  We also want the Pi to be able to set its time via NTP without waiting for DNSCRYPT.

I also deviated from the Network Filter article on how I store and update the dnscrypt-resolvers.csv file and how I select and specify the servers I want to use from that list.

First, download the dnscrypt-resolvers.csv to your PC and open it in Excel.  You want to filter it so that you see only those servers where "No logs" is "yes", the servers are relatively close to you and you want to pick two servers that are not operated by the same organization.  Copy the "name" for each of these two picks over to a text file and use them in place of name1 and name2 in the startup commands for DNSCRYPT in your Pi's /etc/init.d/rc.local:
/usr/local/sbin/dnscrypt-proxy --local-address= --resolvers-list=/home/pi/dnscrypt-proxy/dnscrypt-resolvers.csv --resolver-name=name1 --daemonize

/usr/local/sbin/dnscrypt-proxy --local-address= --resolvers-list=/home/pi/dnscrypt-proxy/dnscrypt-resolvers.csv --resolver-name=name2 --daemonize
Notice /home/pi/dnscrypt-proxy/dnscrypt-resolvers.csv

Since the dnscrypt-resolvers file changes from time to time (servers join and leave the service) all I need to do is run git clone from the /home/pi directory to keep the dnscrypt-resolvers.csv updated.  Typically this is needed when DNS stops working after a while.  I'll have to clone the project, review the csv file and update the rc.local if I need to change one or both DNSCRYPT servers I'm using.

Now all that's left is to override the DNS address given out by the DHCP server on your router and have it give out your Pi's address instead of the normal ISP DNS addresses.  You could give out one of the ISP addresses as the secondary, and Google public DNS as the third.  Just in case your Pi goes down.

If DNS stops working, the first thing to check is if one or both of the DNSCRYPT proxies is not returning lookups.  Log in to the Pi and issue these commands:

dig A +short @
dig A +short @

Each should return lookup results.  If one hangs, that's the proxy that is unable to reach its configured server.  Choose another server from dnscrypt-resolvers.csv to replace it.

Ad Blocking

Sure, you could install some dubious freeware utility into the browsers of each of your computers, but who knows why it is free?  The easiest thing to do since all your devices use your homebuilt DNSMASQ dns server is to block the lookups for these domains completely.

All you need to do is run a script to download a file formatted in the dnsmasq config file format and modify your main DNSMASQ config file to include this file for additional configuration info.

Edit /etc/dnsmasq.conf and add this line at the bottom:


Create /home/pi/updateadblock script:
wget "" --output-document=/home/pi/dnsmasq/adservers.conf
Run this script once (./updateadblock)  then restart DNSMASQ.  You may need to flush the DNS cache on your computers and browsers before the ads start to disappear and you see significant performance improvements.

If ads start showing up again, just run the updateadblock script again to get the latest list.

Encrypted FTP

File Transfer Protocol has been around since the beginning, yet most people I meet nowadays give me the "deer in headlights" look when I mention it.  They've been coddled by "free" services such as Dropbox or You Send It (now Hightail?)

Why send a file up to a server then ask someone to download it back down?  You're using twice the internet bandwidth necessary AND you're exposing your data to snooping by the "free" service.  Sure, you could encrypt your files first, but then you have to hope your recipient knows how to decrypt.

FTP is great because the FTP server is located at one end - adjacent to the sender or receiver's other computers, so depending on its location that user has very fast access to the data.  And when it is transferred, it only goes over the internet once.

Slide a 256GB thumbdrive into your Pi and you'll have plenty of space to store FTP data.  Format it as ext4 and configure your Pi to mount it at boot.

FileZilla is an excellent free FTP file transfer utility that supports "explicit FTP over TLS" (FTP/S).  I highly recommend you use encryption so both your authentication and data transfer occur over encrypted channels.

After playing around with a few popular FTP servers on the Pi, I chose ProFTPd.

First we need to generate a self-signed certificate and key file using OpenSSL.

Step 1:  Generate a private key:
openssl genrsa -out key1.pem 2048

Step 2:  Generate a CSR (this is where you enter the org and org unit info):
openssl req -new -sha256 -key key1.pem -out csr1.csr

Step 3:  Generate a certificate from the CSR:
openssl req -x509 -sha256 -days 365 -key key1.pem -in csr1.csr -out cert1.pem

Installing and configuring ProFTPd on the Pi is a little out of scope for this post; there are plenty of other articles out there you can read to help you do that.  But here are some key settings:

In /etc/proftpd/proftpd.conf:

ServerName - set this to your FQDN (
PassivePorts - make sure this port range matches the passive port range you open on your firewall/router.  Not only do you need to open port 21 for FTP you need to open up a passive range (say 1500-1502) for data streams.
MasqueradeAddress - set this to your FQDN

And this section at the bottom will deny login to any account NOT in the "ftp" group on your Pi (which you may need to create):

<Limit LOGIN>
        DenyGroup !ftp

In /etc/proftpd/modules.conf, make sure this line is NOT commented out (no # in front):
LoadModule mod_tls.c

In /etc/proftpd/tls.conf:

TLSEngine on
TLSProtocol TLSv1
TLSRSACertificateFile cert1.pem
TLSRSACertificateKeyFile key1.pem
TLSRequired on

This last section disables TLS requirement for the guest account if you want to post FTP clients that support secure transfer in the guest ftp directory so users can use this account to first download a client then reconnect using TLS.  Just make sure the guest user only has read access to their ftp directory.

<IfUser guest>
        TLSRequired off

To create an ftp account for (as an example) user "jdoe" issue the following command, assuming you've mounted the USB stick as /media/usb256:

adduser jdoe --home /media/usb256/ftp/jdoe --ingroup ftp

You'll be prompted to enter the user's information and password.  It will add them to the "ftp" group so they can connect over FTP.  When they connect they will be placed in their ftp directory automatically.  And since it is their home directory, they will already have the correct permissions.

Route 53 Dynamic DNS

So now that you are running an FTP server from your house, you may have a dynamic public IP address from your ISP that you need to work with. 

There are still a few free dynamic DNS providers out there, and some even have updaters for the Pi.  After using NOIP for a few months I soon tired of getting emails from them asking me to "confirm" my account or they would stop providing the free dynamic DNS service.  Every time I would "confirm" my account I would be pressured to upgrade to a paid account.

What's funny about NOIP is they try to sell "custom domain name addresses" for an extra charge.  Just so you know, you can create a CNAME DNS record that points to the address they give you.  Meaning you can do that for free.

Since I've been looking at using Amazon Web Services for my personal needs I thought I could use the AWS SDK to programmatically update records on Route 53.  A program would run on the Pi and periodically compare its current public IP against one stored in a local file from the last time it sent an update to AWS.  If different, it would make another call to AWS, change the A record for my ftp server then save the new public IP in the local file.  This program would run every 10 minutes or so, but would only talk to AWS on the rare occasion my public IP changes.

And from my professional experience I know Route 53 is extremely cheap for a hosted DNS service.

Turns out I'm not the first to come up with this idea:

setting up a dyndns service with route53
Roll your own dynamic DNS service using Amazon Route53
Moving DNS to Amazon Route 53, Dynamic Updates
Dynamic DNS with Route 53
Setting up Dynamic DNS to your Home with Route 53 (although the term subdomain is used incorrectly)

You will need to create an IAM user with Access Key and Access Secret for the API call, but there are a couple of ways to grant this IAM user access to update your ftp record.
  1. Entire zone.  arn:aws:route53:::hostedzone/HOSTED_ZONE_ID
    Easiest to set up, but this means the IAM user can edit ANY record on your hosted domain.
  2. Use an AWS Lambda function that makes this change (only for this single record) and make an invocation policy for this function.
  3. Create a subdomain for ftp, and maintain its A record.  This subdomain will have no other address records.  Add as another hosted zone, which means it will have its own HOSTED_ZONE_ID, and add the appropriate NS records to the parent domain's zone.
    This is essentially as secure as option #2 and not much harder than #1 to implement.
Going with option 3 is fairly straightforward.  Just make sure you create the subdomain first then copy its assigned Amazon nameservers into a text file.  Then go into the parent's zone and add an NS record for ftp specifying the same Amazon nameservers.

Add an Inline Policy to the IAM user granting Allow to the arn resource address for the following actions:


The policy should look like this, with HOSTED_ZONE_ID replaced with the actual Amazon-assigned ID:
    "Version": "2012-10-17",
    "Statement": [
            "Sid": "Stmt0000000000000",
            "Effect": "Allow",
            "Action": [
            "Resource": [
Now on to setting up a program and job on the Pi to update this record.

How can we programmatically determine our current public IP?  These commands all work from the Pi:
  1. dig +short
  2. dig TXT +short | awk -F'"' '{ print $2}' 
  3. curl
Install AWS CLI for Python 2:

apt-get install python-pip
pip install awscli

aws configure

Enter your IAM Access Key and Secret Access Key.

Create directory /home/pi/dyn53 and change to it.
Create file crrs.template, replacing with your A record:
        "Comment": "Test",
        "Changes": [
                "Action": "UPSERT",
                "ResourceRecordSet": {
                    "Name": "",
                    "Type": "A",
                    "TTL": 600,
                    "ResourceRecords": [
                            "Value": "A.B.C.D"
Create update53, with HOSTED_ZONE_ID replaced with the actual Amazon-assigned ID:
#! /bin/bash

#get last ip address sent to AWS
while IFS=, read ipa;do
done < lastip.txt
echo "Last IP: $LAST_IP"

#get current public ip
CURRENT_IP=`dig +short`
echo "Current IP: $CURRENT_IP"

#compare last vs current
if [ "$CURRENT_IP" != "$LAST_IP" ]; then
   echo "Different, updating Route 53..."
   sed 's/A.B.C.D/'"$CURRENT_IP"'/g' crrs.template > crrs.send
   aws route53 change-resource-record-sets --hosted-zone-id HOSTED_ZONE_ID --change-batch file://crrs.send
   echo "$CURRENT_IP" > lastip.txt
   rm crrs.send
   echo "Same, no update needed."
Make the script executable by running chmod 777 update53
Test the script by running ./update53.  You should see a response like this:
    "ChangeInfo": {
        "Status": "PENDING",
        "Comment": "Test",
        "SubmittedAt": "2017-03-06T15:25:17.202Z",
        "Id": "/change/C3F5SHBFTKJ0QU"
Schedule this to run every 5 minutes by running crontab -e and adding this line:
5 * * * * cd /home/pi/dyn53 && ./update53

No comments:

Post a Comment