This turned out to be more complicated than expected. Here is how to configure your network so that a Raspberry Pi behind a Unitymedia router can be reached via DynDNS.

Setup:

• Internet from Unitymedia via the Connect Box
• A DynDNS subdomain (from inwx in my case)
• A Raspberry Pi connected via WLAN, running Debian Stretch (9.3)

Problems that need to be solved:

• The connection must use IPv6, because Unitymedia does not provide dedicated IPv4 addresses
• The Raspberry Pi should automatically reconnect to the WLAN
• The Connect Box must forward the packets
• The Raspberry Pi must not use IPv6 Privacy Extensions (random IP suffixes), because otherwise the Connect Box's filtering rules become invalid after a week.
• Let's Encrypt certificates for a connection protected against eavesdropping
• Access restrictions
• My smartphone should be able to establish the connection

I assume that the Raspberry Pi is working properly. All $ commands refer to the Linux shell of the Raspberry Pi. Do not enter the $ character. Paul McWhorter explains here how to edit configuration files.

My Raspberry runs Debian Stretch (9.3). You can determine the Debian version with

cat /etc/debian_version

WLAN

The official guide explains how to connect to a WLAN.

To reconnect automatically after a WLAN outage, create a new configuration file ($ sudo nano /etc/network/interfaces.d/99-wlan) with the following contents:

allow-hotplug wlan0
iface wlan0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp

IPv6 address

Unlike the familiar and more or less exhausted IPv4 addresses (e.g. 221.14.142.35), there are simply incredibly many IPv6 addresses (e.g. 2a02:8071:1ec:d900:ba27:ebff:fef3:2eaf). There are so many that entire ranges are assigned to networks, from which the participants (such as the Raspberry Pi) can choose an address. An address is therefore composed as follows:

  2a02:8071:1ec:d900:ba27:ebff:fef3:2eaf
  <  prefix        > < interface       >
                       identifier

The Connect Box is assigned the prefix by Unitymedia, and the interface identifier contains the participant's MAC address (plus ff:fe in the middle and a bit flip in the eighth bit). Since this is not particularly privacy-friendly, there are Privacy Extensions that randomize the interface identifier and change it every few hours. And because Unitymedia also wants to provide a little privacy, the prefix changes every few days. You can see the exact expiry date on the Connect Box (192.168.0.1) under Admin -> Information -> IPv6 lease expire.

The Raspberry (or rather Raspbian) is now configured by default so that it does not use the Privacy Extensions, but at least generates its interface identifier semi-randomly, namely from a combination of the prefix, the MAC address, and the time. This means that the interface identifier changes whenever the prefix changes. Since we want to set up port filters in the Connect Box shortly, the interface identifier must at least remain constant. The Connect Box is clever enough to update the prefix of the IPv6 addresses in the port filter automatically.

To do this, edit $ sudo nano /etc/dhcpcd.conf, find the line

# Generate Stable Private IPv6 Addresses instead of hardware based ones
slaac private

and comment out the slaac private setting:

# Generate Stable Private IPv6 Addresses instead of hardware based ones
#slaac private

After a restart, the IPv6 address ($ ip -6 addr show dev wlan0) should contain the MAC address of the WLAN adapter ($ ip link show dev wlan0) (with the mangling described above). The former command displays two addresses. The one beginning with fe80: is the local address, like 192.168.x.x used to be.

Port forwarding

In the Connect Box interface (192.168.0.1), under Advanced Settings -> Security -> IP and Port Filter -> IPv6 Port Filter, create a new incoming port rule. Anyone who has created port rules for IPv4 before must be careful. The source port field has a different meaning here.

[x] Aktiviert    [ ] Deaktiviert
Traffic policy: [x] Ja    [ ] Nein
Protokoll: TCP
Quell IP-Adresse: Alle
Ziel IP-Adresse: <Die aktuelle Adresse des Raspberry Pi.
                  Diejenige, die NICHT mit fe80 anfängt.>
Quell Port Range: Beliebig !!!
Ziel Port Range: Manuell. Start: 80. Ende: 80.

Then click “Apply rule” and, very importantly, click “Apply changes” afterward.

Then create a second port forwarding rule for SSL connections (SSL is not SSH):

Ziel Port Range: Manuell. Start: 443. Ende: 443.

DynDNS

This is where things get more complicated, since there are various providers. I made sure that my (sub)domain has only an AAAA record and no A record. (The A record translates a domain name into an IPv4 address, while the AAAA record translates it into an IPv6 address.) I want to ensure that external connections do not even try to establish an IPv4 connection. Besides, where would I route it? 0.0.0.0?

My provider is inwx.de, where I have a domain for 5 euros a year. This includes a “free” DynDNS function that lets you update the AAAA record by calling a URL. I simply set up the DynDNS function for a new subdomain. An A and an AAAA record were created automatically. I then simply deleted the A record. As long as I leave out the myip=ipaddr parameter when calling the URL, it will not be created again, as intended. Interestingly, the DynDNS configuration displays the error message DynDNS record deleted, but it still works to update the AAAA record.

For all commands, scripts, and configuration files below, you must enter your own domain name. Mine is radi.invisibletower.de. Keeping it secret is pointless, because all domains for which a certificate is issued are published in a large list.

On the Raspberry Pi, create a script that updates the DynDNS URL:

$ mkdir ~/domain
$ touch ~/domain/dyndns_update.sh
$ chmod +x ~/domain/dyndns_update.sh
$ nano ~/domain/dyndns_update.sh

with the following contents:

#!/bin/bash

HOSTNAME=radi.invisibletower.de
DYNDNSUSER=hier_username_eintragen
DYNDNSPW=hier_passwort_eintragen
DYNDNSURL=hier_die_update_url_bis_zum_Fragezeichen_aber_ohne_das_Fragezeichen_eintragen

CURRENT_IP6=$(ip address show dev wlan0 | grep ff:fe | grep -v fe80: | awk '{print $2}' | sed 's/^\([0-9a-f:]*\).*/\1/g')
DNS_IP6=$(host -t AAAA ${HOSTNAME} | tr " " "\n" | grep ff:fe)

if [[ "${CURRENT_IP6}" == "" ]]; then
  date
  echo ERROR - COULD NOT GET OWN IP
  exit
fi

if [[ "${CURRENT_IP6}" == "${DNS_IP6}" ]]; then
  # Printing if everything is ok just spams the log.
  # echo OK.
  :
else
  date
  echo The DNS knows ip: ${DNS_IP6}
  echo The real ipv6 address is: ${CURRENT_IP6}
  echo Renew DNS ...
  curl ${DYNDNSURL}\?myipv6\=${CURRENT_IP6}\&hostname\=${HOSTNAME}\&username\=${DYNDNSUSER}\&password\=${DYNDNSPW}
  echo
  echo Done. Sleep 60 seconds.
  sleep 60
  echo Lookup again ...
  DNS_IP6=$(host -t AAAA ${HOSTNAME} | tr " " "\n" | grep ff:fe)
  if [[ "${CURRENT_IP6}" == "${DNS_IP6}" ]]; then
    echo DNS ok
  else
    echo ERROR - COULD NOT UPDATE DNS \(or dns TTL is longer than expected\)
  fi
fi

Run it with $ ~/domain/dyndns_update.sh. The DNS part should now work.

If it does not, unboundtest.com may help with troubleshooting. My mistake was a simple typo in the domain name. If something goes wrong, it often also helps to simply wait 3600 seconds. Especially if it works with unbound but not with host -t AAAA domain.de.

To make this work automatically, create a cron job for this script. Cron jobs run scripts regularly without any intervention from us.

Open the table of all cron jobs with $ sudo crontab -e and add the following line at the end:

*/5 * * * * /home/pi/domain/dyndns_update.sh >> /var/log/dyndns_update.log

Let's Encrypt no. 1

First, we need an SSL certificate that we can use in the next step. I used this guide as a reference.

Let's Encrypt provides a self-updating tool called certbot-auto. Install it first:

$ cd ~/domain
$ wget https://dl.eff.org/certbot-auto
$ chmod a+x certbot-auto
$ ./certbot-auto --version

To update the certificates, create a script:

$ cd ~/domain
$ touch letsencrypt_init.sh
$ chmod +x letsencrypt_init.sh
$ nano letsencrypt_init.sh

with the following contents:

#!/bin/bash

DOMAIN=radi.invisibletower.de

service haproxy stop
/home/pi/domain/certbot-auto certonly --standalone -d $DOMAIN

mkdir -p /etc/haproxy/certs
cat /etc/letsencrypt/live/$DOMAIN/fullchain.pem /etc/letsencrypt/live/$DOMAIN/privkey.pem > /etc/haproxy/certs/$DOMAIN.pem
chmod -R go-rwx /etc/haproxy/certs

service haproxy start

HAProxy already appears here. The two service commands will fail for now, but that is okay. Run it with sudo ~/domain/letsencrypt_init.sh.

The certificates should now be found at $ sudo ls -l /etc/haproxy/certs.

HAProxy

I use neither Apache nor Nginx as a web server. Instead, a proxy runs on ports 80 and 443 and forwards individual subdirectories to other server programs on the Raspberry Pi (such as OctoPrint). The proxy is called HAProxy. It handles both SSL and authentication.

First install it with $ sudo apt-get install haproxy. Then edit the configuration file $ sudo nano /etc/haproxy/haproxy.cfg:

global
        daemon
        log 127.0.0.1 local0 debug
        tune.ssl.default-dh-param 2048

defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        retries 3
        option redispatch
        option http-server-close
        option forwardfor
        maxconn 2048
        timeout connect 5s
        timeout client  15min
        timeout server  15min

frontend http-in
        bind :::80 v4v6

        acl is_letsencrypt path_beg /.well-known/acme-challenge/

        use_backend letsencrypt-backend if is_letsencrypt
        redirect scheme https code 301 if !{ ssl_fc } !is_letsencrypt

frontend https-in
        bind :::443 v4v6 ssl crt /etc/haproxy/certs/radi.invisibletower.de.pem
        reqadd X-Forwarded-Proto:\ https
        option forwardfor

        acl myvaliduser http_auth(L1)

        use_backend webcam if myvaliduser { path_beg /webcam/ }
        use_backend octoprint if myvaliduser { path_beg /octoprint/ }
        use_backend octoprint_socket if myvaliduser { path_beg /sockjs/ }
        use_backend askforauth if !myvaliduser

backend webcam
        reqrep ^([^\ :]*)\ /webcam/(.*)     \1\ /\2
        server webcam1  127.0.0.1:8080

backend octoprint
        reqrep ^([^\ :]*)\ /octoprint/(.*)  \1\ /\2
        reqadd X-Script-Name:\ /octoprint
        option forwardfor
        reqadd X-Scheme:\ https if { ssl_fc }
        server octoprint1 127.0.0.1:5000

backend askforauth
        http-request auth realm Hallo

backend octoprint_socket
        reqrep ^([^\ :]*)\ /(.*)     \1\ /\2
        server octoprint1 127.0.0.1:5000

backend letsencrypt-backend
        server letsencrypt 127.0.0.1:54321

userlist L1
    group G1
    user meinusername insecure-password meinpasswort groups G1
    user zweiter_username insecure-password anderespasswort groups G1

This looks complicated (documentation), but you have to get into it a little. Here is some context.

When you open a website in a browser, this happens via the so-called HTTP protocol. It defines precisely how the browser asks for a particular URL and how the server must respond. The question is called an HTTP request, and the answer an HTTP response. Both consist of plain text: first come several so-called headers, followed by the actual content, which is usually empty in a request and contains the HTML code in responses. For example, the HTTP request used to open this post looks like this:

GET /?p=241&preview=true HTTP/1.1
Host: invisibletower.de
User-Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://invisibletower.de/
Cookie: wordpress_logged_in=geheim; wp-settings-1=geheim; wp-settings-time-1=geheim
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0

The first line specifies the type (GET/POST/...) and the path. Every following line is a header and serves some purpose. A web server does nothing more than open an arbitrary port (port 80 by default) and wait for HTTP requests.

HAProxy opens this port 80, accepts all requests, examines the headers, and forwards the requests as an intermediary to other servers according to the configured rules. The rules can, for example, check the requested path or whether a correct password was included in the headers. HAProxy can also modify the request, such as by adding new headers or changing the path. The “other” servers can run on other computers or on the same computer, but on different ports. A server is essentially just a program that has opened a port. (P.S. HAProxy can also route ordinary network connections that do not use the HTTP protocol.)

HAProxy is essentially a router. Internally, it implements this using so-called frontends and backends. A frontend specifies where HAProxy accepts requests and to which backend they are forwarded, while the backend specifies how the server behind it can be reached.

Let's go through the configuration file.

frontend http-in
        bind :::80 v4v6

        acl is_letsencrypt path_beg /.well-known/acme-challenge/

        use_backend letsencrypt-backend if is_letsencrypt
        redirect scheme https code 301 if !{ ssl_fc } !is_letsencrypt

frontend says that a new frontend begins here. http-in is the freely selectable name of the frontend. The name is actually only important for the log files. bind :::80 v4v6 says that this frontend should open port 80 on all IPv4 and IPv6 addresses of this computer. acl rule-name rule defines a new rule that can later be used with if statements. In this case, the is_letsencrypt rule is satisfied when the path begins with /.well-known/acme-challenge/. The use_backend line ensures that the backend named letsencrypt-backend is used when the is_letsencrypt rule is satisfied. In all other cases, the redirect instruction takes effect. It returns a response with an HTTP error code to the requesting browser, telling the browser to try the encrypted variant (https://).

What is deliberately missing here are additional use_backend entries: this ensures that an unencrypted request to port 80 cannot do anything harmful, because neither encryption nor authentication takes place here.

frontend https-in
        bind :::443 v4v6 ssl crt /etc/haproxy/certs/radi.invisibletower.de.pem
        reqadd X-Forwarded-Proto:\ https
        option forwardfor

        acl myvaliduser http_auth(L1)

        use_backend letsencrypt-backend if { path_beg /.well-known/acme-challenge/ }
        use_backend webcam if myvaliduser { path_beg /webcam/ }
        use_backend octoprint if myvaliduser { path_beg /octoprint/ }
        use_backend octoprint_socket if myvaliduser { path_beg /sockjs/ }
        use_backend askforauth if !myvaliduser

The next frontend listens on port 443 and provides SSL encryption using the specified private certificate. reqadd X-Forwarded-Proto:\ https adds an X-Forwarded-Proto header to the HTTP request. You will have to Google what exactly it means. (P.S. The slash escapes the space.) option forwardfor also adds some (important?) header. acl myvaliduser http_auth(L1) defines a new rule with the freely selectable name myvaliduser. The rule is satisfied when http_auth(L1) is true. http_auth examines the request headers, looks for a username:password combination, and compares it with the list L1. The use_backend lines use the result of this check to decide which backend to forward the request to. The askforauth backend receives all requests with an incorrect or missing password (the ! means negation). Rules can either be specified using an acl instruction or inside curly braces { .. } after the if. A single space is sufficient between individual rules; they are treated as though a logical AND appeared between them.

The password is sent over the connection, but SSL encryption keeps it encrypted from everyone else. To prevent someone from intercepting the password on a public WLAN, for example, it is best not to require authentication in the http-in frontend, only in the https-in frontend as shown here.

backend webcam
        reqrep ^([^\ :]*)\ /webcam/(.*)     \1\ /\2
        server webcam1  127.0.0.1:8080

This backend forwards all requests to the server listening locally on port 8080. (127.0.0.1 is always the server's own IP address.) webcam1 is another freely selectable name. The line containing reqrep removes the /webcam/ part from the request path.

backend askforauth
        http-request auth realm Hallo

This backend asks the browser for a username:password combination using a special HTTP response header.

userlist L1
        group G1
        ....

This is the list of permitted users. Self-explanatory.

With this knowledge, you should now be able to create additional backends yourself.

After changing the configuration, restart HAProxy as follows:

$ sudo service haproxy stop
$ sudo service haproxy start
$ sudo service haproxy status

Use $ journalctl -xe to see more details about configuration-file errors.

Let's Encrypt 2

Now that the proxy is in place, regular Let's Encrypt certificate renewal can also be handled through the proxy. This guide was also shamelessly copied.

To do this, create a cron job that calls $ certbot renew --deploy-hook letsencrypt_renewhook.sh every night. After a successful renewal, this calls the letsencrypt_renewhook.sh script, which in turn copies the new certificates together for HAProxy.

First create the script with $ sudo nano ~/domain/letsencrypt_renewhook.sh:

#!/bin/sh

SITE=radi.invisibletower.de

# move to the correct let's encrypt directory
cd /etc/letsencrypt/live/$SITE

# cat files to make combined .pem for haproxy
cat fullchain.pem privkey.pem > /etc/haproxy/certs/$SITE.pem

# reload haproxy
service haproxy reload

Now make the script executable: $ sudo chmod u+x ~/domain/letsencrypt_renewhook.sh, and test it with $ sudo /home/pi/domain/letsencrypt_renewhook.sh. It should run without errors.

Now certbot must be configured to run behind HAProxy. Open the configuration file with $ sudo nano /etc/letsencrypt/renewal/radi.invisibletower.de.conf and add the following setting to the [renewalparams] section:

http01_port = 54321

You can test the process with $ sudo certbot renew --dry-run. No red error text should appear.

Open the table of all cron jobs with $ sudo crontab -e and add the following line at the end:

30 2 * * * /home/pi/domain/certbot-auto renew --deploy-hook "/home/pi/domain/letsencrypt_renewhook.sh" >> /var/log/le-renewal.log

Android

I still had the problem that my phone did not want to open the site (Congstar on the Telekom network). The problem was—or is—that IPv6 is not supported by default. At least, that is what they say on the Congstar forum. But:

On your phone, open Settings. Then go to Mobile networks (on my phone, this is under Wireless networks => More). Then go to Access point names and select the active access point. Scroll all the way down and set APN protocol and APN roaming protocol to IPv4/IPv6. You may need to briefly switch airplane mode on and off again. Ta-da.

Security

The configuration now has the following problems:

• From the internal network, the “subservers” can also be reached on their special ports. This is not necessarily a problem. The following command would remedy it:

$ sudo apt-get install iptables-persistent
$ PORT=8080 sudo /sbin/iptables -A INPUT -p tcp -i wlan0 ! -s 127.0.0.1 --dport ${PORT} -j DROP && sudo /sbin/ip6tables -A INPUT -p tcp -i wlan0 ! -s ::1 --dport ${PORT} -j DROP && sudo /sbin/iptables-save | sudo tee /etc/iptables/rules.v4 && sudo /sbin/ip6tables-save | sudo tee /etc/iptables/rules.v6

• Anyone who has edited the files certbot-auto, dyndns_update.sh, or letsencrypt_renewhook.sh automatically has root privileges because of the cron job, even if a root password is configured. The following fixes this:

$ sudo chown -R root:root ~/domain
$ sudo chmod -R 755 ~/domain

Conclusion

There, all the information is now collected in one place.

∎

Comments

[WARNING] 166/145656 (5486) : parsing [/etc/haproxy/haproxy.cfg:26] : a 'redirect' rule placed after a 'use_backend' rule will still be processed before.
[ALERT] 166/145656 (5486) : parsing [/etc/haproxy/haproxy.cfg:29] : 'bind :::443' : unable to load SSL private key from PEM file '/etc/haproxy/certs/kerem.spdns.org.pem'.
[ALERT] 166/145656 (5486) : Error(s) found in configuration file : /etc/haproxy/haproxy.cfg
[ALERT] 166/145656 (5486) : Fatal errors found in configuration.

/home/pi/domain/dyndns_update.sh: Zeile 20: Syntaxfehler beim unerwarteten Wort `else'
/home/pi/domain/dyndns_update.sh: Zeile 20: `else'

/home/pi/domain/dyndns_update.sh: Zeile 20: Syntaxfehler beim unerwarteten Wort `else'
/home/pi/domain/dyndns_update.sh: Zeile 20: `else'

# Configuration file for ddclient generated by debconf
#
# /etc/ddclient.conf

protocol=dyndns2
usev6=if, if=wlan0
if-skip=Scope:link
server=dyndns.strato.com/nic/update
login=mein-login
password='mein-PW'
meine-domain.de, subdomain1.meine-domain.de, subdomain2.meine-domain.de