Traefik + Docker-Compose Stack

I’ve been messing around with traefik and docker (using docker compose) for a while now, and I think it’s finally time for me to write out some steps on how to get it installed and working. In the past, I followed SmartHomeBeginner’s excellent guides, but the previous few times I’ve made some of my own changes, and it often helps to have a cumulative guide for others to follow. This will be frequently updated (probably) as I add more steps that I originally forgot, so feel free to leave a comment below.

Public IP and port forwards

You need to have a host with a public IP and ports 80 and 443 public. This can be checked with online tools such as YouGetSignal’s open port checker.

Installing stuff

First, we need to have an up to date server running ubuntu 20.04 or similar.

sudo apt update
sudo apt upgrade

Installing docker

Now, we need to install docker. A more in depth guide can be found here.

Some prerequisites:

sudo apt install apt-transport-https ca-certificates curl software-properties-common

Get docker repo gpg key:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Add docker repo to apt:

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"

Update apt lists:

sudo apt update

Install docker:

sudo apt install docker-ce

Check systemctl to ensure docker is running:

sudo systemctl status docker

Now, we need to add our username to the docker group.

Reboot the server (yes I know there are other ways to do this but rebooting is a simple approach) and then run this command:

sudo usermod -aG docker ${USER}

Reboot again (logging out and back in might also work, but rebooting is a simple approach). Then run

id

You should see the docker group now.

Installing docker compose

For a more in depth guide, go here. Note that this may not even be necessary as docker-compose functionality is being added to docker itself.

First, go to the docker compose release page (https://github.com/docker/compose/releases) and make note of the release number. As of now (July 2021) it is 1.29.2

Now, run this command, but before you hit enter, change the version number to the latest.

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

Now, make it executable:

sudo chmod +x /usr/local/bin/docker-compose

To ensure that docker-compose is installed, run this:

docker-compose --version

You should see a build number. If not, something went wrong.

Setting Up Docker Folders

Now that we have docker and docker compose setup, we need to setup our directories and environmental variables.

THIS INFO HAS BEEN COPIED FROM ANOTHER GUIDE. PLEASE VISIT THE OTHER GUIDE!!! https://www.smarthomebeginner.com/traefik-2-docker-tutorial/

In your home directory, make the docker directory.

mkdir ~/docker
sudo setfacl -Rdm g:docker:rwx ~/docker
sudo chmod -R 775 ~/docker

Note, this command above might fail. In this case, install acl and retry.

sudo apt install acl

Now, within the docker directory, we will make our .env file with all of our environmental variables.

cd docker
nano .env

Add this to the .env file. You could use a command line tool like nano, or something like Visual Studio Code/Atom with a remote ssh plugin.

PUID=1000
PGID=997
TZ="America/New_York"
USERDIR="/home/USER"
DOCKERDIR="/home/USER/docker"

You will probably need to replace some information. See here: https://www.smarthomebeginner.com/traefik-2-docker-tutorial/

PUID is the user id of the server/computer. If you are using a free oracle server, this will be “ubuntu”. The PGID is the group ID of docker. You can find this information by running:

id

This might yield a result like below (the username replaced with USER)

uid=1000(USER) gid=1000(USER) groups=1000(USER),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),116(lxd),997(docker)

In this case, the uid of USER is 1000, so PUID will be =1000. The group ID of docker is 997, so PGID will be =997

The USERDIR and DOCKERDIR also need to be changed, simply substitute USER for your username. This can also be found by running

cd ~ ; pwd

Domain Name Setup and Cloudflare

Before we go any further with the guide, I think it is probably time to get the domain and DNS setup. For many of my setups, I get a domain from freenom.com and cloudflare’s DNS service. Both are FREE. However, due to spam, cloudflare has limited using the API with a freenom TLD, including .tk, .ml and others. While you won’t be able to get a certificate from Let’s Encrypt via the automatic dnsChallenge method in Traefik, you can still run the container interactively and get a certificate. However, this process must be repeated every 90 days (expiration of the Let’s Encrypt Certificate). Another option is to just use cloudflare, and don’t even worry about a certificate. A self signed one can be used, and the cloudflare cert will be used (it is already proxying between the server and client anyway). However, this won’t work for non HTTPS services, like an email server.

This is where my guide finally starts to be different/unique. But it is convenient to have all the steps in one place. Regardless, please visit SmartHomeBeginner’s website and show some support/click on a few of the ads.

Getting a Free Domain Name

Head on over to freenom.com. Search for a domain name. Sometimes when you click on it, the domain will say unavailable, despite being available originally. If this happens, try searching for the exact domain name, including the TLD.

When checking out, make sure to change the period from 3 months to 12 months, so you can keep the domain for the longest time. One thing to keep in mind is to remember to cancel the domain name before that period ends, and then renew. Once, I forgot to cancel and renew, and when I went to get the domain again, Freenom was charging $10 for it. I had to wait a few months before I could get it for free again.

Agree to the terms and conditions, and then get the domain name.

Cloudflare

Now, we need to setup the DNS with Cloudflare. Go to cloudflare.com and create an account. Then click add a site on your dashboard. Type in the free domain name that you just got. If you do this immediately after you get the domain, it might fail. It should take you to a page asking what plan you want next. Just scroll to the bottom and select the free plan.

DNS records

You’ll need to add some DNS records now. First, add an A record. Under name, put “@”, and under IPv4 address, put your WAN IP. Leave it proxied.

Now, we need to add a wildcard record to allow for various subdomains. In the future, we can remove this and just make a CNAME record per service, but for now, a wildcard record will be simpler. Make a CNAME record, put “*” in the name field, and put “@” in the IPv4 address field. It should look like this

Now, click continue, and we have to change the nameservers. Copy the first nameserver that cloudflare gives you. Then, in Freenom, click services>my domains>manage domain on the recently acquired name. Then, under management tools, click nameservers. From here, click the “Use custom nameservers”. Paste the Cloudflare names ervers here and when finished, click “change nameservers”. Back in Cloudflare, click “Done, changed nameservers.”

Back to the Server

Now, we need to add a few more details to our .env file.

DOMAINNAME=example.com
[email protected]
CLOUDFLARE_API_KEY=XXXXXXXXXXXX

You can find instructions for these over at the SmartHomeBeginner guide here

Essentially, change DOMAIN to your domain from Freenom, CLOUDFLARE_EMAIL to your email for the Cloudflare account, and CLOUDFLARE_API_KEY to your global Cloudflare API key. Pretty self explanatory.

HTPASSWD

You will also probably want to create HTPASSWD credentials for basic auth. Go here for a generator. Now, create a new folder in the docker root directory:

mkdir shared

In here, make a new file called .htpasswd and fill it with the credentials that you just generated.

Traefik!

Make the Traefik folder.

mkdir traefik
mkdir traefik/acme

Now make the file containing our certificates and set permissions.

touch traefik/acme/acme.json
chmod 600 traefik/acme/acme.json

Create the Traefik log file

touch traefik/traefik.log

Create the traefik.yml configuration file.

touch traefik/traefik.yml

And fill it with this (change your email accordingly):

global:
  checknewversion: "true"
  sendanonymoususage: "true"
log:
  level: ERROR
  filepath: "/traefik.log"
api:
  dashboard: true
  insecure: false
#  debug: true
entryPoints:
  http:
    address: ":80"
  https:
    address: ":443"
        # Allow these IPs to set the X-Forwarded-* headers - Cloudflare IPs: https://www.cloudflare.com/ips/
    forwardedheaders:
      trustedIPs:
        - "173.245.48.0/20"
        - "103.21.244.0/22"
        - "103.22.200.0/22"
        - "103.31.4.0/22"
        - "141.101.64.0/18"
        - "108.162.192.0/18"
        - "190.93.240.0/20"
        - "188.114.96.0/20"
        - "197.234.240.0/22"
        - "198.41.128.0/17"
        - "162.158.0.0/15"
        - "104.16.0.0/12"
        - "172.64.0.0/13"
        - "131.0.72.0/22"
  traefik:
    address: ":8080"
providers:
  docker:
    exposedByDefault: false
    endpoint: "unix:///var/run/docker.sock"
    network: traefik
    defaultRule: "Host(`{{ index .Labels "com.docker.compose.service" }}.$DOMAINNAME`)"
    swarmmode: false
  file:
    directory: "/rules"
#    filename: "/etc/traefik/dynamic_config.yml"
    watch: true
certificatesResolvers:
  dns-cloudflare:
    acme:
#      caserver: "https://acme-staging-v02.api.letsencrypt.org/directory"
      email: [email protected]
      storage: "/acme.json"
      dnsChallenge:
        provider: manual
#        delayBeforeCheck: 120
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"

Now create the rules folder within the traefik folder:

mkdir traefik/rules

This is where we can configure middlewares for traefik. Within the rules folder, create a file called middlewares.toml and fill it with this (be sure to change example.com to your domain name):

[http.middlewares]
  [http.middlewares.middlewares-basic-auth]
    [http.middlewares.middlewares-basic-auth.basicAuth]
#      username=user, password=mystrongpassword (listed below after hashing)
#      users = [
#        "user:$apr1$bvj3f2o0$/01DGlduxK4AqRsTwHnvc1",
#      ]
      realm = "Traefik2 Basic Auth"
      usersFile = "/shared/.htpasswd" #be sure to mount the volume through docker-compose.yml

  [http.middlewares.middlewares-rate-limit]
    [http.middlewares.middlewares-rate-limit.rateLimit]
      average = 100
      burst = 50

  [http.middlewares.middlewares-secure-headers]
    [http.middlewares.middlewares-secure-headers.headers]
      accessControlAllowMethods= ["GET", "OPTIONS", "PUT"]
      accessControlMaxAge = 100
      hostsProxyHeaders = ["X-Forwarded-Host"]
      sslRedirect = true
      stsSeconds = 63072000
      stsIncludeSubdomains = true
      stsPreload = true
      forceSTSHeader = true
#      frameDeny = true #overwritten by customFrameOptionsValue
      customFrameOptionsValue = "allow-from https:example.com" #CSP takes care of this but may be needed for organizr.
      contentTypeNosniff = true
      browserXssFilter = true
#      sslForceHost = true # add sslHost to all of the services
#      sslHost = "example.com"
      referrerPolicy = "same-origin"
#      Setting contentSecurityPolicy is more secure but it can break things. Proper auth will reduce the risk.
#      the below line also breaks some apps due to 'none' - sonarr, radarr, etc.
#      contentSecurityPolicy = "frame-ancestors '*.example.com:*';object-src 'none';script-src 'none';"
      featurePolicy = "camera 'none'; geolocation 'none'; microphone 'none'; payment 'none'; usb 'none'; vr 'none';"
      [http.middlewares.middlewares-secure-headers.headers.customResponseHeaders]
        X-Robots-Tag = "none,noarchive,nosnippet,notranslate,noimageindex,"
        server = ""

  [http.middlewares.middlewares-oauth]
    [http.middlewares.middlewares-oauth.forwardAuth]
      address = "http://oauth:4181" # Make sure you have the OAuth service in docker-compose.yml
      trustForwardHeader = true
      authResponseHeaders = ["X-Forwarded-User"]

Now, create another file in the rules directory called middleware-chains.toml and fill it with this:

[http.middlewares]
  [http.middlewares.chain-no-auth]
    [http.middlewares.chain-no-auth.chain]
      middlewares = [ "middlewares-rate-limit", "middlewares-secure-headers"]

  [http.middlewares.chain-basic-auth]
    [http.middlewares.chain-basic-auth.chain]
      middlewares = [ "middlewares-rate-limit", "middlewares-secure-headers", "middlewares-basic-auth"]

  [http.middlewares.chain-oauth]
    [http.middlewares.chain-oauth.chain]
      middlewares = [ "middlewares-rate-limit", "middlewares-secure-headers", "middlewares-oauth"]

Create the traefik network. This is what will allow our containers to be networked in the reverse proxy.

docker network create --gateway 192.168.90.1 --subnet 192.168.90.0/24 traefik

In the docker root folder, we need to make our docker-compose file.

sudo nano docker-compose.yml

(again, you can use tools like VS Code or Atom, but nano also works)

Fill it with this content:

version: "3.7"

########## NETWORKS ##########
networks:
  traefik:
    external:
      name: traefik
  default:
    driver: bridge

########## SERVICES ###########
services:
# This is where services go

Now, we will add our traefik snippet. Simply copy and paste and append it to the docker-compose file.

# Traefik 2 - Reverse Proxy
  traefik:
    container_name: traefik
    image: traefik:latest # the chevrotin tag refers to v2.2.x but introduced a breaking change in 2.2.2
    restart: unless-stopped
    networks:
      - traefik
    security_opt:
      - no-new-privileges:true
    stdin_open: true #comment these out after first interactive run
    tty: true #comment out after first interactive run
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
      - target: 8080
        published: 8080
        protocol: tcp
        mode: host
    volumes:
      - $DOCKERDIR/traefik/rules:/rules
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - $DOCKERDIR/traefik/acme/acme.json:/acme.json
      - $DOCKERDIR/traefik/traefik.log:/traefik.log
      - $DOCKERDIR/traefik/traefik.yml:/traefik.yml
      - $DOCKERDIR/shared:/shared
      - $DOCKERDIR/certs:/certs

    labels:
      - "traefik.enable=true"
      # HTTP-to-HTTPS Redirect
      - "traefik.http.routers.http-catchall.entrypoints=http"
      - "traefik.http.routers.http-catchall.rule=HostRegexp(`{host:.+}`)"
      - "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
      # HTTP Routers
      - "traefik.http.routers.traefik-rtr.entrypoints=https"
      - "traefik.http.routers.traefik-rtr.rule=Host(`traefik.$DOMAINNAME`)"
      - "traefik.http.routers.traefik-rtr.tls=true"
      - "traefik.http.routers.traefik-rtr.tls.options=default"
#      - "traefik.http.routers.traefik-rtr.tls.certresolver=dns-cloudflare" # Comment out this line after first run of traefik to force the use of wildcard certs
      - "traefik.http.routers.traefik-rtr.tls.domains[0].main=$DOMAINNAME"
      - "traefik.http.routers.traefik-rtr.tls.domains[0].sans=*.$DOMAINNAME"
#      - "traefik.http.routers.traefik-rtr.tls.domains[1].main=$SECONDDOMAINNAME" # Pulls main cert for second domain
#      - "traefik.http.routers.traefik-rtr.tls.domains[1].sans=*.$SECONDDOMAINNAME" # Pulls wildcard cert for second domain
      ## Services - API
      - "traefik.http.routers.traefik-rtr.service=api@internal"
      ## Middlewares
      - "traefik.http.routers.traefik-rtr.middlewares=chain-basic-auth@file"

In the above snippet, traefik is configured so the user can interact with it. This is important if you are using a free domain name. On the first run, we will run the docker compose without disconnecting from it (using the -d flag) so we can add the proper dnschallenge records in cloudflare.

sudo docker-compose up