Minimum effort, maximum personal cloud infrastructure

Table of Contents

• Introduction
• Goal
• Hardware
• Tools
  • Linux
  • Docker Compose
  • Traefik proxy
  • VSCode (optional)
• Building your personal infrastructure
  • Getting the Linux Server up and running
  • Install Docker and Docker Compose
  • Running Nextcloud
  • Tweaking Nextcloud
• More services, more examples
  • A static website
  • Home Assistant and Mosquitto
  • The Unify controller
  • A MineCraft server
• Feedback

Introduction

I never really enjoy blogs with paragraphs of philosophical explanations of why one should use the presented solution so I’ll get down to business right away and address some of the philosophy later.

Goal

To create as much high quality personal (cloud) infrastructure with as little effort as possible.

I’ll focus on getting Nextcloud running as minimum viable personal cloud infrastructure (MVPCI). The methods used to get Nextcloud running are easily extended to other services as well, I’ll present some examples later on.

Hardware

I recently bough new home-server hardware, the goal was to strike a balance between power and power consumption. These are the specs:

The URLs link to tweakers.net’s price comparison section, this is a popular comparison site for hardware in the Netherlands. I didn’t include any cables, since the motherboard is for OEMs, it does not come with any but I had them lying around from my previous server build.

Tools

Linux

I’ll use Ubuntu Desktop because I can get it up and running within 10 min and so far it has worked flawlessly on any hardware I tried it on (and indeed it works perfectly with the hardware mentioned above). Ubuntu Desktop presents a desktop so I can easily open some terminals and drag some files around and use VNC to do the same after I stowed the server away in the basement. But since we are going to use Docker Compose, it really does not matter which distribution you choose; after you booted it, everything will be the same as described here.

Docker Compose

When I first tried to use Docker for my personal infrastructure, I concluded that it made things more complicated for me. When I first heard the term Docker Compose, I thought “hmm, another layer of complexity”… But no, Docker Compose = Docker for Dummies. One can simply write down (in Yaml) what services one wants and Docker Compose will download them down and make them run.

Traefik proxy

Traefik is a reverse proxy and can seamlessly integrate Let’s Encrypt to get you valid certificates for your (sub) domains. It can be fully configured from within a Docker Compose yaml file and allows you to run multiple services in the same port, routing requests based on the contacted (sub) domain.

VSCode (optional)

Whenever I work on a server, I use VSCode with the remote-ssh plugin, then I can open files on the remote server and start using them in a ssh session directly. I highly recommend it. Does require you apt install openssh-server on the server.

Building your personal infrastructure

Getting the Linux Server up and running

Pick whatever GNU/Linux distribution you prefer, and install it on your hardware. I have used Ubuntu Desktop so if you don’t, some small things may be different.

Install Docker and Docker Compose

Both Docker and Docker Compose are best installed not from your distributions repositories (unless you run Arch maybe) but using direct methods. First we install Docker (instructions optimized for Ubuntu 20.04) (source: digitalocean.com/community/tutorials), run the following commands in succession, see the source for more details

sudo apt update

sudo apt install apt-transport-https ca-certificates curl software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"

sudo apt update

apt-cache policy docker-ce

sudo apt install docker-ce

sudo usermod -aG docker ${USER}

su - ${USER}

Verify that you can use docker by issuing the following command, you should see a version number printed

docker --version

Now we install Docker Compose

sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/

docker-compose

sudo chmod +x /usr/local/bin/docker-compose

Verify that you can use docker-compose by issuing the following command, you should see a version number printed

docker-compose --version

Running Nextcloud

Make a new directory, the name is not important, I usually use “docker”

mkdir docker

Open a text file called docker-compose.yaml and paste the following content into it:

version: "3.9"
services:
  traefik:
    image: traefik:latest
    container_name: traefik
    command:
      - --log.level=DEBUG
      - --api.insecure=true
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --certificatesresolvers.mytlschallenge.acme.tlschallenge=true
      - --certificatesresolvers.mytlschallenge.acme.email=you@email.nl
      - --certificatesresolvers.mytlschallenge.acme.storage=/letsencrypt/acme.json
      - --serversTransport.insecureSkipVerify=true
    ports:
      - 80:80
      - 443:443
      - 8888:8080
    volumes:
      - ./letsencrypt:/letsencrypt
      - /var/run/docker.sock:/var/run/docker.sock:ro

  nextcloud_db:
    container_name: nextcloud_db
    image: mariadb
    restart: always
    command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW
    volumes:
      - ./cloud.yourdomain.nl/db:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=your_MYSQL_ROOT_PASSWORD
      - MYSQL_PASSWORD=your_MYSQL_PASSWORD
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud

  nextcloud_app:
    container_name: nextcloud_app
    image: nextcloud
    restart: always
    depends_on:
      - nextcloud_db
    volumes:
      - ./cloud.yourdomain.nl/nextcloud:/var/www/html
    environment:
      - MYSQL_PASSWORD=your_MYSQL_PASSWORD
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - MYSQL_HOST=nextcloud_db
    labels:
      - traefik.enable=true
      - traefik.http.routers.nextcloud_app-http.entrypoints=web
      - traefik.http.routers.nextcloud_app-http.rule=Host(`cloud.yourdomain.nl`)
      - traefik.http.routers.nextcloud_app-http.middlewares=nextcloud_app-https
      - traefik.http.middlewares.nextcloud_app-https.redirectscheme.scheme=https
      - traefik.http.routers.nextcloud_app.entrypoints=websecure
      - traefik.http.routers.nextcloud_app.rule=Host(`cloud.yourdomain.nl`)
      - traefik.http.routers.nextcloud_app.tls=true
      - traefik.http.routers.nextcloud_app.tls.certresolver=mytlschallenge

But before we can use this we need to do some things.

• Make sure you have a valid A record for your (sub) domain pointing to the IP address of your server.
• Set port-forwarding on you router such that requests on ports 80 (http) and 443 (https) point to the IP address of your server. It’s best if you also give it a fixed IP address within your network.
• Replace the following values in the yaml text:
  • you@email.nl: your email address
  • your_MYSQL_ROOT_PASSWORD: A mysql root password, it can be whatever you want
  • your_MYSQL_PASSWORD: The password for the nextcloud mysql user, make up a nice one
  • cloud.yourdomain.nl: Set the domainname you made an A record for, pointing to your server

Save the modified file. Now we are ready to pull down and start Nextcloud by issuing:

docker-compose up -d

You will see the containers being pulled down. When they are started, visit your Nextcloud install at https://cloud.yourdomain.nl (the domainname you choose).

Tweaking Nextcloud

When you visit your domain, you’ll find Nextcloud ready for intialization. I ran into some issues going through this process which I’d like to warn you about.

The first one is that I initially choose to “install recommended apps”, this means that after setting up the database and the admin user, Nextcloud will install several extra apps. For me choosing this option repeatedly led to a very slow Nextcloud install, really unuseable. So I deselected this option and found a fast and snappy Nextcloud install waiting for me.

The second is that Nextcloud requires a form of very regular maintenance, regular as in every 5 min. There are several ways to get this done, if you click your avatar in the top left of your Nextcloud install, then choose “Settings” and the “Basic settings” (under Adminstration), the methods are listed. The best method is to use Cron, you can have your server run the required cron job every 5 min by adding the following line to your crontab (open crontab by typing crontab -e on the command line):

*/5 * * * * docker exec -u www-data nextcloud_app php cron.php

My third problem was that I had issues connecting my sync apps, they are described here, I hope you will not run into them, otherwise, read this thread.

And then we have a forth issue, to set up caldav and carddav correctly for some clients we need to add some configuration to resolve so called well-known caldav and carddav. Here is some more information: docs.nextcloud.com. I hope you can use the Traefik 2 examples to avoid having to edit .htaccess manually. I did do that however, I changed these 2 lines (I added the https://cloud.yourdomain.nl part):

  RewriteRule ^\.well-known/carddav https://cloud.yourdomain.nl/remote.php/dav/ [R=301,L]
  RewriteRule ^\.well-known/caldav https://cloud.yourdomain.nl/remote.php/dav/ [R=301,L]

More services, more examples

You can add the following snippets to your docker-compose.yaml file to get more services.

A static website

This service serves you this very website (replace pci.hmrt.nl with a (sub) domain of your choosing).

  nginx_static_site:
    image: nginx
    container_name: nginx_static_site
    volumes:
      - ./nginx:/usr/share/nginx/html
    labels:
      - traefik.enable=true
      - traefik.http.routers.nginx_static_site-http.entrypoints=web
      - traefik.http.routers.nginx_static_site-http.rule=Host(`pci.hmrt.nl`)
      - traefik.http.routers.nginx_static_site-http.middlewares=nginx_static_site-https
      - traefik.http.middlewares.nginx_static_site-https.redirectscheme.scheme=https
      - traefik.http.routers.nginx_static_site.entrypoints=websecure
      - traefik.http.routers.nginx_static_site.rule=Host(`pci.hmrt.nl`)
      - traefik.http.routers.nginx_static_site.tls=true
      - traefik.http.routers.nginx_static_site.tls.certresolver=mytlschallenge

Home Assistant and Mosquitto

This software controls many things in my home. Make sure you create /mosquitto/config/mosquitto.conf, to use the persistant storage of data and logs, add these line to said file:

persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log

Then add the following snippet to your docker-compose.yaml file:

  homeassistant:
    container_name: home-assistant
    image: homeassistant/home-assistant:stable
    volumes:
      - ./homeassistant:/config
    environment:
      - TZ=Europe/Amsterdam
    restart: always
    network_mode: host

  mosquitto:
    container_name: mosquitto
    image: eclipse-mosquitto
    volumes:
      - ./mosquitto/data:/mosquitto/data
      - ./mosquitto/log:/mosquitto/log
      - ./mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf
    ports:
      - 1883:1883
      - 9001:9001

and run docker-compose up -d.

The Unify controller

This is the controller software for a Ubiquiti Unify network. I use it to control my home network. I don’t use it with Traefik (yet).

  unifi:
    image: linuxserver/unifi-controller
    container_name: unifi-controller
    environment:
      - PUID=1000
      - PGID=1000
      - MEM_LIMIT=1024M #optional
    volumes:
      - ./unify-controller:/config
    ports:
      - 3478:3478/udp
      - 10001:10001/udp
      - 8080:8080
      - 8443:8443
      - 1900:1900/udp #optional
      - 8843:8843 #optional
      - 8880:8880 #optional
      - 6789:6789 #optional
      - 5514:5514 #optional
    restart: unless-stopped
    labels:
      - traefik.enable=false   

A MineCraft server

My son likes MineCraft, I like servers…

  minecraft:
    image: itzg/minecraft-server
    container_name: minecraft
    ports:
     - 25565:25565
    #  - 19132:19132/udp # Enable this port to use with GeyserMC so Bedrock clients can join too.
    volumes:
     - ./mc-paper/data:/data
     - ./mc-paper/plugins:/plugins
    environment:
      EULA: "TRUE"
      TYPE: "PAPER"
      MEMORY: "4G"
      TZ: Europe/Amsterdam
      MODE: creative
      MOTD: Bedrock vs Java
    #   OPS: your minecraft id
    tty: true
    stdin_open: true
    restart: always

Feedback

For feedback, you can sent me an email at pci at hmrt dot nl.