Systemd: How I made Docker wait for encrypted drive.

At home I am running a little Server with Fedora 29 Linux Server Edition. This box has two encrypted solid state drives (SSD), one for the OS and one for data. The encrypted partitions are unlocked during boot (cf. “TPM Encryption in Fedora Linux“). I am using docker and the data for some of the containers is stored on the second SSD. The problem I ran into was, that docker was started during boot by systemd before the data-partition was unlocked.

In this article I describe how I tweaked my systemd settings to let docker.service wait for the successful mount of an encrypted drive.

This article describes one way to solve the described issue. I do not claim this to be best practice and I encourage readers with a background in Linux server administration to tell me how to do it in a better way.

What you need to know about the boot process in modern Linux distributions is that the process is controlled by systemd. This system service manager follows the concept of system units, small configuration units that are processed during the boot process. I have linked the Red Hat documentation below.

A feature of this concept is that several services are started in parallel at boot time. This makes the process very fast, but can cause problems if one unit is dependent on another.

I encountered such a problem: My Docker containers were started before the drive on which they store their data was mounted. The drive is encrypted and is unlocked during boot. At that time, however, Docker was already launching its containers.

To solve the problem, I tried several approaches. Actually, it was quite easy to define a dependency for the Docker unit.

With this command all dependencies of the units can be displayed:

systemctl list-dependencies

The service that unlocks the encrypted drives is cryptsetup. In the dependency tree it looks like this:

● │ │ ├─cryptsetup.target
● │ │ │ ├─systemd-cryptsetup@luks\xxxxxxxx\....service
● │ │ │ └─systemd-cryptsetup@luks\xxxxxxxx\....service

The dependencies of a unit are defined in the corresponding file in the /etc/systemd/system/ directory. For Docker this is the following file in Fedora 29:

/etc/systemd/system/multi-user.target.wants/docker.service

The dependencies are defined in the [Unit] section:

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target
Wants=network-online.target 
Requires=docker.socket
...

My first attempt was to add cryptsetup.target as a required unit. This did not work. Therefore I added the specific systemd-cryptsetup@luks unit to the line “After=”:

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target systemd-cryptsetup@luks\xxxxxxxx\....service
Wants=network-online.target 
Requires=docker.socket
...

This modification now works and at boot docker is started after the encrypted drive was unlocked and mounted.

Make it permanent

Most files in /etc/systemd/system/ are links into /usr/lib/systemd… Making changes in these files is not safe, because they are overwritten when the system is updated. To permanently add features use the command

systemctl edit <servicename>

In this example:

systemctl edit docker

This opens your default editor with a temporary (and empty) file. Paste your additions here.

[Unit]
After=systemd-cryptsetup@luks\xxxxxx....service

After saving and leaving the editor, you find a new folder “docker.service.d” in your /etc/systemd/system/ folder containing your additions.

Further reading

Leave a Reply

Your email address will not be published. Required fields are marked *