A hardened Gentoo-Linux/openrc base installation

This blog entry will demonstrate how to install a hardened x86_64 Gentoo Linux musl/openrc/UEFI signed UKI operating system on an encrypted ZFS pool with automatic decryption using TPM. This entry is based on the Gentoo x86_64 handbook and the Gentoo wiki. Gentoo supplies the right tools to build a Linux operating system from scratch, suited to the hardware and needs of the user. This form of customizability and optimizability together with the strong community behind Gentoo makes it a good choice for a desktop operating system.

Provisioning

To install Gentoo this guide will be using the Alpine Extended ISO. It provides all of the necessary utilities for bootstrapping Gentoo. Make sure to boot with secureboot in setup mode or to already have keys ready to deploy.

After booting the Alpine Linux extended ISO, partition the disks. For this action internet is required since zfs, sgdisk and various other necessary packages are not included on the extended ISO, therefore they need to be obtained from the Alpine package repository.

Set it up with setup-interfaces and setup-apkrepos:

sh# setup-interfaces -ar #(1)!
sh# setup-apkrepos -c1

To use Wi-Fi simply run setup-interfaces -r and select wlan0 or similar.

Install the necessary packages:

sh# apk add zfs lsblk sgdisk wipefs dosfstools zlevis #(1)!

The zlevis package is as of this moment not yet in the alpine package repository. Try to get it into the bin via a different method and add its dependencies tpm2-tools and jose.

and load the ZFS kernel module:

sh# modprobe zfs

Wipe the existing disk partitions:

sh# zpool labelclear -f /dev/<disk>
sh# wipefs -a /dev/<disk>
sh# sgdisk --zap-all /dev/<disk>

Create on the disk an EFI system partition (ESP) and a Linux filesystem partition (LFP):

sh# sgdisk -n 1:1m:+512m -t 1:ef00 /dev/<disk>
sh# sgdisk -n 2:0:-10m -t 2:8300 /dev/<disk>

Reload the device nodes:

sh# mdev -s

Format the ESP with a FAT32 filesystem:

sh# mkfs.fat -F 32 -n esp /dev/<disk>1

ZFS pool creation

The ZFS system pool is going to be encrypted. First generate an encryption key and save it temporarily to the file /tmp/rpool.key with:

sh# cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 20 | head -n 1 > /tmp/rpool.key && cat /tmp/rpool.key

All the while we use zlevis for automatic decryption, this key is required when making changes are made to the BIOS or secureboot, so make sure to save it.

Create the system pool:

sh# zpool create -f \
        -o ashift=12 \
        -O compression=lz4 \
        -O acltype=posix \
        -O xattr=sa \
        -O dnodesize=auto \
        -O encryption=on \
        -O keyformat=passphrase \
        -O keylocation=prompt \
        -m none \
        rpool /dev/<disk>2

Then create the system datasets:

sh# zfs create -o mountpoint=none rpool/root
sh# zfs create -o mountpoint=legacy -o quota=48g rpool/root/gentoo
sh# zfs create -o mountpoint=legacy -o quota=32g rpool/root/gentoo/var
sh# zfs create -o mountpoint=/home -o atime=off -o setuid=off -o devices=off -o quota=<home-quota> rpool/home

Setting the <home-quota> depends on the total size of the pool, generally try to reserve some empty space in the pool.

Write the encryption key to TPM with zlevis:

sh# zlevis encrypt rpool '{"pcr_ids":"0,5,7"}' < /tmp/rpool.key

To check if it worked, perform zlevis decrypt rpool.

Finally, export the zpool:

sh# zpool export rpool

Installation

To install Gentoo Linux on the system, the ESP and the datasets of the system pool have to be mounted to the live (ISO) environment.

First import and decrypt the system pool:

sh# zpool import -N -R /mnt rpool
sh# zfs load-key -L file:///tmp/rpool.key rpool

Then mount the datasets and the ESP on /mnt:

sh# mount -t zfs rpool/root/gentoo /mnt
sh# mkdir /mnt/var
sh# mount -t zfs rpool/root/gentoo/var /mnt/var
sh# mkdir /mnt/efi
sh# mount -t vfat /dev/disk/by-label/esp /mnt/efi

Now we are going to fetch a stage 3 tarball, the archive containing a minimal Gentoo environment. Which will act as the seed of the Gentoo install. Replace the <release_date> with the latest stage file release:

sh# wget https://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3-amd64-musl-hardened/stage3-amd64-musl-hardened-<release_date>.tar.xz #(1) #(2)!

It is also possible to use links instead of wget which provides a small user interface for navigation:

sh# links https://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3-amd64-musl-hardened

There are also other mirrors like https://ftp.snt.utwente.nl/pub/os/linux/gentoo/releases/amd64/autobuilds/current-stage3-amd64-musl-hardened/ which might provide a faster download depending on your location. Check out https://www.gentoo.org/downloads/mirrors/ for other mirrors.

Unpack the stage file in the root of the system:

sh# tar -xpf stage3-*.tar.xz --numeric-owner -C /mnt

To have a functional chroot into the system, copy resolv.conf and bind the system process directories:

sh# cp --dereference /etc/resolv.conf /mnt/etc/
sh# for dir in dev proc sys run; do
> mount --rbind --make-rslave /$dir /mnt/$dir
> done
sh# chroot /mnt

Configure portage (1) before doing anything else, an example file of its make.conf is given below:

The package manager of Gentoo Linux, which provisions build files from the Gentoo ebuild repository or any additional ebuild repositories.

/etc/portage/make.conf
# A Gentoo installation is highly personal so diverting from this example is encouraged.
# Please consult /usr/share/portage/config/make.conf.example for a more detailed example.

# Compiler flags
COMMON_FLAGS="-march=native -O3 -pipe"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
RUSTFLAGS="${RUSTFLAGS} -C target-cpu=native"

# Compile options
MAKEOPTS="-j8 -l9" #(1)!

# WARNING: Changing your CHOST is not something that should be done lightly. 
# Please consult https://wiki.gentoo.org/wiki/Changing_the_CHOST_variable before changing.
CHOST="x86_64-pc-linux-musl"

# NOTE: This stage was built with the bindist USE flag enabled

# This sets the language of build output to English.
# Please keep this setting intact when reporting bugs.
LC_MESSAGES=C.utf8

# Logging
PORTAGE_ELOG_CLASSES="log warn error"
PORTAGE_LOGDIR="/var/log/portage"
PORTAGE_LOGDIR_CLEAN="find \"\${PORTAGE_LOGDIR}\" -type f ! -name \"summary.log*\" -mtime +7 -delete"

# Only accept free licenses
ACCEPT_LICENSE="-* @FREE"

# USE flags
USE="${USE} -debug -telemetry -modemmanager -ext* -ppp -systemd -elogind -X -kde -gnome -gtk-doc -webengine hardened dist-kernel udev initramfs hostonly secureboot modules-sign apparmor acpi networkmanager dbus hwaccel bash-completion man pam pipewire vulkan wayland eme-free jpeg png svg" #(2)!

# Emerge flags
EMERGE_DEFAULT_OPTS="${EMERGE_DEFAULT_OPTS} --with-bdeps y --quiet-build y"

# Secureboot settings
SECUREBOOT_SIGN_KEY="/var/lib/sbctl/keys/db/db.key"
SECUREBOOT_SIGN_CERT="/var/lib/sbctl/keys/db/db.pem"
MODULES_SIGN_KEY="${SECUREBOOT_SIGN_KEY}"
MODULES_SIGN_CERT="${SECUREBOOT_SIGN_CERT}"
MODULES_SIGN_HASH="sha512"

The MAKEOPTS defines (-j the number of jobs/processes) and limits (-l the load average) how many parallel make/compilation processes can be launched from portage. The number of parallel processes are limited by the number of available logical CPUs and by RAM, each process can take up to 2 GB of RAM. The load average -l is generally set slighly above the number of jobs -j.
This selection of global (removed -) USE flags is rather personal and should be set according to the preferences of the user.

Synchronise the ebuild repositories and emerge some relevant packages:

sh# emaint sync
sh# emerge -a app-portage/gentoolkit app-editors/vim sys-libs/timezone-data net-misc/openntpd sys-apps/musl-locales

Configure some key aspects of the system:

sh# echo <hostname> > /etc/hostname
sh# echo TZ="/usr/share/zoneinfo/<Region>/<City>" > /etc/env.d/00tz
sh# echo MUSL_LOCPATH="/usr/share/i18n/locales/musl" > /etc/env.d/01locales #(1)!
sh# env-update && source /etc/profile
sh# eselect locale set <locale> #(2)!
sh# rc-update add ntpd default
sh# passwd root #(3)!

Musl does not support locales out of the box. They are not necessary but some programs rely on them to set the language of their application.
The correct <locale> can be found in the locale list:
```
sh# eselect locale list
```
The root password does not really matter because it is going to be locked after a user has been created.

Edit the fstab to set the correct mounts:

/etc/fstab

rpool/root/gentoo       /       zfs     rw,noatime,xattr,posixacl,casesensitive                 0 1
rpool/root/gentoo/var   /var    zfs     rw,noatime,nodev,nosuid,xattr,posixacl,casesensitive    0 2
/dev/disk/by-label/esp  /efi    vfat    defaults,nodev,nosuid,noexec,umask=0077                 0 2
tmpfs                   /tmp    tmpfs   rw,nodev,nosuid,noexec,mode=1777                        0 0
proc                    /proc   proc    nodev,nosuid,noexec,hidepid=2                           0 0

Configure the kernel command-line to be able to boot correctly:

/etc/kernel/cmdline

rw root=ZFS=rpool/root/gentoo rootflags=noatime quiet splash

Emerge sbctl and sbsigntools which will be used alongside dracut (1) to sign the build Unified Kernel Image (UKI):

The initramfs builder.

sh# emerge -a app-crypt/sbctl app-crypt/sbsigntools

Verify that secureboot is in setup mode with sbctl status.

Create and enroll the secureboot keys into the system:

sh# sbctl create-keys
sh# sbctl enroll-keys #(1)!

Whilst enrolling the keys it might be necessary to add the --microsoft flag if you are unable to use custom keys.

Enable dracut to create a UKI with ukify:

/usr/lib/kernel/install.conf

layout=uki
initrd_generator=dracut
uki_generator=ukify

and enable automatic signing with sbsign:

/etc/kernel/uki.conf

[UKI]
SecureBootSigningTool=sbsign

Tell portage to generate a UKI when installing a kernel:

/etc/portage/package.use/installkernel

sys-kernel/installkernel dracut ukify uki

Set the required USE flags for systemd-utils such that on emerging the bootloader systemd-boot will be installed:

/etc/portage/package.use/systemd-utils

sys-apps/systemd-utils kernel-install boot ukify

The installation of the hardware firmware on the system, such as the CPU microcode, is hardware specific:

AMD CPUIntel CPU

The microcode updates for systems with an AMD CPU are all contained in sys-kernel/linux-firmware, accept its license:

/etc/portage/package.license/kernel

sys-kernel/linux-firmware linux-fw-redistributable @BINARY-REDISTRIBUTABLE

and emerge them together with the Gentoo kernel, its necessary kernel modules for this system and the bootloader:

sh# emerge -a sys-kernel/linux-firmware sys-fs/zfs-kmod sys-kernel/gentoo-kernel sys-apps/systemd-utils

The microcode updates for systems with an Intel CPU require alongside sys-kernel/linux-firmware also sys-kernel/intel-microcode, accept its licenses:

/etc/portage/package.license/kernel

sys-kernel/linux-firmware linux-fw-redistributable @BINARY-REDISTRIBUTABLE
sys-firmware/intel-microcode intel-ucode

and emerge them together with the Gentoo kernel, its necessary kernel modules for this system and the bootloader:

sh# emerge -a sys-kernel/linux-firmware sys-firmware/intel-microcode sys-fs/zfs-kmod sys-kernel/gentoo-kernel sys-apps/systemd-utils

Install the bootloader on the ESP:

sh# bootctl install
Copied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi.signed" to "/efi/EFI/systemd/systemd-bootx64.efi".
Copied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi.signed" to "/efi/EFI/BOOT/BOOTX64.EFI".
Random seed file /efi/loader/random-seed successfully refreshed (32 bytes).
Created EFI boot entry "Linux Boot Manager".

and configure the bootloader:

/efi/loader/loader.conf

timeout 3
editor no

One may verify the signed files by running sbctl verify.

Generate the hostid and reconfigure the Gentoo Kernel:

sh# zgenhostid
sh# emerge --config gentoo-kernel

Finally, add some services for ZFS:

sh# rc-update add zfs-mount sysinit
sh# rc-update add zfs-import sysinit #(1)!
sh# rc-update add zfs-load-key sysinit #(2)!

Omit if a faster boot time is preferred.
Omit if a faster boot time is preferred.

Now exit the chroot, unmount the datasets and reboot:

sh# exit
sh# umount -lf /mnt
sh# reboot

Optional installation

Repositories

GURU is an extra repository containing ebuilds that are not available in the Gentoo repository. Although the packages it contains might not be as well tested as in the main repository they are still necessary for some setups, such as some dependencies required for automatic decryption or graphical user interfaces. Add GURU with:

sh# emerge -a app-eselect/eselect-repository
sh# eselect repository enable guru
sh# emaint sync --repo guru

Portage-ample is an extra extra repository containing ebuilds maintained by us, the Ampel organisation. Add portage-ample with:

sh# eselect repository add portage-ample git https://git.ampel.dev/ampel/portage-ample
sh# emaint sync --repo portage-ample

Automatic decryption

Automatic decryption will be managed with zlevis, which is able to unlock an encrypted ZFS root pool with keys saved in a TPM. Currently it is only available in the portage-ample repository and also has some dependencies in the guru repository.

Set the dracut flag for zlevis:

/etc/portage/package.use/zlevis

app-crypt/zlevis dracut

emerge zlevis:

sh# emerge -a app-crypt/zlevis

and enable the zlevis module for dracut:

/etc/dracut.conf.d/zlevis.conf

nofsck="yes"
add_dracutmodules+=" zlevis "

Swap

To add swap to the system emerge zram-init:

sh# emerge -a sys-block/zram-init

Configure zram-init to create a swap device of size one fourth of the ram size:

/etc/conf.d/zram-init

load_on_start="yes"
unload_on_stop="yes"
num_devices="1"

type0="swap"
size0=`LC_ALL=C free -m | awk '/^Mem:/{print int($2/4)}'`
maxs0=1
algo0=zstd
labl0=zram_swap

and add zram-init to the default runlevel:

sh# rc-update add zram-init default

Compiler cache

Compiler cache can speed up recompile's, by avoiding recompilation of the same object files by fetching the result from a cache directory. The package ccache enables a compiler cache for C/C++ object files, present in the Gentoo Kernel for example. Emerge it with:

sh# emerge -a dev-util/ccache

and edit its configuration file:

/etc/ccache.conf

cache_dir = /var/cache/ccache

max_size = 20G
umask = 002
hash_dir = false
compiler_check = %compiler% -dumpversion

compression = true
compression_level = 1

Configure portage to allow ccache for specified packages:

/etc/portage/env/enable-ccache.conf

FEATURES="ccache"
CCACHE_DIR="/var/cache/ccache"

such that for every C/C++ package with which you want to use ccache:

/etc/portage/package.env/ccache

sys-kernel/gentoo-kernel enable-ccache.conf
...

Users

To run processes securely, in an environment with fewer privileges, a user is necessary.

Before creating the user, emerge doas, to be able to "do as" root when it is required:

which requires us to set:

/etc/portage/package.use/doas

app-admin/doas persist

then

sh# emerge -a app-admin/doas

and configure doas by editing:

/etc/doas.conf

permit persist :wheel as root

Now users who are in the wheel group are allowed to use the doas command to gain root privileges.

We can add a user, set its password and add it to the wheel group (if admin) with:

sh# useradd -m -G wheel -s /bin/bash <username>
sh# passwd <username>

The wheel group should ideally only be assigned to one singular admin account. The users in the group are allowed to use the doas command to gain root privileges. This is necessary for installing packages and changing system files but not for a normal user.

If you have checked that doas works with the user then you can lock the root account because it imposes security risks if it is kept open. This can be done with:

sh# passwd -l root

and by changing its login shell to:

/etc/passwd

root:x:0:0:root:/root:/sbin/nologin

Networking

For desktop use NetworkManager is preferred over dhcpcd as network daemon, due to its versatility, i.e. Wi-Fi and VPN compatibility, MAC randomisation, et cetera.

First set the relevant USE flags:

/etc/portage/package.use/networkmanager

net-misc/networkmanager dhcpcd -wext

Also make sure the networkmanager USE flag is enabled in your make.conf.

Now emerge networkmanager:

sh# emerge -a net-misc/networkmanager

and configure networkmanager to have MAC randomisation by editing:

/etc/NetworkManager/NetworkManager.conf

[main]
hostname-mode=none
plugins=ifupdown,keyfile

[ifupdown]
managed=true

[device]
wifi.scan-rand-mac-address=yes

[connection-mac-randomization]
ethernet.cloned-mac-address=random
wifi.cloned-mac-address=random

Then stop any other network service, such as dhcpcd running in the dynamic runlevel, and enable NetworkManager:

sh# rc-update add NetworkManager default

For users to be able to modify connections on the system they will have to be added to the plugdev group.

Concluding remarks

This is the bare minimum for a Gentoo Linux desktop system. Some additional features such as bluetooth, laptop battery management, printer compatiblity, et cetera, have been documented well in the Gentoo Wiki, and can thus be found there. The next steps are the improvement of the security of the system and the configuration of the graphical session.