openzfs/zfs
Photo Credits: Unsplash and ZFS
Background
My work on the ZFS filesystem was my first open-source contribution, and it remains my most noteworthy (because of ZFS's prominence, not because my contribution was particularly large).
Back in March-April of 2023, I was installing multiple Debian-based servers with the ZFS filesystem, and I wanted to implement full-disk encryption. ZFS natively supports encryption for datasets, but supports only a couple, very basic methods for unlocking said encryption. For non-root datasets, this is a non-issue, as a script, cronjob, or systemd unit file can be created to run any arbitrary key retrieval code, then feed that key to zfs load-key.
However, full-disk encryption implies an encrypted root dataset, and the root dataset must be decrypted before Linux reaches the systemd portion of the boot process. The two natively supported solutions for this - at the time of my commit - were:
- Save the root dataset decryption key to a file, and make that file available in the initramfs
- Manually type the decryption key at boot
Neither of these are particularly appealing. #1 defeats the point of the security control, and #2 is cumbersome in any scenario, particularly any involving remote access without IPMI or a serial jump box.
Previously, I had addressed this scenario by installing the dropbear ssh server onto the initramfs, logging into the preboot environment remotely, and entering the decryption key. That was okay, but not great. This time around, I decided to find the portion of the zfs initramfs scripts that attempt to decrypt the root dataset and add my own custom function. (The file location of the relevant script is /usr/share/initramfs-tools/scripts/zfs).
After finding the decrypt_fs() function, I added a line or two that called a separate script, defined my own unlocking mechanism in said script, and proceeded to test and deploy this solution on each of my servers.
(If you're curious, my key storage solution used clevis to create and use TPM-backed encryption keys.)
Contributing
I initially had no plans for a pull request, but, after a day or two, realized my efforts may have actually improved the filesystem in a way that would benefit others - certainly, during my research, I had seen others looking for a similar solution and seemingly finding nothing.
With a renewed search effort, I found a year-old issue opened in the ZFS repo. Despite quite a bit of discussion, it seemed to have stagnated several months previously. I followed up on the thread and created a PR.
After roughly a week, one of the primary ZFS contributors reviewed the issue, made some minor improvements to the scripts, and created his own PR here. I was mildly disappointed to have my changes commited through a different PR, but appreciated both the improvements and the maintainer's care to list me as a co-author in the commit message (which can be viewed here).
Implemented Solution
In the final, agreed-upon implementation, only 13 lines of code were necessary to create a standard location for a key-loading, dataset-decrypting initramfs script:
First, in (repo location) contrib/initramfs/hooks/zfs.in / (installed location) /usr/share/initramfs-tools/hooks/zfs:
for f in "@sysconfdir@/zfs/initramfs-tools-load-key" "@sysconfdir@/zfs/initramfs-tools-load-key.d/"*; do
copy_file config "$f"
done
This ensures any custom scripts in certain pre-defined locations are copied during the initramfs creation process.
Next, in (repo location) contrib/initramfs/scripts/zfs / (installed location) /usr/share/initramfs-tools/scripts/zfs:
for f in "/etc/zfs/initramfs-tools-load-key" "/etc/zfs/initramfs-tools-load-key.d/"*; do
[ -r "$f" ] || continue
(. "$f") && {
# Successful return and actually-loaded key: we're done
KEYSTATUS="$(get_fs_value "${ENCRYPTIONROOT}" keystatus)"
[ "$KEYSTATUS" = "unavailable" ] || return 0
}
done
This iterates through the pre-defined script locations, and, for any paths that exist, sources the script and tests whether the root dataset has been successfully decrypted.
And finally, the obligatory documentation updates:
### Unlocking a ZFS encrypted root via alternate means
If present, a shell program at `/etc/zfs/initramfs-tools-load-key`
and files matching `/etc/zfs/initramfs-tools-load-key.d/*`
will be copied to the initramfs during generation
and sourced to load the key, if required.
The `$ENCRYPTIONROOT` to load the key for and `$KEYLOCATION` variables are set,
and all initramfs-tools functions are available;
use unquoted `$ZPOOL` and `$ZFS` to run `zpool` and `zfs`.
A successful return (and loaded key) stops the search.
A failure return is non-fatal,
and loading keys proceeds as normal if no hook succeeds.
A trivial example of a key-loading drop-in that uses the BLAKE2 checksum
of the file at the `keylocation` as the key follows.
<codeblock>
key="$(b2sum "${KEYLOCATION#file://}")" || return
printf '%s\n' "${key%% *}" | $ZFS load-key -L prompt "$ENCRYPTIONROOT"
</codeblock>
Impact
Because ZFS is so widely-used on Debian and BSD-based systems, most System Administrators or homelab enthusiasts are likely to have my code on at least one server in their environment - which I find quite exciting.
Here's a quick Bash command to determine whether my contribution has made its way onto your machine:
grep '/etc/zfs/initramfs-tools-load-key' /usr/share/initramfs-tools/scripts/zfs 2>&1 >/dev/null && echo "Code is present on the local machine!"