I've gathered information primarily from:
- https://forums.freebsd.org/threads/how-to-dual-boot-windows-11-and-freebsd-14-geli-encrypted-zfs-root-ufs-boot-drive.92472/
- https://forums.freebsd.org/threads/uefi-gpt-dual-boot-how-to-install-freebsd-with-zfs-alongside-another-os-sharing-the-same-disk.75734/
- http://kev009.com/wp/2016/07/freebsd-uefi-root-on-zfs-and-windows-dual-boot/
- https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot
- https://xyinn.org/md/freebsd/zfs_manual_partition_encrypted
These instructions are closest to the first link above, but differ in important ways:
- Unlike the first two links above (and other sources), everything except the partitioning and filesystem setup use the installer. This requires, among other things, mounting the target filesystem in a specific location.
- I set up GELI encryption with a password-only
- I try to give more details on the bootloader setup. I assume you're not afraid of the bootloader if you're trying to triple-boot FreeBSD with Windows and Linux, but even most people in that group aren't experts on the UEFI boot process or multi-stage bootloaders
Install Windows.
Notes:
- You might want to have a local-only account, instead of being forced to sign into a Microsoft account: https://www.tomshardware.com/how-to/install-windows-11-without-microsoft-account
- On a blank drive, Windows 11's installer will create 4 GPT partitions when you tell it to create one. This is annoying but expected (and partly necessary, as we'll discuss when we talk bootloaders).
- On the upside, the 100MB EFI system partition it creates is roomy enough that all the booloader stuff we want to do fits there.
- Windows ISOs are.... not actually bootable by default. You can't just image a USB stick with them and boot. Why? Nobody knows. But there are many workarounds. If you have a copy of Windows already, you can use Rufus. The first source at the top contains instructions on creating a bootable USB Win11 USB stick from FreeBSD.
Install FreeBSD.
FreeBSD's installer doesn't have any automated support for sharing disks with another OS. There's a "manual" partitioning tool, but it's at least somewhat broken; some errors send you back to the start of installation, and I've seen it segfault.
If you're using a desktop, multiple hard drives is the way to go. But if you're trying to do this on a laptop like me, you may only have a single storage device.
The solution is to use the installer for almost everything, but when it comes time for partioning and filesystem creation, we'll drop to a shell and set that up manually. But we'll do it in such a way that we can resume and finish everything but the bootloader with the regular installer.
- Boot from the installation media
- Go through the installation process normally, until you get to the screen that asks you to choose between automatic ZFS, automatic UFS, manual partitioning, or a terminal. Choose the terminal option.
- Load the ZFS kernel module:
kldload zfs
- Force 4KB sectors (for perf):
sysctl vfs.zfs.min_auto_ashift=12
- If you get a complaint about this sysctl not existing, you forgot to load the zfs module
- Figure out which device is your hard drive. For me this was
nda0
. You can usually tell from a quick look at thegpart show
output- From now on I'll use
<DEV>
to refer to this.
- From now on I'll use
- Create a swap partition. This command creates a 16GB swap, but generally specify your RAM size:
gpart add -a 4k -l swap -s 16G -t freebsd-swap nda0
- Create a single partition for your ZFS pool, adjusting size according to your needs:
gpart add -a 4k -l zfs -t freebsd-zfs nda0
- Make sure to leave space for the third OS, which you haven't installed yet!
- Check which partition numbers are your swap and ZFS partitions. This should be visible in the
gpart show
output (which shows the partition types; presumably you only have two FreeBSD-related partitions right now)- From now on, I'll use
<S>
to refer to the number of your swap partition. This was 5 for me. - From now on, I'll use
<Z>
to refer to the number of your ZFS partition. This was 6 for me.
- From now on, I'll use
- Set up GELI encryption
- This is optional, but of course, highly recommended. Another option to consider is adjusting the instructions below to create an encrypted ZFS pool for your main user, but right now this is a little awkward on FreeBSD (e.g., you must log in on the terminal at least once to decrypt your home directory before you can log in via an X session manager, for example). I've done this, and it's not terrible, but I personally find the GELI password-at-boot a little less awkward.
- It's also possible to do this in a way that requires a USB key with a secret on it. I tend to lose these a lot, so I'm just sticking with a password.
- Let's initialize the volume; run the following command, hit enter, type your password, hit enter again, and then type 'EOF':
geli init -g -s 4096 /dev/<DEV>p<Z>
- This might take a moment, hang in there.
- DO NOT forget the -g, or the bootloader will not be able to boot from this. The rest of the installer will run normally even if you forget this, so you won't find the mistake until you try to reboot into the new system, at which point you'll have to redo the install from scratch (you can't change this option on an existing encrypted partition).
- Really, do not omit it, and do not confuse it with other single-letter boot-related options to
geli
which are used in other tutorials. There is a "nearby" letter used in some tutorials which requires the contents of /boot to live on an unencrypted partition. We're not doing that!- I totally didn't mix up the options once and end up with an unbootable system...
- Really, do not omit it, and do not confuse it with other single-letter boot-related options to
- Now we need to mount the drive:
geli attach /dev/<DEV>p<Z>
- That last command will create a device
/dev/<DEV>p<Z>.eli
which routes accesses through GELI. That's what the rest of these instructions will use for the disk device, but if for some reason you're not encrypting, just drop the.eli
everywhere it appears from this point onwards.
- Create the zpool:
zpool create -f -o altroot=/mnt -O atime=off -m none zroot /dev/<DEV>p<Z>.geli
- If you want compression, also pass
-O compress=lz4
- If you want compression, also pass
- Now we create a filesystem hierarchy, with different ZFS datasets for different mount points. If you're relatively new to ZFS and haven't thought about this: this makes it easy to revert part of the system to an old snapshot without losing everything (e.g., rolling back a borked upgrade without losing changes to your home directory). It's similar to the reasons to put different things on different partitions in older Linux setups. These instructions are essentially what the installer itself does
zfs create -o mountpoint=none zroot/ROOT
zfs create -o mountpoint=none zroot/ROOT/default
mount -t zfs zroot/ROOT/default /mnt
- This line above is critical. Many instructions for this process have you mount things somewhere other than
/mnt
, and then do a lot of stuff manually. But this isn't necessary! If you did partitioning through the main installer, it would mount the root filesystem of the new installation at/mnt
. So by doing the same, we make it possible to use the regular installer for unpacking sets, creating new users, etc.
- This line above is critical. Many instructions for this process have you mount things somewhere other than
zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp
zfs create -o canmount=off -o mountpoint=/usr zroot/usr
zfs create zroot/usr/home
zfs create -o exec=off -o setuid=off zroot/usr/src
zfs create zroot/usr/obj
zfs create -o mountpoint=/usr/ports -o setuid=off zroot/usr/ports
zfs create -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create -o canmount=off -o mountpoint=/var zroot/var
zfs create -o exec=off -o setuid=off zroot/var/audit
zfs create -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/log
zfs create -o atime=on -o exec=off -o setuid=off zroot/var/mail
zfs create -o exec=on -o setuid=off zroot/var/tmp
- Now we make the standard symlink for the home directories, and fix some expected permissions
ln -s /usr/home /mnt/home
chmod 1777 /mnt/var/tmp
chmod 1777 /mnt/tmp
- And set this pool to be bootable
zpool set bootfs=zroot/ROOT/default zroot
- Exit the shell, which should return you to the main installer at the step after disk partitioning.
- Finish the installation BUT do not reboot. Instead, when it asks if you want to drop into the installed system (the "Manual Configuration" box), choose yes, so we can do a little more setup, including modifying some files created by the rest of the installer.
- If you miss this and already rebooted, just boot from the install media again. It will ask you for the password to decrypt the GELI device! But it won't mount it because the installer isn't set up for that. On the first screen choose a live system instead of the installer. Then log in as root (no password) and:
- use the
geli attach
command to get the GELI-decrypted device. You'll need to provide your password again. - Run
zpool import
to verify it can now see the zpool. zpool import zroot
(you'll see warnings about a couple subdirectories being unmountable since they conflict with live system directories, but that's okay, we don't need those directories)- Mount the zfs filesystem to /mnt as above
- Now you can continue below.
- use the
- If you miss this and already rebooted, just boot from the install media again. It will ask you for the password to decrypt the GELI device! But it won't mount it because the installer isn't set up for that. On the first screen choose a live system instead of the installer. Then log in as root (no password) and:
- We need to tell FreeBSD to use the swap. The command below sets up encrypted swap, but if you don't want that for some reason, just drop the
.eli
. Note that this is independent of the GELI encrypted ZFS pool: you can have neither, both, or either one encrypted.printf "/dev/<DEV>p<S>.eli\tnone\tswap\tsw\t0\t0\n" >> /mnt/etc/fstab
- Note the switch to specify the swap partition above, not the ZFS partition.
- Set up a few settings that are required for this to work
echo 'zfs_enable="YES"' >> /mnt/etc/rc.conf
- Should already be there, but good to make sure. Without this the kernel won't look for ZFS datasets
echo 'dumpdev="AUTO"' >> /mnt/etc/rc.conf
echo 'powerd_enable="YES"' >> /mnt/etc/rc.conf
- Typically already there
echo 'sendmail_enable="NONE"' >> /mnt/etc/rc.conf
- Most of us don't actually need sendmail
echo 'zfs_load="YES"' >> /mnt/boot/loader.conf
- This tells the loader to pre-load ZFS support for the kernel
echo 'geom_eli_load="YES"' >> /mnt/boot/loader.conf
- This tells the loader to pre-load GELI support for the kernel
echo 'kern.geom.label.disk_ident.enable="0"' >> /mnt/boot/loader.conf
echo 'kern.geom.label.gptid.enable="0"' >> /mnt/boot/loader.conf
Modern machines use UEFI to boot instead of BIOS. If you've followed the instructions this far you've probably heard this before, and perhaps vague claims about it being more modern / extensible / whatever, but this is usually pretty vague. We'll only scratch the surface here, but let's talk about UEFI booting.
Modern UEFI-based systems are set up by default to look on the primary storage device for a partition marked with the EFI type, which is assumed to be formatted as FAT32 (because support for it is ubiquitous and not patent-encumbered, unlike later iterations) and to use EFI/BOOT/BOOT<ARCH>.efi
as the bootloader. (The filename is case insensitive, but FAT32 filenames are case-sensitive; if you have multiple capitalizations of the directories and/or files, good luck sorting out what happens.)
That bootloader is just straight up compiled machine instructions, not a packaged ELF or similar executable, so the UEFI firmware by default finds it, reads it into memory, and jumps to executing the code. That code is expected to do OS-specific loading.
At this point in the instructions, Windows 11's installer already created and populated an EFI system partition with room to spare: it has both the aforementioned bootloader (EFI/BOOT/BOOTX64.EFI
on x86-64 / AMD64 systems) and some supporting files the loader would like to use.
So how does multiboot work?
There are two ways to do this:
- I said earlier that was the default setup. The EFI firmware actually stores an array of loaders in non-volatile memory, along with a choice of what to boot by default. It is possible to simply add an entry to that list for FreeBSD (and later Linux) using
efibootmgr
, and manually change that setting when you want to reboot to another OS. In practice this is a pain. The optional command from the last section added the FreeBSD loader to the EFI list - Alternatively, we could add another bootloader! Which is what is usually done, and what we'll do. These instructions will just hook into the bootloader Linux will install, but we need to copy over the FreeBSD bootloader. Systems like this (typically GRUB these days) become the default thing loaded by EFI, but store a menu configuration on the EFI system partition that they use to populate a menu at boot time, allowing you to easily switch each time you boot.
Look at your gpart show
output and see which partition is marked EFI, then:
mkdir /tmp/efipart
mount -t msdosfs -o longnames /dev/<DEV>p<EFI> /tmp/efipart
- If you're curious, you can now look around the boot partition that Windows set up, but this isn't necessary.
mkdir /tmp/efipart/EFI/freebsd
cp /boot/loader.efi /tmp/efipart/EFI/freebsd/loader.efi
printf "/dev/<DEV>p<EFI>\t/boot/efi\tmsdosfs\tsw\t0\t0\n" >> /mnt/etc/fstab
- The command above sets up the newly-installed system to mount the EFI system partition at the standard location of
/boot/efi
. This makes it easier to copy over an updated loader.efi after a subsequent FreeBSD upgrade.
- The command above sets up the newly-installed system to mount the EFI system partition at the standard location of
This last step is technically optional, but strongly recommended (we'll explain in the next section):
efibootmgr --create --activate --label "FreeBSD" --loader "/tmp/efipart/efi/freebsd/loader.efi"
Now you can shut down the machine. Note that we didn't replace the the windows loader, we just stuck a FreeBSD loader next to it. We also didn't tell EFI specifically about FreeBSD, we'll rely on Linux to install its own bootloader (GRUB) and we'll later tell that where the FreeBSD loader is.
TODO: Check this after I finish bootloader setup:
If you look around at other instructions, you might see people copying the full contents of /boot
into this partition. This is not necessary in 2025. For many years now, the FreeBSD loader has had support for GELI decryption and ZFS built in. The loader will locate the GELI partition, ask for the decryption password, and then poke around in the ZFS filesystem for relevant configurations.
If you're reading very closely, you might have noticed that we set a ZFS flag in loader.conf
, and might be wondering why that's necessary if the loader already has ZFS support. That flag doesn't load ZFS support for the bootloader --- as I said, that's built in.
What it does is pre-load the FreeBSD kernel's ZFS module on behalf of the kernel, which otherwise wouldn't have the code required to access ZFS filesystems!
If you subsequently compile your own kernel, you can opt to have ZFS support built-in rather than in a loadable module, in which case that option would become unnecessary. But the default GENERIC kernel installed by the installer leaves it in a module.
Install your favorite flavor of Linux, getting it to dual-boot with Windows. Most modern distributions will automatically detect the Windows install and add it to Grub.
Some distributions will automatically detect the EFI entry we added for the FreeBSD loader and therefore automatically set up GRUB with an entry for FreeBSD. If you see the FreeBSD entry upon your first reboot after installing Linux, great! You should be able to select it and boot to FreeBSD, and skip the rest of these instructions.
The rest of these instructions assume your Linux distro installed GRUB during setup. If it didn't, there are instructions around to install manually, though they're rough (I've done this exactly twice, 30 years ago, so I'm not writing my own instructions for that). If your distro installed rEFInd instead, check the links above for a relevant entry.
Different Linux distributions vary widely in how, exactly, to update GRUB. With Debian derivatives it's pretty easy, with Fedora it's a bit weird, and others range from somewhere in between to much worse.
The two things in common are the need to update a GRUB configuration file, and the need to install the updated configuration to the EFI system partition.
Find your distribution's GRUB config (some have multiple, find the one you're supposed to update), and add:
menuentry "FreeBSD" {
insmod part_gpt
insmod fat
set root=(hd0,gpt1)
chainloader /efi/freebsd/loader.efi
}
This tells GRUB to load GPT partitioning support (as opposed to older legacy partitioning systems) and FAT32 support. It then sets the root device to the first partition (gpt1
) of the first hard drive (hd0
), which should be correct for most systems, but adjust as needed.
Yes, the indexing for hard drives and partitions start at 0 and 1 respectively. No, I don't know why.
Notice that the chainloader specifies the place we copied the FreeBSD loader to on the EFI system partition.
Finally, you need to run the distribution-appropriate command to update the actual GRUB install with the new config.