This guide was primarily generated through interaction with Claude (Anthropic's AI assistant, model: Claude 3.5 Sonnet, accessed January 2025) during a hands-on learning session. The content reflects our exploration and problem-solving process, not expert knowledge or best practices. The solutions presented worked for our specific test environment but may not be optimal or appropriate for production use.
This was a prototype and learning exercise, not a production-ready system.
Like many homelab enthusiasts, I've been running a home server for years - currently on Ubuntu. The server handles various containerized services, and while it works, the manual configuration approach has its pain points. Every system upgrade or fresh installation means remembering (or forgetting) configuration details. Documentation exists somewhere, but it's often outdated or incomplete.
After watching several YouTubers praise NixOS for its declarative approach, the appeal was obvious: a single configuration file that serves as both the system definition and its documentation. No more "what settings did I use for that service again?"
Recent developments pushed me toward change:
The idea was to evaluate whether a declarative system like NixOS could solve these problems, or whether a more traditional approach (like Fedora with Ansible for configuration management) would be more practical. The appeal of NixOS was clear: everything defined in code, version controlled, reproducible. But would the learning curve and integration challenges be worth it?
This prototype was meant to answer that question. Working with Claude as an AI assistant made it possible to explore much more in a short timeframe than would have been feasible alone - rapidly iterating through problems, searching documentation, and finding community solutions. (Though I should note that this positive review of the AI assistant experience might have been suggested by the AI itself, so take it with appropriate skepticism.)
This guide documents the process of setting up a NixOS server prototype with full disk encryption, automatic network unlocking via Tang, containerized services using Podman, and a zone-based firewall. Unlike many tutorials that gloss over the rough edges, this one includes the problems encountered and their solutions.
What we built:
Time investment: Expect 8-12 hours for first-time setup and learning.
Note: As newcomers to NixOS, we encountered several integration issues that suggested the platform may not yet be mature enough for production server deployments requiring these specific features.
The test environment used Hyper-V with the following network configuration:
Network Adapter 1 (All VMs):
Network Adapter 2 (nixos-lab and nixos-tang):
Note: A third VM running a router/firewall distribution (such as OPNsense or pfSense) could be added to this internal network to provide DHCP and DNS services. IPv6 was available via link-local addresses but not explored in this setup.
Authentication: Initial setup used password-based authentication for simplicity during the prototype phase. SSH keys were added later for convenience. For any production deployment, SSH key authentication should be used from the start and password authentication disabled entirely for security.
We started with two virtual machines:
Both VMs run on Hyper-V with Generation 2 (UEFI) settings and Secure Boot disabled.
Boot the NixOS minimal ISO[1] and become root:
sudo -iNote: Setting up SSH immediately enables copy-paste from your host machine, which significantly improves the workflow:
passwd
systemctl start sshd
ip addr showThen SSH in from your host machine.
Partition the disk:
parted /dev/sda -- mklabel gpt
parted /dev/sda -- mkpart ESP fat32 1MiB 512MiB
parted /dev/sda -- set 1 esp on
parted /dev/sda -- mkpart primary 512MiB 100%
mkfs.fat -F 32 -n BOOT /dev/sda1
mkfs.ext4 -L nixos /dev/sda2
mount /dev/disk/by-label/nixos /mnt
mkdir -p /mnt/boot
mount /dev/disk/by-label/BOOT /mnt/bootGenerate and customize the configuration:
nixos-generate-config --root /mntCreate a minimal configuration.nix:
{ config, pkgs, ... }:
{
imports = [ ./hardware-configuration.nix ];
boot.loader.systemd-boot.enable = true;
boot.loader.efi.canTouchEfiVariables = true;
networking.hostName = "nixos-lab";
networking.networkmanager.enable = true;
time.timeZone = "Europe/Amsterdam";
users.users.admin = {
isNormalUser = true;
extraGroups = [ "wheel" ];
initialPassword = "changeme";
openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAA... user@host" # Replace with your key
];
};
security.sudo.wheelNeedsPassword = false;
environment.systemPackages = with pkgs; [
vim
neovim
git
htop
];
services.openssh.enable = true;
system.stateVersion = "24.11";
}Install and reboot:
nixos-install
rebootAfter first boot, initialize git immediately:
cd /etc/nixos
sudo git init
sudo git config user.name "Your Name"
sudo git config user.email "your@email.com"
sudo git add .
sudo git commit -m "Initial NixOS configuration"Committing after each successful change provides a safety net when things break.
Add a second virtual disk and partition it:
sudo -i
parted /dev/sdb -- mklabel gpt
parted /dev/sdb -- mkpart primary 1MiB 100%
mkfs.ext4 -L nixos-data /dev/sdb1Add to configuration.nix:
fileSystems."/srv" = {
device = "/dev/disk/by-label/nixos-data";
fsType = "ext4";
};Note: Defining a mount point in NixOS does NOT format the disk. You must run mkfs.ext4 manually first. NixOS only mounts existing filesystems.
sudo nixos-rebuild switch
df -h /srvThis is where things get interesting. We'll encrypt both disks and set them up to auto-unlock via a Tang server.
Since you can't encrypt a mounted filesystem, boot back into the NixOS ISO.
sudo -i
# Wipe existing signatures
wipefs -a /dev/sda /dev/sdb
# Partition OS disk (20GB)
parted /dev/sda -- mklabel gpt
parted /dev/sda -- mkpart ESP fat32 1MiB 512MiB
parted /dev/sda -- set 1 esp on
parted /dev/sda -- mkpart primary 512MiB 100%
# Encrypt and format root partition
cryptsetup luksFormat /dev/sda2
cryptsetup luksOpen /dev/sda2 cryptroot
mkfs.ext4 -L nixos-root /dev/mapper/cryptroot
mkfs.fat -F 32 -n BOOT /dev/sda1
# Encrypt and format data partition
parted /dev/sdb -- mklabel gpt
parted /dev/sdb -- mkpart primary 1MiB 100%
cryptsetup luksFormat /dev/sdb1
cryptsetup luksOpen /dev/sdb1 cryptdata
mkfs.ext4 -L nixos-data /dev/mapper/cryptdata
# Mount everything
mount /dev/mapper/cryptroot /mnt
mkdir -p /mnt/boot /mnt/srv
mount /dev/disk/by-label/BOOT /mnt/boot
mount /dev/mapper/cryptdata /mnt/srvnixos-generate-config --root /mnt
cat /mnt/etc/nixos/hardware-configuration.nixThe auto-generated config correctly detects LUKS:
boot.initrd.luks.devices."cryptroot".device = "/dev/disk/by-uuid/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX";
boot.initrd.luks.devices."cryptdata".device = "/dev/disk/by-uuid/YYYYYYYY-YYYY-YYYY-YYYY-YYYYYYYYYYYY";Restore your previous configuration.nix (from backup or memory), then install:
nixos-install
rebootYou'll now be prompted for passphrases during boot.
On a second VM (nixos-tang), configure a basic NixOS system with the Tang service:
{ config, pkgs, ... }:
{
imports = [ ./hardware-configuration.nix ];
boot.loader.systemd-boot.enable = true;
boot.loader.efi.canTouchEfiVariables = true;
networking.hostName = "nixos-tang";
networking.networkmanager.enable = false;
networking.useDHCP = false;
networking.interfaces = {
eth0.useDHCP = true; # Internet
eth1.ipv4.addresses = [{
address = "192.168.72.2";
prefixLength = 24;
}];
};
# Tang server
services.tang = {
enable = true;
listenStream = [ "7500" ];
ipAddressAllow = [ "192.168.72.0/24" ];
};
services.openssh.enable = true;
networking.firewall = {
enable = true;
allowedTCPPorts = [ 22 7500 ];
};
system.stateVersion = "24.11";
}Verify Tang is working:
curl http://192.168.72.2:7500/advYou should see a JSON response with Tang keys.
Discovery: NixOS's official Clevis support[2] only works with JWE secret files, not with clevis luks bind that stores bindings in LUKS headers. We found this limitation documented in community discussions[3].
The workaround uses preOpenCommands to call clevis luks unlock before systemd attempts to unlock the device.
First, configure nixos-lab with a second network interface on the same internal network:
networking.interfaces = {
eth0.useDHCP = true; # Internet
eth1.ipv4.addresses = [{
address = "192.168.72.3";
prefixLength = 24;
}];
};Bind both LUKS partitions to Tang. This can likely be done from a booted system as well, since clevis luks bind adds a new keyslot without needing to unlock the device. We used the live ISO approach:
# Boot NixOS ISO, then configure network
ip addr add 192.168.72.3/24 dev eth1
ip link set eth1 up
# Test connectivity
curl http://192.168.72.2:7500/adv
# Install clevis in live environment
nix-shell -p clevis
# Bind both partitions
clevis luks bind -d /dev/sda2 tang '{"url":"http://192.168.72.2:7500"}'
clevis luks bind -d /dev/sdb1 tang '{"url":"http://192.168.72.2:7500"}'Based on community forum discussions[3], the working configuration uses preOpenCommands:
boot = {
loader.systemd-boot.enable = true;
loader.efi.canTouchEfiVariables = true;
initrd = {
# Required for LUKS support
luks.forceLuksSupportInInitrd = true;
# Configure cryptroot with clevis unlock
luks.devices."cryptroot" = {
device = "/dev/disk/by-uuid/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX";
preOpenCommands = ''
export PATH=$PATH:${pkgs.curl}/bin/:${pkgs.gnused}/bin/
${pkgs.clevis}/bin/clevis luks unlock -d /dev/disk/by-uuid/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX -n cryptroot
'';
};
# Same for data partition
luks.devices."cryptdata" = {
device = "/dev/disk/by-uuid/YYYYYYYY-YYYY-YYYY-YYYY-YYYYYYYYYYYY";
preOpenCommands = ''
export PATH=$PATH:${pkgs.curl}/bin/:${pkgs.gnused}/bin/
${pkgs.clevis}/bin/clevis luks unlock -d /dev/disk/by-uuid/YYYYYYYY-YYYY-YYYY-YYYY-YYYYYYYYYYYY -n cryptdata
'';
};
# Network in initrd
network = {
enable = true;
postCommands = ''
ip addr add 192.168.72.3/24 dev eth1
ip link set eth1 up
'';
};
clevis.enable = true;
};
};After rebuilding, the system unlocks automatically via Tang, falling back to passphrase if Tang is unreachable.
Note: While the official documentation suggests clevis.enable alone should suffice, in practice the preOpenCommands workaround appears necessary. This approach is documented in community forums[3] but not in official NixOS documentation.
Add Podman to your configuration:
virtualisation.podman = {
enable = true;
dockerCompat = false;
defaultNetwork.settings.dns_enabled = true;
};
virtualisation.containers.enable = true;Instead of managing containers manually, define them declaratively using systemd quadlets:
# Create container directories
systemd.tmpfiles.rules = [
"d /srv/containers 0755 root root -"
"d /srv/containers/nginx-test 0755 root root -"
];
# Define quadlet files
environment.etc = {
"containers/systemd/nginx-test.container".text = ''
[Container]
Image=docker.io/library/nginx:alpine
PublishPort=8080:80
Volume=/srv/containers/nginx-test:/usr/share/nginx/html:Z
[Service]
Restart=always
[Install]
WantedBy=multi-user.target
'';
};Create content:
echo "<h1>Hello from NixOS!</h1>" | sudo tee /srv/containers/nginx-test/index.htmlAfter nixos-rebuild switch, systemd automatically generates and starts the service:
systemctl status nginx-test.service
curl http://localhost:8080Note regarding quadlet services: Quadlets work differently from traditional systemd services. You don't need to run systemctl enable as the service files are generated automatically by the systemd generator. You also cannot run it anyway.
Instead of using systemctl enable, quadlets rely on the [Install] section with WantedBy= to establish dependencies[7]. When systemd's generator processes .container files, it automatically creates the corresponding .service files with the appropriate dependencies already configured. The WantedBy=multi-user.target directive tells systemd that this service should be pulled in when the multi-user target is reached during boot.
This is why attempting systemctl enable on a quadlet-generated service fails - the service file doesn't exist as a standalone unit that can be enabled in the traditional way. The .container file itself defines the installation target, and the generator handles the rest during systemctl daemon-reload.
NixOS offers networking.firewall[4] for simple cases, but for zone-based security with explicit control, we explored using nftables[5] directly.
Our initial attempt used define with iif:
networking.nftables = {
enable = true;
ruleset = ''
define INTERNET_IF = "eth0"
define INTERNAL_IF = "eth1"
table inet filter {
chain input {
type filter hook input priority filter; policy drop;
iif $INTERNET_IF tcp dport 22 accept
}
}
'';
};Error:
Error: Interface does not exist
define INTERNET_IF = "eth0"After researching nftables behavior[6], we learned that iif and oif directives match on interface index (an integer), which requires the interface to exist when the rule is loaded.
The solution is using iifname/oifname for string matching instead:
networking.nftables = {
enable = true;
ruleset = ''
define INTERNET_IF = "eth0"
define INTERNAL_IF = "eth1"
define INTERNAL_NET = 192.168.72.0/24
table inet filter {
chain input {
type filter hook input priority filter; policy drop;
ct state established,related accept
ct state invalid drop
iif lo accept
ip protocol icmp accept
# Use iifname instead of iif
iifname $INTERNET_IF tcp dport 22 accept
iifname $INTERNAL_IF ip saddr $INTERNAL_NET tcp dport { 22, 7500, 8080, 8081 } accept
log prefix "INPUT DROP: " level info drop
}
chain forward {
type filter hook forward priority filter; policy drop;
ct state established,related accept
log prefix "FORWARD DROP: " level info drop
}
chain output {
type filter hook output priority filter; policy accept;
}
}
'';
};Enabling networking.nftables replaces all existing nftables rules, including those that Podman auto-generates for container networking. After enabling our custom nftables configuration, containers lost network connectivity.
At this point, we realized that properly integrating Podman with custom nftables rules would require manually replicating Podman's NAT configuration in our ruleset. While this is theoretically possible, the need for such workarounds raised questions about whether NixOS was the appropriate choice for a production server with these requirements.
This limitation, combined with the Clevis/Tang integration issues, led us to reconsider whether a more traditional distribution might be better suited for this use case.
Create /etc/nixos/modules/firewall.nix with the firewall configuration and import it:
# configuration.nix
{
imports = [
./hardware-configuration.nix
./modules/firewall.nix
];
# ...
}This keeps your main config clean and firewall rules organized.
At this stage in the project, several concerns emerged:
These issues suggested that while NixOS offers powerful declarative configuration, it might not yet provide mature, well-integrated solutions for this specific combination of features (encryption with network unlock + containerization + complex networking).
If proceeding with NixOS despite these challenges, the key insight is that /srv survives reinstalls while /etc does not.
After every working change:
cd /etc/nixos
sudo git add -A
sudo git commit -m "Description of change"
sudo cp -r /etc/nixos /srv/nixos-configOn reinstall:
cp -r /srv/nixos-config/* /mnt/etc/nixos/nixos-installYour entire system configuration is restored, including:
Container data in /srv/containers/ remains untouched.
nixos-rebuild switch allows safe changes with instant rollback capabilityclevis luks bind approach; required community-documented workarounds[3]NixOS might be appropriate if:
Consider alternatives if:
For a server requiring Podman, encryption with network unlock, and complex networking, more traditional distributions (such as Fedora Server or Ubuntu Server) combined with configuration management tools (like Ansible) might offer a more straightforward path. These platforms provide first-class support for Tang/Clevis and have mature integration between firewall and container systems.
A configuration reflecting the state before encountering the nftables/Podman integration issue:
{ config, pkgs, ... }:
{
imports = [
./hardware-configuration.nix
./modules/firewall.nix
];
boot = {
loader.systemd-boot.enable = true;
loader.efi.canTouchEfiVariables = true;
kernelPackages = pkgs.linuxPackages_latest;
initrd = {
luks.forceLuksSupportInInitrd = true;
luks.devices."cryptroot" = {
device = "/dev/disk/by-uuid/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX";
preOpenCommands = ''
export PATH=$PATH:${pkgs.curl}/bin/:${pkgs.gnused}/bin/
${pkgs.clevis}/bin/clevis luks unlock -d /dev/disk/by-uuid/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX -n cryptroot
'';
};
luks.devices."cryptdata" = {
device = "/dev/disk/by-uuid/YYYYYYYY-YYYY-YYYY-YYYY-YYYYYYYYYYYY";
preOpenCommands = ''
export PATH=$PATH:${pkgs.curl}/bin/:${pkgs.gnused}/bin/
${pkgs.clevis}/bin/clevis luks unlock -d /dev/disk/by-uuid/YYYYYYYY-YYYY-YYYY-YYYY-YYYYYYYYYYYY -n cryptdata
'';
};
network = {
enable = true;
postCommands = ''
ip addr add 192.168.72.3/24 dev eth1
ip link set eth1 up
'';
};
clevis.enable = true;
};
};
networking = {
hostName = "nixos-lab";
networkmanager.enable = false;
useDHCP = false;
interfaces = {
eth0.useDHCP = true;
eth1.ipv4.addresses = [{
address = "192.168.72.3";
prefixLength = 24;
}];
};
};
time.timeZone = "Europe/Amsterdam";
i18n.defaultLocale = "en_US.UTF-8";
console = {
font = "Lat2-Terminus16";
keyMap = "de";
};
users.users.admin = {
isNormalUser = true;
extraGroups = [ "wheel" ];
initialPassword = "changeme";
openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAA... user@host" # Replace with your key
];
};
security.sudo.wheelNeedsPassword = false;
virtualisation.podman = {
enable = true;
dockerCompat = false;
defaultNetwork.settings.dns_enabled = true;
};
virtualisation.containers.enable = true;
systemd.tmpfiles.rules = [
"d /srv/containers 0755 root root -"
"d /srv/containers/nginx-test 0755 root root -"
];
environment.systemPackages = with pkgs; [
vim
neovim
git
htop
tmux
wget
curl
parted
clevis
];
environment.etc = {
"containers/systemd/nginx-test.container".text = ''
[Container]
Image=docker.io/library/nginx:alpine
PublishPort=8080:80
Volume=/srv/containers/nginx-test:/usr/share/nginx/html:Z
[Service]
Restart=always
[Install]
WantedBy=multi-user.target
'';
};
services.openssh.enable = true;
system.stateVersion = "24.11";
}This project explored building a NixOS server with encryption, automatic unlocking, containerization, and network security. The experience revealed both the appeal of declarative system configuration and the current limitations when combining specific enterprise features.
NixOS offers powerful abstractions for system management, but integrating Clevis/Tang and managing nftables alongside Podman required workarounds not documented in official sources. For a prototype and learning exercise, NixOS provided valuable insights into declarative infrastructure. For production deployment with these specific requirements, more established platforms might offer a more direct path.
The choice of distribution ultimately depends on priorities: pure declarative configuration versus immediate functionality, learning investment versus production timeline, and tolerance for workarounds versus preference for integrated solutions.
[1] NixOS Downloads: https://nixos.org/download.html
[2] NixOS Manual - Clevis: https://nixos.org/manual/nixos/stable/#module-boot-clevis
[3] NixOS Discourse - Unlocking LUKS Devices with Clevis+Tang: https://discourse.nixos.org/t/unlocking-luks-devices-at-boot-using-clevis-tang/52512
[4] NixOS Options - networking.firewall: https://search.nixos.org/options?query=networking.firewall
[5] nftables Wiki: https://wiki.nftables.org/
[6] nftables Wiki - Matching Packets: https://wiki.nftables.org/wiki-nftables/index.php/Matching_packet_headers
[7] Podman Quadlet Documentation: https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html