Post

General ZFS Notes

Just some random notes on how to setup ZFS.

ZFS on Linux

When setting up a zpool you specify disks in a way that remains consistent across reboots and systems. Reading https://wiki.archlinux.org/index.php/Persistent_block_device_naming#by-id_and_by-path it mentions that the wwn-XXXX names should be the fully persistent.

ZFS Permissions:

1
2
3
zfs mount mypool/myzfs
chown myuser:mygroup /mypool/myzfs

ZFS volumes

View volume list:

1
zfs list

ZFS volume devices:

These devices are in /dev/zvol//

Pool can be multi-layer (e.g. rpool/data).

With a setup like the following (done via proxmox host but should apply to other ZFS systems):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
root@pve001:~# zfs list
NAME                           USED  AVAIL  REFER  MOUNTPOINT
rpool                          663G   236G   104K  /rpool
rpool/ROOT                     567G   236G    96K  /rpool/ROOT
rpool/ROOT/pve-1               567G   236G   567G  /
rpool/data                    66.0G   236G    96K  /rpool/data
rpool/data/subvol-102-disk-0   404M  7.61G   404M  /rpool/data/subvol-102-disk-0
rpool/data/vm-100-disk-0       104K   236G   104K  -
rpool/data/vm-100-disk-1      18.8G   236G  18.8G  -
rpool/data/vm-100-disk-2        64K   236G    64K  -
rpool/data/vm-101-disk-0      3.82G   236G  3.82G  -
rpool/data/vm-103-disk-0      3.99G   236G  3.99G  -
rpool/data/vm-104-disk-0      25.4G   236G  25.4G  -
rpool/data/vm-105-disk-0      4.38G   236G  4.38G  -
rpool/data/vm-106-disk-0      9.27G   236G  9.27G  -
rpool/var-lib-vz              29.3G   236G  29.3G  /var/lib/vz

And a VM config like the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
root@pve001:~# qm config 101
balloon: 0
boot: order=scsi0;ide2;net0
cores: 2
cpu: x86-64-v2-AES
description:
ide2: none,media=cdrom
machine: q35
memory: 4096
meta: creation-qemu=9.0.2,ctime=1725865589
name: ansible001
net0: virtio=BC:24:11:23:36:AE,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-zfs:vm-101-disk-0,iothread=1,size=80G
scsihw: virtio-scsi-single
smbios1: uuid=5adf1be4-29ba-4fdb-86fb-e470e7042498
sockets: 1
vga: qxl
vmgenid: 84875a30-c554-423d-96d1-7602dcf93b92

If you want to view partition layout via fdisk:

1
2
3
4
5
6
7
8
9
10
11
root@pve001:~# fdisk -l /dev/zvol/rpool/data/vm-101-disk-0
Disk /dev/zvol/rpool/data/vm-101-disk-0: 80 GiB, 85899345920 bytes, 167772160 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 16384 bytes
I/O size (minimum/optimal): 16384 bytes / 16384 bytes
Disklabel type: gpt
Disk identifier: E4685CB0-C04D-43E5-B7C5-45919E117E79

Device                               Start       End   Sectors Size Type
/dev/zvol/rpool/data/vm-101-disk-0p1  2048      4095      2048   1M BIOS boot
/dev/zvol/rpool/data/vm-101-disk-0p2  4096 167770111 167766016  80G Linux filesystem

View devices names:

1
2
3
4
root@pve001:~# ls -lh /dev/zvol/rpool/data/vm-101-disk-0*
lrwxrwxrwx 1 root root 12 Oct 10 15:54 /dev/zvol/rpool/data/vm-101-disk-0 -> ../../../zd0
lrwxrwxrwx 1 root root 14 Oct 10 15:54 /dev/zvol/rpool/data/vm-101-disk-0-part1 -> ../../../zd0p1
lrwxrwxrwx 1 root root 14 Oct 10 15:54 /dev/zvol/rpool/data/vm-101-disk-0-part2 -> ../../../zd0p2

Mount a VM disk (WARNING: DO NOT DO THIS AGAINST A RUNNING VM)

1
2
3
4
5
6
root@pve001:~# mkdir /mnt/vm-disk
root@pve001:~# mount /dev/zvol/rpool/data/vm-101-disk-0-part2 /mnt/vm-disk/
root@pve001:~# ls /mnt/vm-disk/
bin  bin.usr-is-merged  boot  cdrom  dev  etc  home  lib  lib64  lib.usr-is-merged  lost+found  media  mnt  opt  proc  root  run  sbin  sbin.usr-is-merged  snap  srv  swap.img  sys  tmp  usr  var

root@pve001:~# umount /mnt/vm-disk

You can also resize partitions on those disks (the current resize disk only increases the disk itself, you can just add a new partition as well).

This post is licensed under CC BY 4.0 by the author.