ZFS commands cheatsheet for snapshots etc

ZFS Pools

ZFS admin

All pools' status: zpool status
specific pools status: zpool status pdata1

Filesystems

list filesystems / datasets ...

List Help: zfs list --help
List specific TYPES: zfs list -t <TYPE> TYPE = filesystem | volume | snapshot | bookmark
List filesystems: zfs list -t filesystem
List snapshots: zfs list -t snapshot

work with snapshots

List by age and size: zfs list -t snapshot -o name,creation,used,referenced
List all snapshots: zfs list -t snapshot
List all snapshots in pool "pdata1": zfs list -r -t snapshot pdata1
List all snapshots of dataset virtualbox: zfs list -r -t snapshot pdata1/virtualbox
List all snapshots with creation date: zfs list -r -t snapshot -o name,creation
List specific snapshot "pdata1/virtualbox@snap": zfs list -r -t snapshot pdata1/virtualbox@snap
Create new snapshot "snap1" for "pdata1/virtualbox": zfs snapshot pdata1/virtualbox@snap1
Destroy snapshot "snap1" for "pdata1/virtualbox": zfs destroy pdata1/virtualbox@snap1

Restore a snapshot (rollback)

Rollback to snapshot snap1: zfs rollback pdata1/virtualbox@snap1
Rollback to older snapshot snap1, when a more recent snapshot snap2 exists, requires -r:
zfs rollback -r pdata1/virtualbox@snap1

Accessing files of snapshots

Snapshots of file systems are accessible in the hidden directory .zfs/snapshot

ls -l /mnt/pdata1/virtualbox/.zfs/snapshot/

Pools

Pool Status

# zpool status -x
  pool: pdata1
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://illumos.org/msg/ZFS-8000-9P
  scan: scrub repaired 1.88M in 0 days 05:48:56 with 0 errors on Fri Oct  1 06:48:56 2021
config:

        NAME        STATE     READ WRITE CKSUM
        pdata1      ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            ada0    ONLINE       0     0     2
            ada1    ONLINE       0     0     0

errors: No known data errors

Hints: https://www.truenas.com/community/threads/is-this-a-bad-sign-smartd-1-currently-unreadable-pending-sectors.9824/

Replace disks in a mirror

shut-down, remove faulty disk ada0, insert new disk into same slot, and reboot.

ZFS pool looks like this now:

# zpool status

  pool: pdata1
 state: ONLINE
status: One or more devices could not be used because the label is missing or
        invalid.  Sufficient replicas exist for the pool to continue
        functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://illumos.org/msg/ZFS-8000-4J
  scan: scrub repaired 1M in 0 days 05:55:00 with 0 errors on Sun Oct  3 20:10:44 2021
config:

      NAME                     STATE     READ WRITE CKSUM
      pdata1                   ONLINE       0     0     0
        mirror-0               ONLINE       0     0     0
          9543251779912959081  UNAVAIL      0     0     0  was /dev/ada0
          ada1                 ONLINE       0     0     0

errors: No known data errors

run cmd to start resilvering: # zpool replace pdata1 ada0

nas: ~# zpool status pdata1
  pool: pdata1
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Thu Oct  7 22:01:02 2021
        119G scanned at 2.25G/s, 2.65M issued at 51.2K/s, 2.10T total
        0 resilvered, 0.00% done, no estimated completion time
config:

        NAME                       STATE     READ WRITE CKSUM
        pdata1                     ONLINE       0     0     0
          mirror-0                 ONLINE       0     0     0
            replacing-0            UNAVAIL      0     0     0
              9543251779912959081  UNAVAIL      0     0     0  was /dev/ada0/old
              ada0                 ONLINE       0     0     0
            ada1                   ONLINE       0     0     0

errors: No known data errors

once resilvering of new ada0 is complete, do the same for second mirror-disk ada1:
# zpool replace pdata1 ada1
Wait a few hours (ca 5 for 3TB). Finally pool looks like this:

# zpool status

  pool: pdata1
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: resilvered 2.10T in 0 days 06:31:19 with 0 errors on Fri Oct  8 04:32:21 2021
config:

        NAME        STATE     READ WRITE CKSUM
        pdata1      ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            ada0    ONLINE       0     0     0
            ada1    ONLINE       0     0     0

errors: No known data errors

Expand pool to new disk size (from 3TB to 6TB disks)

show unmodified pool on new, larger disks. Note field EXPANDSZ which tells us we have 2.72TB of unused disk-capacity in pool.

# zpool list

NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
pdata1  2.72T  2.10T   633G        -     2.72T    45%    77%  1.00x  ONLINE  -

Need to expand the pool’s size. We’ll do that by expanding the capacity on each device.

# zpool online -e pdata1 ada0
# zpool online -e pdata1 ada1

# zpool list
NAME     SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT  
pdata1  5.45T  2.11T  3.35T        -         -    22%    38%  1.00x  ONLINE  -

thikade/zfs_cheatsheet.md