Failed to run raid array

README.md

Situation

System failed to boot with messages:

1. "Failed to run raid array"
2. cache_check is missing/not found

uname -a output:
Linux hostname 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u1 (2017-06-18) x86_64 GNU/Linux

Reason

Some kernel modules and binary files were missing in the boot image. They are related to lvmcache if there are LVs backed by dm-cache.

Solution

Add required modules dm-cache, dm-raid and raid1, as well as binary files cache_check in the package thin-provisioning-tools.

1. Install the thin-provisioning-tools package:
   apt-get install thin-provisioning-tools
2. Create a script in /etc/initramfs-tools/hooks/:

   #!/bin/sh
   
   PREREQ="lvm2"
   
   prereqs() {
       echo "$PREREQ"
   }
   
   case $1 in
   prereqs)
       prereqs
       exit 0
   ;;
   esac
   
   . /usr/share/initramfs-tools/hook-functions
   
   copy_exec /usr/sbin/cache_check
   copy_exec /usr/sbin/cache_repair
   
   manual_add_modules raid1
   manual_add_modules dm_raid
   manual_add_modules dm_cache
   manual_add_modules dm_cache_smq
   

   and make it executable: chmod u+x lvm_addition The script does two things:
   • Copy cache_check and cache_repair and their dynamically linked library into the initramfs.
   • Add raid1, dm_raid, dm_cache_smq into the initramfs.
     Note that they are independent to each other.
3. Run update-initramfs -u so the required modules and binary files are included in the initramfs.

Rescue the system

In case I break the system again, use the Debian Live CD to boot into a working system.

1. Use ip addr and ip route to connect to the Internet.
2. Mount the original file system, proc, sysfs and dev, and then chroot to it.

   ROOTFS=/mnt/rootfs
   make $ROOTFS
   mount -t ext4 SystemVG/SystemLV $ROOTFS
   mount -t proc proc $ROOTFS/proc
   mount -t sysfs sys $ROOTFS/sysfs
   mount -o bind /dev/ $ROOTFS/dev
   chroot $ROOTFS
   

   Some environment variables are likely to be missing. I didn't re-create them and the procedure worked.
3. Update the APT repository and install neccessary packages:

   apt-get update && apt-get install thin-provisioning-tools lvm2
   

4. Since the script /etc/initramfs-tools/hooks/lvm_addition existed so simply run update-initramfs

Issues

• The program update-grub which update the GRUB bootloader in the /boot partition may issue Error: invalid segments. The error was issued by invoking grub-probe but the error could not break the system and the booting. So simply ignore it.

Keywords

• lvmcache
• "Failed to run raid array"