As a long term ZFS user I have been using it on most of my production and home servers. But how about client machines?. Bringing all its nice features (snapshots,clones,checksums etc) on my laptop, desktop machine sounds like a great idea.Since ZFS is not included in mainline Linux kernel some Linux distributions have decided to integrate ZFS (ZoL) into their repositories. Such examples is Ubuntu, Arch and Gentoo Linux. CentOS also supports ZFS via its ELREPO repository. This guide is about the last one, (CentOS) since that’s what I mainly use at my work. I decided to share my experience by writing this guide hoping that it would be interesting for you as well.
What you will need:
– Ubuntu 16.04 LTS Desktop Live CD.
– An existing CentOS7 installation (ext4,xfs or whatever).
– A spare hard drive to be used for the ZFS installation.
Initial preparation:
Connect the spare hard drive(where ZFS is to be configured) into the machine. This can be done either internally by using SATA cable or externally via a USB case for example.The spare hard drive capacity must be equal or larger than the source drive.
Use Ubuntu LiveCD to boot the machine and select “Try Ubuntu” option.
Once in the Ubuntu live environment, open the Terminal.
First thing we need to do is to download ZFS packages for Ubuntu:
root@ubuntu:/# apt-add-repository universe
‘universe’ distribution component enabled for all sources.
root@ubuntu:/# apt update && apt -y install zfs-initramfs
Now you should have all you need to create a ZFS pool, so let’s proceed further and that would be disk partitioning. Normally ZFS uses whole drives to save data, but in this case we need to reserve a small partition [~1MB] for GRUB to install the boot loader.So, we’ll have to create 2 partitions, one for GRUB and a second for ZFS. Assuming that the destination hard disk is sdb we use the following commands to create the partitions into it:
5a. First clear any previous partitions on the destination disk.
root@ubuntu:/#sgdisk –zap-all /dev/sdb
5b. Then create a boot partition to be used by GRUB. Use this for legacy (BIOS) booting.
root@ubuntu:/#sgdisk -a1 -n2:34:2047 -t2:EF02 /dev/sdb
5c. Now create the 2nd partition to be used for ZFS data.
root@ubuntu:/#sgdisk -n1:0:0 -t1:BF01 /dev/sdb5d. List partitionsroot@ubuntu:/#gdisk -l /dev/sdb
Number Start (sector) End (sector) Size Code Name
1 2048 488397134 232.9 GiB BF01 –> ZFS Data
2 34 2047 1007.0 KiB EF02 –> GRUB boot
Now that we got our hard drive partitioned, we need to create a ZFS pool onto it. One very important thing to note here is that ZFS does not play well with “/dev/sdb, /dev/sdc etc” namings for hard disks. What you will have to do is to use the “/dev/disk/by-id/xxxxxx” naming scheme.Be very careful at this point to select the correct hard drive and partition (you can use gdisk -l /dev/disk/by-id/
** List pool **
root@ubuntu:/#zpool listNAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
rpool 79.5G 274K 79.5G – 0% 0% 1.00x ONLINE /mnt
Some important things to note on the above command and its results are the following:
– The pool will be created with access time property disabled.
– It should not mount itself.
– The compression algorithm will be LZ4.
– Default mountpoint will be (/). This is going to be used to properly mount CentOS ROOT fs later.
– The alternate mount point will be (/mnt). This is the temporary mount point to be used during the live session (Ubuntu LiveCD) to mount the pool. It’s perfectly fine to select something else there, for example (/rpool). If you decide to do this, remember to replace (/mnt) with (/rpool) on the commands that will follow later in this guide.
[Optional] Create rest of datasets as needed.This step is used mainly to separate ROOT (/) system from other system directories, like /var/log and /home for example. This is to ensure that those folders contents will be preserved during rollbacks of the ROOT system (i.e after a failed system upgrade).
** Create ROOT datasets **
root@ubuntu:/#zfs create -o canmount=off -o mountpoint=none rpool/ROOT
root@ubuntu:/#zfs create -o canmount=noauto -o mountpoint=/ rpool/ROOT/centos
** Mount ROOT filesystem **
root@ubuntu:/#zfs mount rpool/ROOT/centos
root@ubuntu:/#df -h /mnt
Filesystem Size Used Avail Use% Mounted on
rpool/ROOT /centos 223G 9.2G 213G 5% /mnt
** Create rest of datasets **
It’s perfectly fine to not create some (or all) of these datasets but they will make your life much easier when in the future you will have to rollback the ROOT system and you need to preserve the content of the (/home) directory or (/var/log) directory for example. Another thing is that you will have to create most of them by using the “legacy” property. This means that these datasets will not auto mount themselves during boot but rather expect from (/etc/fstab) file to mount them.
root@ubuntu:/#zfs create -o mountpoint=legacy -o setuid=off rpool/home
root@ubuntu:/#zfs create -o mountpoint=legacy -o setuid=off rpool/centos-test2
root@ubuntu:/#zfs create -o mountpoint=legacy rpool/home/root
root@ubuntu:/#zfs create -o canmount=off -o setuid=off -o exec=off rpool/var
root@ubuntu:/#zfs create -o mountpoint=legacy -o com.sun:auto-snapshot=false rpool/var/cache
root@ubuntu:/#zfs create -o mountpoint=legacy rpool/var/log
root@ubuntu:/#zfs create -o mountpoint=legacy rpool/var/spool
root@ubuntu:/#zfs create -o mountpoint=legacy -o com.sun:auto-snapshot=false -o exec=on rpool/var/tmp
** Create a ZVOL to be used for SWAP **
This ZFS volume is going to be used as a SWAP partition. Make sure you adjust its size on your system needs.
root@ubuntu:/#zfs create -V 4G -o compression=zle -o logbias=throughput -o sync=always -o primarycache=metadata -o secondarycache=none -o com.sun:auto-snapshot=false rpool/swap
Copy content from original CentOS7 installation [source drive] to the ZFS pool. In the example below, my CentOS7 installation is located in a LV (root) in a VG named “centos_test”. The “sda” is a temporary mount point I created to mount LV.
** Create a mountpoint for mounting original CentOS installation disk, in this example that is “/sda” **
root@ubuntu:/#mkdir /sda
root@ubuntu:/#mount /dev/centos_test/root /sda
** Mount boot parition (sda1). This is where CentOS7 kernel,initramfs and GRUB files are located in. This is a separate ~500MB ext4 partition **
root@ubuntu:/#mount /dev/sda1 /sda/boot
** rsync all content from source disk to destination pool. In this step, we basically copy the whole CentOS7 installation from drive1[LVM/EXT4] to drive2[ZFS] **
root@ubuntu:/#rsync -avPX /sda/ /mnt/
** Unmount original (source) disk. From this point we don’t need our CentOS7 installation drive anymore, so it’s safe to unmount it **
root@ubuntu:/#umount -R /sda
Now we need to mount all previously created datasets into (/mnt) and prepare the chroot environment (CentOS). From this point we leave Ubuntu live environment and we chroot to the CentOS7 ZFS environment since there are still things to do like for example to modify the contents of (/etc/fstab) file and configure GRUB.
root@ubuntu:/# mount -t zfs zpool/var/log /mnt/var/log
root@ubuntu:/# mount -t zfs zpool/var/tmp /mnt/var/tmp
root@ubuntu:/# mount -t zfs zpool/var/cache /mnt/var/cache
root@ubuntu:/# mount -t zfs zpool/var/spool /mnt/var/spool
root@ubuntu:/# mount -t zfs zpool/home /mnt/home
root@ubuntu:/# mount -t zfs zpool/home/root /mnt/root
root@ubuntu:/#mount -o bind /dev /mnt/dev
root@ubuntu:/#mount -o bind /proc /mnt/proc
root@ubuntu:/#mount -o bind /sys /mnt/sys
root@ubuntu:/#chroot /mnt /bin/bash –login
First thing to check once in CentOS7 (chrooted) is to check its version.
root@centos-test:/# lsb_release -a
Distributor ID: CentOS Distributor ID: CentOS Description: CentOS Linux release 7.3.1611 (Core) Release: 7.3.1611 Codename: Core
Install the ZFS packages for CentOS as described here: https://github.com/zfsonlinux/zfs/wiki/RHEL-and-CentOS
** Take a note of the CenOS7 version.You will need that info for downloading **
** proper ZFS package as described in the url above. **
In this case, installed CentOS version is 7.3, so I’m downloading the ZFS package for that version.distro.
root@centos-test:/# lsb_release -a
root@centos-test:/# yum install http://download.zfsonlinux.org/epel/zfs-release.el7_3.noarch.rpm
root@centos-test:/# gpg –quiet –with-fingerprint /etc/pki/rpm-gpg/RPM-GPG-KEY-zfsonlinux
** Modify /etc/yum.repos.d/zfs.repo if needed.By default dkms type packages will be used.Please modify if you prefer kABI type packages. **
**** Install CentOS ZFS packages **
root@centos-test:/# yum install zfs
** Install the ZFS package of dracut to create a ZFS aware initramfs later **
root@centos-test:/# yum install zfs-dracut
** Modify the (/etc/fstab) file as follows (Remove any existing entries since they do not apply anymore to ZFS CentOS7 setup). **
rpool/var/cache /var/cache zfs defaults 0 0
rpool/var/log /var/log zfs defaults 0 0
rpool/var/spool /var/spool zfs defaults 0 0
rpool/var/tmp /var/tmp zfs defaults 0 0
rpool/home /home zfs defaults 0 0
rpool/home/root /root zfs defaults 0 0
/dev/zvol/rpool/swap none swap defaults 0 0
Configuring GRUB and initramfs (dracut)
** Modify dracut configuration as follows **
root@centos-test:/# vi /etc/dracut.conf
** comment out and modify this line as follows: add_dracutmodules+=”zfs”
** Finally generate a new ZFS aware initramfs. Make sure you use your own kernel version in dracut parameters! **
root@centos-test:/# dracut -f -M -kver 3.10.0-514.26.2.el7
Executing: /sbin/dracut -f -M –kver /boot/initramfs-3.10.0-514.26.2.el7.x86_64.img 3.10.0-514.26.2.el7.x86_64
dracut module ‘busybox’ will not be installed, because command ‘busybox’ could not be found!
dracut module ‘nbd’ will not be installed, because command ‘nbd-client’ could not be found!
dracut module ‘biosdevname’ will not be installed, because command ‘biosdevname’ could not be found!
*** Including module: bash ***
*** Including module: fips ***
*** Including module: modsign ***
….
*** Hardlinking files ***
*** Hardlinking files done ***
*** Generating early-microcode cpio image contents ***
*** Constructing AuthenticAMD.bin ****
*** Constructing GenuineIntel.bin ****
*** Store current command line parameters ***
*** Creating image file ***
*** Creating microcode section ***
*** Created microcode section ***
*** Creating image file done ***
*** Creating initramfs image file ‘/boot/initramfs-3.10.0-514.26.2.el7.x86_64.img’
done ***
***Important! If you get a message saying “unknown filesystem when running grub2-install command, then you will have to compile the latest version of grub from github and run that version of grub-install to install the boot loader ***
root@centos-test:/# vi /etc/default/grub
** Modify GRUB_CMDLINE_LINUX as follows (leave only defaults):
GRUB_CMDLINE_LINUX=”rhgb quiet”
** The following line is required to avoid an error when running grub2-mkconfig command later below **
root@centos-test:/# export ZPOOL_VDEV_NAME_PATH=YES
** Generate a new GRUB config **
root@centos-test:/# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file …
Found linux image: /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-514.26.2.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-71ce3f23d5324e69aba211b4405fbf4c
Found initrd image: /boot/initramfs-0-rescue-71ce3f23d5324e69aba211b4405fbf4c.img
Found linux image: /boot/vmlinuz-0-rescue-27a3968f98aa4670a8ce5e4c952d8f77
Found initrd image: /boot/initramfs-0-rescue-27a3968f98aa4670a8ce5e4c952d8f77.img
done
** Install bootloader on ZFS drive **
root@centos-test:/# grub2-install /dev/sdb
Installing for i386-pc platform.
Installation finished. No error reported.
That’s all, if everything went well you can now exit chroot environment, unmount ZFS datasets, export the pool and finally shutdown the machine and use ZFS disk as the boot disk (old hard drive can be unplugged).
root@centos-test#: exit
root@ubuntu#: umount -R /mnt
root@ubuntu#: zpool export rpool
root@ubuntu#: poweroff
Boot from ZFS disk and see if it works, good luck!
No comments:
Post a Comment