12-2-4 利用 ZFS 和 Proxmox VE 自建 NAS

https://www.gitbook.com/book/nabula/-zfs-proxmox-ve-nas/details

參考下列幾篇

1https://forum.level1techs.com/t/how-to-create-a-nas-using-zfs-and-proxmox-with-pictures/117375

2.https://forum.proxmox.com/threads/openmediavault-installation-in-lxc-with-attached-hw-block-device.29918/

3.https://www.facebook.com/notes/chris-hsiang/pve-%E4%B8%8B%E5%AE%89%E8%A3%9D-openmediavault-lxc/10212757987895096/

4.https://newtoypia.blogspot.tw/2017/03/zfs.html

5.https://www.freebsd.org/doc/zh_TW/books/handbook/zfs.html#zfs-differences

6.https://github.com/lxc/lxd/blob/master/doc/storage.md

7.https://pthree.org/2012/04/17/install-zfs-on-debian-gnulinux/

先照第一篇跟著做到 part10

zpool list -v, zpool iostat -v 確定pool 有建成功

pvesm zfsscan 確定 pve有找到zfs

遇到

E:Failed to fetch https://enterprise.proxmox.com/…/pve-…/binary-amd64/Packages401 Unauthorized

執行

deb http://download.proxmox.com/debianstretch pve-no-subscription

rm -f /etc/apt/sources.list.d/pve-enterprise.list

因為我們要裝omv 所以跳到第2篇

注意 OMV 3 只能裝在 Debian8 不能用 Debian9

如果遇到 NO_PUBKEY 7E7A6C592EF35D13

apt-get update

apt-get install openmediavault-keyring postfix

apt-get update

apt-get install openmediavault

omv-initsystem

一直做到 5.1 記得 stop container

5.2 We shoult mount LXC container virtual hard drive for example into /mnt directory in ProxmoxVE

mount /dev/mapper/pve-vm–102–disk–1 /mnt/

大部分新手都卡在這步 ，我也是，跟據下面 c0mputerking 的回文，這步不用做

用 df 找subvol-100-disk1 在那來代換下列的命令

mount -t proc none /apool/subvol-150-disk-1/proc/

mount --rbind /dev/ /apool/subvol-150-disk-1/dev/

mount --rbind /sys/ /apool/subvol-150-disk-1/sys/

chroot /apool/subvol-150-disk-1/ /bin/bash

如果以我的例子來做就變成

mount -t proc none /storage/vmstorage/limited/subvol-100-disk-1/proc/

mount --rbind /dev/ /storage/vmstorage/limited/subvol-100-disk-1/dev/

mount --rbind /sys/ /storage/vmstorage/limited/subvol-100-disk-1/sys/

chroot /storage/vmstorage/limited/subvol-100-disk-1/ /bin/bash

這樣就繼續下面的步驟， 我是遇到3個 error，文章只有2個，不過照學它 mv掉

這樣就可以裝好 omv了

建立 visual disk

zfs create -V 8G storage/vmstorage/omv-disk

ls -al /dev/zvol/storage/vmstorage/ ## find omv-disk node

fdisk /dev/zd32 ## create omv-disk partition /dev/zd32p1

mount /dev/zd32p1 /mnt/omv-disk

找device id

ls -al /dev/zd*

nano /etc/pve/lxc/100.conf

加入

lxc.cgroup.devices.allow: b 230:32 rwm

lxc.cgroup.devices.allow: b 230:33 rwm

lxc.mount.entry: /dev/zd32p1 /dev/sda1 none bind,create=dir,optional 0 0

lxc.mount.entry=/storage storage/ none bind,create=dir,optional 0 0

這行為設定 host上 zfs pool storage 到omv container

nano /var/lib/lxc/100/mount-hook.sh

加入

mknod -m 777 ${LXC_ROOTFS_MOUNT}/dev/sda b 230 32

mknod -m 777 ${LXC_ROOTFS_MOUNT}/dev/sda1 b 230 33

開機自動mount zvol

在pve的/etc/fstab 加入

/dev/zd16p1 /mnt/omv-disk ext4 defaults 0 0

讓omv 在 web gui 自動mount /dev/sda1

root@omv:/# touch /etc/systemd/system/omvmount.service

root@omv:/# chmod 664 /etc/systemd/system/omvmount.service

root@omv:/# nano /etc/systemd/system/omvmount.service

root@omv:/# systemctl daemon-reload

root@omv:/# systemctl start omvmount.service

root@omv:/# systemctl enable omvmount

nano omvmount.service

[Unit]

Description=Stupid Storage Mount for OMV in lxc

[Service]

Type=oneshot

ExecStart=/bin/mount -a

[Install]

WantedBy=multi-user.target