FS#58914 - [libvirt] ZFS pool support missing (build time dependency?)

Attached to Project: Community Packages
Opened by Uwe Sauter (UweSauter) - Thursday, 07 June 2018, 20:37 GMT
Last edited by Christian Rebischke (Shibumi) - Sunday, 19 August 2018, 02:16 GMT
Task Type Feature Request
Category Packages
Status Closed
Assigned To Christian Rebischke (Shibumi)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

libvirt is missing ZFS pool support, possibly a build time dependency (see this old Debian bug [1]).
This is tested with libvirt 4.3.0-1 and 4.4.0-1.

If I try to create a storage pool based on ZFS I get the following error:

### snip ###
virsh # pool-define-as --name ZFS --source-name hpcnsauter --type zfs
error: Failed to define pool ZFS
error: internal error: missing backend for pool type 11 (zfs)
### snap ###


[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=827245
This task depends upon

Closed by  Christian Rebischke (Shibumi)
Sunday, 19 August 2018, 02:16 GMT
Reason for closing:  Implemented
Additional comments about closing:  Implemented. Checkout the newest libvirt version in [community]
Comment by loqs (loqs) - Thursday, 07 June 2018, 21:12 GMT
How could there be ZFS support when there is no ZFS support in the arch repositories?
Comment by Eli Schwartz (eschwartz) - Thursday, 07 June 2018, 22:48 GMT
  • Field changed: Task Type (Bug Report → Feature Request)
  • Field changed: Status (Unconfirmed → Assigned)
  • Field changed: Architecture (x86_64 → All)
  • Task assigned to Christian Rebischke (Shibumi)
We don't provide zfs packages in our repositories, it's legally problematic to provide binary kernel modules anyway, and if this has even a build-time dependency on that then there's no chance of supporting this in the repos. But that doesn't seem to be the case.

Looking at the code, this is detected in https://github.com/libvirt/libvirt/blob/e64f2fab920fe8a08e97b790cb5d959f3566aea3/m4/virt-storage-zfs.m4

Which builds this backend: https://github.com/libvirt/libvirt/blob/31869efe2a8059356559cebc3b18c7dfab438a46/src/storage/storage_backend_zfs.c

Since all it is doing is calling the zfs and zpool binaries, I guess there's no reason we couldn't manually pass define the AC_PATH_PROG for zfs/zpool, and build the backend on the off chance that users installed the AUR packages which provide these filesystem kernel modules (and the executables which drive them).

It's not an unreasonable request -- that being said, it's really up to Shibumi whether he wishes to do so.
Comment by loqs (loqs) - Thursday, 07 June 2018, 23:23 GMT
@eschwartz then how is that part of the build tested what is to stop a repeat of  FS#56677  ?
Comment by Eli Schwartz (eschwartz) - Friday, 08 June 2018, 03:08 GMT
I'm not sure what you mean.

The glusterfs plugin (despite them not really being plugins in the most important way) uses #include <glusterfs/api/glfs.h>

The zfs plugin does not link any foreign libraries. So there's no interaction with other packages to consider -- either it builds or it doesn't. If the plugin simply fails to work and no one notices then 1) upstream is broken, 2) we're no worse off than before.
Comment by loqs (loqs) - Friday, 08 June 2018, 11:06 GMT
I would rather not have any features enabled that can not be tested (without requiring a custom kernel and unsupported utils)
Usually such optional support would be indicated by an optdepends but as the package is in AUR that can not be done.
A question on the binary issue how is zfs any different from nvidia or wl which are packaged. wl is now on the live media so combined with the kernel as distributed.
Comment by Christian Rebischke (Shibumi) - Friday, 08 June 2018, 12:30 GMT
ZFS support will definitly _not_ work when I merge libvirt with libvirt-glusterfs.. and I agree with eli. I really see no need for ZFS here. ZFS is still a problem with linux because of licenses.. so users who want ZFS needs to recompile the whole kernel. So I guess `recompiling` libvirt with zfs instead of glusterfs support shouldn't be a problem. Any Comments on this?
Comment by Uwe Sauter (UweSauter) - Friday, 08 June 2018, 12:39 GMT
I don't need to recompile the kernel in order to use ZFS. I only need to compile the ZFS modules against the Arch kernel (or use the packages provided by the ArchZFS repository, which do exactly that).
Comment by Uwe Sauter (UweSauter) - Friday, 08 June 2018, 17:28 GMT
Again, to emphazise: you do not need to recompile the kernel in order to use ZFS. You only need to compile the ZFS modules and, depending on the ZFS version, the SPL modules. SPL is a compatibility layer that was recently mergend into ZFS.

The zfs and zpool binaries are both installed in /usr/bin.

And ZFS itself is not a problem, the licensing is. But I'm not asking you to ship a package with problematic licening, I'm asking that libvirt supports ZFS if the ZFS binaries are installed.

I understand the subject about testing but there is a solution to that, too. Some friendly fellow runs the ArchZFS repository from where you can install prebuild ZFS packages, independently distributed from the GPL'd kernel. [1]

Shibumi: could you elaborate on your comment that ZFS support would not work with a merged libvirt / libvirt-glusterfs?


[1] https://wiki.archlinux.org/index.php/Unofficial_user_repositories#archzfs
Comment by Christian Rebischke (Shibumi) - Wednesday, 18 July 2018, 19:08 GMT
Sorry for the delay,
Ok I have checked this. And it seems that I would need to push ZFS and zpool into community as build dependencies (that I will not). I don't think zfs will be ever in our repositories as long there are licensing issues with ZFS and Linux:

checking for zfs... no
checking for zpool... no
configure: error: We need zfs and zpool for ZFS storage driver

We can leave this bug ticket open for tracking the demand.
Comment by Eli Schwartz (eschwartz) - Wednesday, 25 July 2018, 01:31 GMT
> Since all it is doing is calling the zfs and zpool binaries, I guess there's no reason we couldn't manually pass define the AC_PATH_PROG for zfs/zpool, and build the backend on the off chance that users installed the AUR packages which provide these filesystem kernel modules (and the executables which drive them).

I told you exactly how you could do it, a month ago. If you're going to turn this down, how about it not be for completely incorrect reasons?

In fact, here's the PKGBUILD I used to successfully build a package that helpfully contains the file usr/lib/libvirt/storage-backend/libvirt_storage_backend_zfs.so as reported by checkpkg, so it is definitively possible. Just override the executable command detection by explicitly telling it where $ZFS and $ZPOOL are...

(It also fixes some jarring use of "two spaces for first-level indent, one tab for second-level indent"??? as well as removing ancient comments and duplicated makedepends/base-devel.)
Comment by Christian Rebischke (Shibumi) - Friday, 27 July 2018, 02:48 GMT
Thanks, you were right. I should check this maybe more carefully.
I have build the package. Please test the package: http://pkgbuild.com/~shibumi/libvirt-4.5.0-2-x86_64.pkg.tar.xz
Comment by Uwe Sauter (UweSauter) - Tuesday, 31 July 2018, 19:17 GMT
I installed the provided package. From the short test I did so far it looks good.
Thanks.

Loading...