OpenZFS is a free and open-source storage platform licensed under CDDL, that encompasses the functionality of traditional filesystems and logical volume manager. It includes protection against data corruption, support for high storage capacities, efficient data compression, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, encryption, remote replication with ZFS send and receive, and RAID-Z. The OpenZFS project brings together developers from the illumos, Linux, FreeBSD and macOS platforms, and a wide range of companies via the annual OpenZFS Developer Summit.[3][4][5][6][7][8]
Founding members of OpenZFS include Matt Ahrens, one of the main architects of ZFS.[8] The OpenZFS project is an open source derivative of the Oracle ZFS project.[9] As of 2019[update], OpenZFS (on some platforms such as FreeBSD) is gradually being pivoted to be based upon ZFS on Linux, which has developed faster than other variants of OpenZFS and contains new features not yet ported to those other versions.[10] The merged code-base will include a number of new features, performance enhancements, and is proposed to be known as OpenZFS 2.0.[11][12]
The following is a list of key events to the development of ZFS and its various implementations, leading to the creation of OpenZFS as an umbrella project:[13][15]: 6, 15
2001: Closed-source development of ZFS started with two engineers at Sun Microsystems.
2005: ZFS source code was released as part of OpenSolaris.
2008: A port to FreeBSD was released as part of FreeBSD 7.0.
2008: Development of a native ZFS Linux port started, known as ZFS on Linux.
2009: Apple's ZFS project closed, and the MacZFS project continued to develop the code.
2010: OpenSolaris was discontinued, resulting in the further development of ZFS on Solaris being no longer open-source.
2010: illumos was forked from OpenSolaris as its open-source successor,[16][17] and continued to develop ZFS in the open. Ports of ZFS to other platforms continued pulling in upstream changes from illumos.
2012: Feature flags were introduced to replace legacy on-disk version numbers, enabling easier distributed evolution of the ZFS on-disk format to support new features.
2013: Coexisting with the stable version of MacZFS, its prototype generation (known as OpenZFS on OS X or O3X) uses ZFS on Linux as the new upstream codebase.[18][19]
2013: The first stable release of ZFS on Linux.[20]
2013: Official announcement of the OpenZFS as an umbrella project.[4][7] New features and fixes are regularly pulled into OpenZFS from illumos and pushed into all ports to other platforms, and vice versa.[13]
2016: Ubuntu 16.04 includes the open-source ZFS file system variant by default.
Ported versions (2005–2010)
As the FSF claimed a CDDL and GPL legal incompatibility in 2005, Sun's implementation of the ZFS file system wasn't used as a basis for the development of a Linux kernelmodule, it wasn't merged into the Linux kernel mainline, and Linux distributions did not include it as a precompiled kernel module.[21][22] As a workaround, FUSE, a framework that allows file systems to run in userspace, was used on Linux as a separation layer for which the licensing issues are not in effect, although with a set of its own issues that include performance penalty.[14][23] However, the April 2016 release of Ubuntu 16.04 LTS includes CDDL-licensed ZFS on Linux[24] as a kernel module that is maintained as a separate project, outside the Linux kernel mainline, claiming license compatibility.[25][26][27]
Apple/macOS
In the release version of Mac OS X 10.5, ZFS was available in read-only mode from the command line, which lacks the possibility to create zpools or write to them.[28] Before the 10.5 release, Apple released the "ZFS Beta Seed v1.1", which allowed read-write access and the creation of zpools;[29] however, the installer for the "ZFS Beta Seed v1.1" has been reported to only work on version 10.5.0, and has not been updated for version 10.5.1 and above.[30] In August 2007, Apple opened a ZFS project on their Mac OS Forge web site. On that site, Apple provided the source code and binaries of their port of ZFS which includes read-write access, but there was no installer available[31] until a third-party developer created one.[32] In October 2009, Apple announced a shutdown of the ZFS project on Mac OS Forge. That is to say that their own hosting and involvement in ZFS was summarily discontinued. No explanation was given. Apple eventually released the legally required, CDDL-derived, portion of the source code of their final public beta of ZFS, code named "10a286". Complete ZFS support was once advertised as a feature of Snow Leopard Server (Mac OS X Server 10.6).[33] However, by the time the operating system was released, all references to this feature had been removed from its features page.[34] Apple has not commented regarding the omission.
Apple's "10a286" source code release, and versions of the previously released source and binaries, have been preserved and new development has been adopted by the MacZFS project[35][36][37] to resume its development elsewhere. As of July 2012, MacZFS implements zpool version 8 and ZFS version 2, from the October 2008 release of Solaris. Additional historical information and commentary can be found on the MacZFS web site and FAQ.[38]
By upgrading from OpenSolaris snv_134 to both OpenIndiana and Solaris 11 Express, one also has the ability to upgrade and separately boot Solaris 11 Express on the same ZFS pool.[40]
BSD
macOS
OpenZFS on OSX (abbreviated to O3X) is an implementation of ZFS for macOS.[41] O3X is under active development, with close relation to ZFS on Linux and illumos' ZFS implementation, while maintaining feature flag compatibility with ZFS on Linux. O3X implements zpool version 5000, and includes the Solaris Porting Layer (SPL) originally written for MacZFS, which has been further enhanced to include a memory management layer based on the illumos kmem and vmem allocators. O3X is fully featured, supporting LZ4 compression, deduplication, ARC, L2ARC, and SLOG.[citation needed]
MacZFS is free software providing support for ZFS on macOS. The stable legacy branch provides up to ZFS pool version 8 and ZFS filesystem version 2. The development branch, based on ZFS on Linux and OpenZFS, provides updated ZFS functionality, such as up to ZFS zpool version 5000 and feature flags.[42][43]
A proprietary implementation of ZFS (Zevo) was available at no cost from GreenBytes, Inc., implementing up to ZFS file system version 5 and ZFS pool version 28.[44] Zevo offered a limited ZFS feature set, pending further commercial development; it was sold to Oracle in 2014, with unknown future plans.[citation needed]
DragonFlyBSD
Edward O'Callaghan started the initial port of ZFS to DragonFlyBSD.[45]
NetBSD
The NetBSD ZFS port was started as a part of the 2007 Google Summer of Code and in August 2009, the code was merged into NetBSD's source tree.[46]
FreeBSD
Paweł Jakub Dawidek ported ZFS to FreeBSD, and it has been part of FreeBSD since version 7.0.[47] This includes zfsboot, which allows booting FreeBSD directly from a ZFS dataset.[48][49]
FreeBSD's ZFS implementation is fully functional; the only missing features are kernel CIFS server and iSCSI, but the latter can be added using externally available packages.[50]Samba can be used to provide a userspace CIFS server.
FreeBSD 13.0-RELEASE switches ZFS implementation from illumos-based code base to the unified OpenZFS 2 code base.[51] This change allows FreeBSD to receive OpenZFS improvements much quicker.[52]
MidnightBSD
MidnightBSD, a desktop operating system derived from FreeBSD, supports ZFS storage pool version 6 as of 0.3-RELEASE. This was derived from code included in FreeBSD 7.0-RELEASE. An update to storage pool 28 is in progress in 0.4-CURRENT and based on 9-STABLE sources around FreeBSD 9.1-RELEASE code.[citation needed]
TrueOS (formerly PC-BSD)
TrueOS (formerly known as PC-BSD, now defunct[53]) was a desktop-oriented distribution of FreeBSD, which inherited its ZFS support.[citation needed]
pfSense, an open source BSD based router, supports ZFS, including installation and booting to ZFS pools, as of version 2.4.
XigmaNAS
XigmaNAS (formerly NAS4Free), an embedded open source network-attached storage (NAS) distribution based on FreeBSD, has the same ZFS support as FreeBSD, ZFS storage pool version 5000. This project is a continuation of FreeNAS 7 series project.[55]
Debian GNU/kFreeBSD
Being based on the FreeBSD kernel, Debian GNU/kFreeBSD has ZFS support from the kernel. However, additional userland tools are required,[56] while it is possible to have ZFS as root or /boot file system[57] in which case required GRUB configuration is performed by the Debian installer since the Wheezy release.[58]
As of January 31, 2013, the ZPool version available is 14 for the Squeeze release, and 28 for the Wheezy-9 release.[59]
Although the ZFS filesystem supports Linux-based operating systems, difficulties arise for Linux distribution maintainers wishing to provide native support for ZFS in their products due to potential legal incompatibilities between the CDDL license used by the ZFS code, and the GPL license used by the Linux kernel. To enable ZFS support within Linux, a loadable kernel module containing the CDDL-licensed ZFS code must be compiled and loaded into the kernel. According to the Free Software Foundation, the wording of the GPL license legally prohibits redistribution of the resulting product as a derivative work,[60][61] though this viewpoint has caused some controversy.[62][63]
ZFS on FUSE
One potential workaround to licensing incompatibility was trialed in 2006, with an experimental port of the ZFS code to Linux's FUSE system. The filesystem ran entirely in userspace instead of being integrated into the Linux kernel, and was therefore not considered a derivative work of the kernel. This approach was functional, but suffered from significant performance penalties when compared with integrating the filesystem as a native kernel module running in kernel space.[64] As of 2016, the ZFS on FUSE project appears to be defunct.
As of August 2014[update], ZFS on Linux uses the OpenZFS pool version number 5000, which indicates that the features it supports are defined via feature flags. This pool version is an unchanging number that is expected to never conflict with version numbers given by Oracle.[68]
KQ InfoTech
Another native port for Linux was developed by KQ InfoTech in 2010.[69][70] This port used the zvol implementation from the Lawrence Livermore National Laboratory as a starting point. A release supporting zpool v28 was announced in January 2011.[71] In April 2011, KQ Infotech was acquired by sTec, Inc., and their work on ZFS ceased.[72] Source code of this port can be found on GitHub.[73]
The work of KQ InfoTech was ultimately integrated into the LLNL's native port of ZFS for Linux.[72]
Source code distribution
While the license incompatibility may arise with the distribution of compiled binaries containing ZFS code, it is generally agreed that distribution of the source code itself is not affected by this. In Gentoo, configuring a ZFS root filesystem is well documented and the required packages can be installed from its package repository.[74]Slackware also provides documentation on supporting ZFS, both as a kernel module[75] and when built into the kernel.[76]
Ubuntu integration
The question of the CDDL license's compatibility with the GPL license resurfaced in 2015, when the Linux distribution Ubuntu announced that it intended to make precompiled OpenZFS binary kernel modules available to end-users directly from the distribution's official package repositories.[77] In 2016, Ubuntu announced that a legal review resulted in the conclusion that providing support for ZFS via a binary kernel module was not in violation of the provisions of the GPL license.[78] Others,[79] such as the Software Freedom Law Center[80] followed Ubuntu's conclusion, while the FSF and SFC reiterated their opposing view.[81][82]
Ubuntu 16.04 LTS ("Xenial Xerus"), released on April 21, 2016, allows the user to install the OpenZFS binary packages directly from the Ubuntu software repositories.[83][84][85][86] As of 2019[update], no legal challenge has been brought against Canonical regarding the distribution of these packages.
As of 2019, Ubuntu supports experimental installation of ZFS as a root filesystem, starting with the 19.10 release ("Eoan Ermine"), to support coexistence of a nearly pure ZFS OS with GRUB and other operating systems on the same disk.[87][88]
TrueNAS Scale
A version of TrueNAS by iXsystems, but based on Debian Linux still uses OpenZFS for main storage but adds some container support via Kubernetes & Docker. Furthermore, it will allow clustered Docker and ZFS via gluster. It is in Beta as of 22 June 2021.[89]
Microsoft Windows
A port of open source ZFS was attempted in 2010 but after a hiatus of over one year development ceased in 2012.[90] In October 2017 a new port of OpenZFS was announced by Jörgen Lundman at OpenZFS Developer Summit.[91][92]
Releases and feature histories
Within illumos and (from 2013) OpenZFS
2010: illumos founded as an open source successor,[93] and continued to develop ZFS in the open. Ports of ZFS to other platforms continued porting upstream changes from illumos.[citation needed]
2013: OpenZFS project founded, aiming at coordinated open-source development of ZFS. The OpenZFS project provides a common foundation for any interested groups and organizations to contribute and collaborate towards a common open source ZFS core, and in addition, to also maintain any specific code and validation processes needed for core ZFS code to work with their own individual systems.
Other ports and forks
The following is a list of open-source ZFS ports and forks:[67][94]
2009: Apple's ZFS project closed. The MacZFS project continued to develop the code.
Detailed release histories
With ZFS in Oracle Solaris: as new features are introduced, the version numbers of the pool and file system are incremented to designate the format and features available. Features that are available in specific file system versions require a specific pool version.[95][96]
Distributed development of OpenZFS involves feature flags[42] and pool version 5000, an unchanging number that is expected to never conflict with version numbers given by Oracle. Legacy version numbers still exist for pool versions 1–28, implied by the version 5000.[97] Illumos uses pool version 5000 for this purpose.[98][99] Future on-disk format changes are enabled / disabled independently via
feature flags.
Unchanging pool version to signify that the pool indicates new features after pool version 28 using ZFS feature flags rather than by incrementing the pool version
Originally, version numbers of the pool and file system were incremented as new features were introduced, in order to designate the on-disk file system format and available features. This worked well when a single entity controlled the development of ZFS, and this versioning scheme is still in use with the ZFS in OracleSolaris.[100][101]
In a more distributed development model, having a single version number is far from ideal as all implementations of OpenZFS would need to agree on all changes to the on-disk file system format. The solution selected by OpenZFS was to introduce feature flags as a new versioning system that tags on-disk format changes with unique names, and supports both completely independent format changes and format changes that depend on each other. A pool can be moved and used between OpenZFS implementations as long as all feature flags in use by the pool are supported by both implementations.[15]: 20, 26–27 [102]: 2–3 [103]
In OpenZFS, the pool version is permanently set to 5000, signifying that the pool indicates new features by setting or unsetting ZFS feature flags rather than by incrementing the pool version.[42] The number 5000 was chosen because it is expected to never conflict with version numbers given by Oracle. Legacy version numbers still exist for pool versions 1–28.[104][105][106] Future on-disk format changes are enabled / disabled independently via these feature flags.
Legacy version numbers still exist for pool versions 1–28, and are implied by the pool version 5000;[107] the initial proposal was to use 1000 as the pool version.[102]: 4 Future on-disk format changes are enabled and disabled independently via feature flags.
Feature flags are exposed as pool properties, following these naming scheme rules:[102]: 4
Format of the property name is feature@<org-name>:<feature-name>
<org-name> is the reverse DNS name of the organization that developed the feature, ensuring unique property names.
Property names can be shortened to feature@<feature-name> when they remain unambiguous.
For example, feature@com.foocompany:async_destroy is a valid property name, and it could be shortened to feature@async_destroy.[102]: 4
Each pool feature can be in either disabled, enabled, or active state. Disabled features are those that will not be used, and no on-disk format changes will be made; as a result, such features are backward-compatible. Enabled features are those that will be used, no on-disk format changes have been made yet, but the software may make the changes at any time; such features are still backward-compatible. Active features are those that have made backward-incompatible on-disk format changes to the pool.[102]: 5
When any pool feature is enabled, legacy version of the pool is automatically upgraded to 5000 and any other prerequisite features are also enabled. By default, new pools are created with all supported features enabled. In general, state of a feature can be changed from active back to enabled, undoing that way performed on-disk format changes and making the pool compatible again with an older OpenZFS implementation; however, for some features that might not be possible.[102]: 5, 9 [107]
On-disk format changes can be associated with either features for write or features for read. The former are the features that an OpenZFS implementation must support to be capable of writing to the pool, while supporting such features is not mandatory for opening the pool in read-only mode. The latter are the features that an OpenZFS implementation must support to be able to read from the pool or to just open it, because opening a pool is not possible without actually reading from it.[102]: 7
For example, async_destroy feature adds a new on-disk data structure to keep track of freed datasets, but an OpenZFS implementation does not need to know about this data structure to access the pool in read-only mode. Additionally, writing to a pool that has some features in active state is not possible by an OpenZFS implementation that does not support the same features.[102]: 7–8
A list of feature flags and which operating systems support them is available from the Open-ZFS.org Web site[108]
OpenZFS 2.0
Historically, OpenZFS has been implemented as a core ZFS code, with each operating system's team adapting it to fit into their projects. This led in some cases to feature stagnation and divergence of features and command lines, as different operating systems developed divergent features and bug fixes, often for a single platform rather than across all platforms. Over time, new feature development shifted from Illumos to Linux.[109] These new features and fixes then had to be backported to Illumos before they could be re-ported for FreeBSD.[109] But this was difficult because the Linux version also included many smaller changes, which were hard to disentangle.[109]
In 2018, it was agreed that OpenZFS development would be overhauled to remedy these issues.[109] Rather than try to import all the Linux changes to other platforms piecemeal, the entire Linux ZFS code would be 'pivoted' as a whole, with other platforms being based on the more actively developed Linux version.[109] A wide range of ported and new features, including many long-desired enhancements, would also be rolled out or ported across platforms, and future changes would be discussed across platforms before being implemented.[109] The plans included appropriate porting layers to prevent Linux, GPL or Linux-KPI shim code from being introduced to other platform kernels.[109]
The features in progress or ported for OpenZFS 2.0 is lengthy, and includes:
Faster rollout of enhancements and new features across platforms[109]
Log spacemap and other metaslab management enhancements - a project to re-implement ZFS' management of free space and "metaslabs" for much greater efficiency[12]
Deduplication enhancements – dedup-log (high speed deduplication), dedup table size limits, and deduplication table preloading (loaded fully at one time rather than piecemeal as needed), listed as "nice to have" in 2018, were all stated in April 2020 to be "coming along nicely" or largely complete[110]
^"Debian GNU/kFreeBSD FAQ". Can I use ZFS as root or /boot file system?. Archived from the original on January 18, 2019. Retrieved September 24, 2013.
^"Debian GNU/kFreeBSD FAQ". What grub commands are necessary to boot Debian/kFreeBSD from a zfs root?. Archived from the original on January 18, 2019. Retrieved September 24, 2013.
^Behlendorf, Brian (May 28, 2013). "spl/zfs-0.6.1 released". zfs-announce mailing list. Archived from the original on June 8, 2013. Retrieved October 9, 2013.
^ abMatt Ahrens; Brian Behlendorf (September 17, 2013). "LinuxCon 2013: OpenZFS"(PDF). linuxfoundation.org. Archived(PDF) from the original on June 7, 2020. Retrieved November 13, 2013.
^"ZFS on Linux". zfsonlinux.org. Archived from the original on May 22, 2019. Retrieved August 13, 2014.
^Dustin Kirkland (February 18, 2016). "ZFS Licensing and Linux". Ubuntu Insights. Canonical. Archived from the original on July 29, 2016. Retrieved June 30, 2016.((cite web)): CS1 maint: uses authors parameter (link)
^Are GPLv2 and CDDL incompatible?Archived March 1, 2016, at the Wayback Machine on hansenpartnership.com by James E.J. Bottomley "What the above analysis shows is that even though we presumed combination of GPLv2 and CDDL works to be a technical violation, there's no way actually to prosecute such a violation because we can't develop a convincing theory of harm resulting. Because this makes it impossible to take the case to court, effectively it must be concluded that the combination of GPLv2 and CDDL, provided you're following a GPLv2 compliance regime for all the code, is allowable." (February 23, 2016)
^GPL Violations Related to Combining ZFS and LinuxArchived June 5, 2016, at the Wayback Machine on sfconservancy.org by Bradley M. Kuhn and Karen M. Sandler"Ultimately, various Courts in the world will have to rule on the more general question of Linux combinations. Conservancy is committed to working towards achieving clarity on these questions in the long term. That work began in earnest last year with the VMware lawsuit, and our work in this area will continue indefinitely, as resources permit. We must do so, because, too often, companies are complacent about compliance. While we and other community-driven organizations have historically avoided lawsuits at any cost in the past, the absence of litigation on these questions caused many companies to treat the GPL as a weaker copyleft than it actually is." (February 25, 2016)
^Siden, Christopher (January 2012). "ZFS Feature Flags"(PDF). Illumos Meetup. Delphix. p. 4. Archived from the original(PDF) on April 3, 2013. Retrieved September 22, 2013.
^Siden, Christopher (January 2012). "ZFS Feature Flags"(PDF). Illumos Meetup. Delphix. p. 4. Archived from the original(PDF) on April 3, 2013. Retrieved September 22, 2013.
