Normal view

There are new articles available, click to refresh the page.
Before yesterdayTrueNAS

TrueNAS 24.10 “Electric Eel” Powers Up Your Storage

30 October 2024 at 02:33

After a tremendously successful and widely deployed BETA and RC, we’re pleased to announce that TrueNAS 24.10 “Electric Eel” has reached its official RELEASE version today, and is downloadable now from truenas.com/download-truenas-scale/ or by selecting to upgrade from within your existing SCALE installation.

Electric Eel succeeds Dragonfish (24.04.2.3), which is widely adopted in both Community and Enterprise installations. Dragonfish has become the most deployed version of TrueNAS, surpassing 13.0. TrueNAS 24.04 is also recommended for Mission-Critical deployments with significantly improved security and faster failover capabilities.

With over 9000 testers in our pre-release period, it’s no secret that Electric Eel is our most-anticipated release yet, and with good reason – it’s packed with several long-anticipated features like Docker Compose, both on the surface and under the hood, so let’s dig right in.

RAIDZ Expansion Is Here

One of the sticking points that we’ve heard from our community for years was the limitation that OpenZFS wasn’t able to expand its parity-based RAIDZ layouts by a single drive at a time.

After several years of intense development, testing, and debugging, we’re proud to announce that you can now pair the legendary resilience of OpenZFS with the easy expansion of conventional RAID solutions – drive-at-a-time expand is here.

Systems can be expanded online, one drive at a time, with no interruption in service – regardless of whether you’re using single, double, or triple-parity protection.

Traditional expansion using full vdevs is still available as before, and is the preferred method, but the new single-drive expansion offers new flexibility for smaller systems that may not have as many available drive bays.

Docker Touches Down with Improved Apps Handling

Since the initial launch of TrueNAS SCALE, Apps have played a major role in its adoption and flexibility.

Being able to run applications directly on the same system as their storage allows for both small “micro-service” style apps to leverage available power on a server, and for I/O-intensive applications to cut out network latency entirely from their workflow.

TrueNAS 24.10 migrates the previous Kubernetes-based Application back-end to the simpler Docker Compose solution, while seamlessly migrating and preserving the data of existing App installations.

If we haven’t built your preferred application out in our easy-to-install App catalog yet, or you’d like to customize it for your own specific needs, TrueNAS 24.10 also has full support for custom YAML config files (with the exception of individual IP addresses per application – coming in a post-release update) allowing you to import any of the hundreds of thousands of public Docker applications.

Want still more customization? Install the Dockge or Portainer runtimes on top of TrueNAS, directly from our App catalog – and tweak to your heart’s content.

For those who still want to leverage Kubernetes applications, a Kubernetes runtime can still be installed into a containerized or virtualized solution directly on TrueNAS; however, the primary method of App development and delivery will be through Docker and Docker Compose.

Fast Dedup Breaks Cover

Over a year ago, the TrueNAS development team and Klara Systems, along with members of the OpenZFS community, embarked on a journey to improve the data-reduction capabilities of OpenZFS through the Fast Deduplication project.

Several use cases can benefit from deduplication, including virtualization and office file storage where files may be copied to multiple locations by end-users; however, with the legacy OpenZFS deduplication algorithms, the overhead of maintaining the deduplication metadata tables in-memory at all times led to performance challenges and usability issues at scale.

Fast Dedup addresses these issues with multiple adjustments, including a more efficient metadata structure, a log-based write queue, and pruning of non-duplicate entries – all of which combine to shrink the memory footprint of deduplication by up to 90% in many scenarios.

The Fast Dedup feature is now ready for testing in TrueNAS 24.10, but is not recommended for serious production use at this time. We expect to provide testing results and any necessary code improvements in early 2025.

Global Search and Customizable Dashboard Widgets

Our new global UI search option helps you get to the settings you want faster than ever before. With just a few keystrokes, find the page you want, go there with a single click, and helpful highlights will appear to guide your eyes to the correct form, button, or area to explore next.

Can’t find what you want or need to dig deeper? Use the same menu to search the TrueNAS Docs site for more information. You can also use the new TrueNAS AI Search tool to ask more conversational questions and generate solutions to specific TrueNAS problems.

Usability and customization go hand-in-hand. While the TrueNAS team has designed a default dashboard with essential information, we know users have unique needs.

With our new customizable dashboard, you can place your most crucial information front and center, ready the moment you log in.

TrueNAS H-Series Gains NVMe Support

This spring, we launched the newest member of our TrueNAS Enterprise hardware family, the versatile TrueNAS H-Series, the perfect vehicle for delivering the power of TrueNAS in a compact, power-efficient package.

Now, the H-Series gets a jolt of extra horsepower from the release of TrueNAS 24.10 with the enabling of NVMe storage options on all twelve bays, bringing the maximum capacity of the H-Series to 360 TB using twelve 30 TB NVMe drives.

This new functionality is ready to be enabled in the field with an upgrade to Electric Eel; no controller replacement or component swaps needed. This tri-mode (SAS & NVMe) capability with High Availability is relatively unique in a 2U cost-effective platform.

New TrueNAS H-Series units configured with NVMe drives will ship with TrueNAS 24.10 already installed; existing TrueNAS Enterprise customers looking to take advantage of NVMe on H-Series platforms should reach out to our Support team to discuss an upgrade path that fits their needs.

Ready When You Are

The initial version, TrueNAS 24.10.0, is released and ready to download immediately.

Keep an eye on the Software Status page to see when your use case aligns with the new version, and when you’re ready, join the thousands of users already powering up with Electric Eel by downloading the installer or upgrading from within the TrueNAS UI; and don’t forget to stop by the TrueNAS Forums to share your knowledge and experience.

Join today and help others unlock the power of True Data Freedom with TrueNAS.

The post TrueNAS 24.10 “Electric Eel” Powers Up Your Storage appeared first on TrueNAS - Welcome to the Open Storage Era.

TrueNAS Delivers the Industry’s First Integration of OpenZFS 2.3

22 October 2024 at 03:44

TrueNAS uses OpenZFS as the foundation for its data management layer and is the deployment vehicle for the majority of OpenZFS storage systems used today. Here at TrueNAS, we love OpenZFS, and it continues to improve with the branching of OpenZFS 2.3 on October 4th, 2024.

The TrueNAS Engineering team has been significantly contributing to the codebase of OpenZFS 2.3, and Electric Eel (TrueNAS 24.10) includes several new and long-anticipated OpenZFS 2.3 features. The current development version of TrueNAS, “Fangtooth”, aligns with the full OpenZFS 2.3 release and will use this version of OpenZFS throughout its version lifecycle.

With every new version of both TrueNAS and OpenZFS, additional features, test cases, and bug fixes are included. The previous OpenZFS 2.2 brought dRAID and block cloning to TrueNAS Dragonfish (24.04) and CORE 13.3. TrueNAS 24.10 adds a number of new features, including two highly anticipated enhancements: Fast Dedup and RAIDZ expansion.

This blog highlights these new capabilities and their status within TrueNAS.

Fast Dedup Development is Complete

Deduplication is highly desirable for many workloads, including virtualization and several file storage use cases. Where there is naturally a high ratio of redundant data within a pool, deduplication effectively increases the usable capacity of the drives and the efficiency of the Adaptive Replacement Cache (ARC) and Level 2 ARC (L2ARC).

One of the primary challenges with traditional ZFS deduplication has been keeping the large deduplication metadata tables in memory at all times to avoid significant performance penalties. This existing functionality was not performant and led to usability issues during operation. As the size of the deduplication tables increased, the ZFS ARC would shrink, and performance would degrade as the pool filled, leading to a poor user experience and challenges when scaling to higher storage capacities.

With the inclusion of Fast Dedup, the metadata size can now be automatically constrained to fit in either primary RAM or dedicated flash devices to avoid hitting the performance penalty wall. In addition, the metadata structure for Fast Dedup has been completely re-engineered to enable efficient updates and the ability to prune non-duplicate blocks, effectively shrinking the memory footprint of the deduplication tables by 90% in many cases.

Combining these metadata improvements with properly configured storage will improve deduplication performance by an order of magnitude for larger systems. Performance is more stable as the pool is filled, leading to predictable behavior and enhanced space-efficiency.

This Fast Dedup project started in 2023 and was committed to the OpenZFS project as a “Valentine’s Day Gift” in 2024. Allan Jude and Klara Systems collaborated with Alexander Motin and the TrueNAS Engineering team along with members of the OpenZFS community, and development was completed in September of this year. We appreciate the hard work and dedication shown by all contributors and testers to help bring this project through to completion.

With development completed, Fast Dedup is now ready for testing but not yet suggested for serious production use. Within TrueNAS, it is marked as Experimental. We expect to provide test results along with any necessary code improvements in early 2025.

RAIDZ Expansion is Finally Available

A much-anticipated feature for smaller systems and home users of TrueNAS, RAIDZ expansion allows a small pool (e.g., a single RAIDZ vdev) to be gradually expanded with one drive at a time. Existing data is preserved with its original parity level and rewritten across all drives, while new data is written with the new parity configuration. This simplified administrative process gives smaller TrueNAS systems the flexibility to expand in single drive increments, rather than adding a full vdev of drives. The same expansion feature works regardless of the parity level used – RAIDZ1, Z2, or Z3 – but cannot migrate between protection levels.

The expansion process is done while the ZFS pool is online, similar to the resilvering process when a drive fails and is replaced. Once completed, the larger pool’s full performance is available. The new disk is used immediately, with additional capacity being reclaimed as existing data is rewritten.

This project took several years to complete and test, will be included in OpenZFS 2.3, and is available now in TrueNAS 24.10. TrueNAS sponsored this work to benefit smaller systems and is fully supported within TrueNAS Electric Eel.

OpenZFS Direct IO Improves NVMe Performance

Direct IO is one of the latest features included in OpenZFS 2.3 and was provided by Los Alamos Labs. It provides the option to bypass the ARC when storing data directly on NVMe drives. Removing memory copies can increase a system’s bandwidth by over 30%. The primary use case for Direct IO is storing checkpoint data in High-Performance Computing (HPC) clusters. This specific use case sees very little benefit from the read caching of the ZFS ARC (Adaptive Replacement Cache).

The Direct IO feature will only be available via Fangtooth. We are looking forward to testing for use cases that benefit our customers. Most workloads, however, will continue to benefit significantly from the default settings with ARC fully enabled.

What else is in TrueNAS 24.10?

In addition to the major features highlighted above, TrueNAS 24.10 includes an upgraded and improved webUI, enhancements to cloud backup integration, the replacement of Kubernetes with Docker for TrueNAS Apps, improved hardware support and drivers, and much more. For more details, see the Release Notes and join the discussion on the TrueNAS Forums, where some of the over 5,000 testers of 24.10 pre-release versions are sharing their feedback and tips.

TrueNAS 24.10 “Electric Eel” is planned for formal release near the end of October 2024. Want to learn more about TrueNAS solutions in your business? Contact us to speak to a product specialist, and find out how to harness the power of open source storage.

The post TrueNAS Delivers the Industry’s First Integration of OpenZFS 2.3 appeared first on TrueNAS - Welcome to the Open Storage Era.

Electric Eel is Now Feature-Complete

3 October 2024 at 01:21
TrueNAS Electric Eel (SCALE 24.10) is now feature-complete with the availability of the first Release Candidate. TrueNAS SCALE 24.10-RC.1 is now available for download, or update directly from within your existing TrueNAS installation. As always, please carefully read the Release Notes before deploying or updating your system.

Electric Eel emerged into the BETA stage on August 29th. In the last four weeks, a record number of users have volunteered to test Electric Eel and explore the new Docker-powered App capabilities. The Electric Eel uptake rate is over 3X Dragonfish and over 10X the 13.3 BETA versions.

The BETA process has proven that in-place App migration from Kubernetes to Docker will work for the existing catalog applications, with user data preserved. As of RC1, we have reached 100% in the Porting and Migration progress, as tracked on our GitHub Apps page. All previous Apps have been ported for new installs under the Docker framework, and users with existing Kubernetes-based installations from Dragonfish can automatically migrate during the TrueNAS upgrade process. Shout-out to our amazing Apps engineering team, as well as the community members who helped us test the migration processes during BETA. With this important migration milestone reached, we can now turn our attention to adding new Apps and Features in the coming months.

In addition to achieving parity between the old and new App catalogs, Custom Apps deployed under Dragonfish and earlier as Docker images can now be migrated to the native Docker framework in Electric Eel. More advanced settings such as manually installed Docker provisioning in a systemd-nspawn container using the community Jailmaker will need to be manually migrated.

What’s new in Electric Eel RC.1

We’d like to extend our thanks to our community members who jumped into the BETA release with both feet, eagerly testing, reporting, and helping us correct bugs. Our first BETA version was a success, and today’s RC.1 has 200 additional fixes and improvements.

Building on our community feedback, the engineering team has made several major improvements and has now completed the Electric Eel feature set. New functionality and changes in 24.10-RC.1 include:

  • Login Alerts on root/admin user login or attempted login
  • App logs are better captured and displayed in the UI
  • Improved App Info cards (now with links)
  • App Utilization (CPU/Memory/Network/Disk IO) displayed on the Apps page
  • New Dashboard is completed with better mobile support. The legacy “Old Dashboard” has been removed.
  • Custom App YAML Editor allows for custom application configurations to be deployed. (If a GUI is desired, we suggest deploying the built-in Portainer App.)
  • Custom App Migration is enabled for users who deployed Docker images in Dragonfish and earlier using the “Custom App” UI option
  • NVIDIA drivers are now handled in a more modular manner, and can be installed dynamically post-installation

Install the new modular NVIDIA drivers from the Apps -> Settings Page in 24.10

One of the major anticipated features of the Docker framework in Electric Eel that users have expressed interest in is the YAML editor for advanced Apps configuration. In 24.10-RC.1, the Custom App YAML editor now allows more complex Apps to be created and deployed through editing of the configuration file. For RC1, the ability to allocate a unique IP address for an installed App is not yet present. This functionality is planned as an App infrastructure update after the RELEASE version of Electric Eel is completed.

With BETA completed and now RC.1 released, the total feature set of Electric Eel can be summarized. We’re looking forward to more feedback (and bug reports!) from our community.

TrueNAS Electric Eel

You can look forward to more blog posts and emails highlighting these new features and upgrades in 24.10 – while many of them are already present in RC.1, some of these features won’t be ready until RELEASE, while others such as Fast Deduplication are labeled as Experimental and should be handled with care by early adopters and testers only.

With Electric Eel now feature-complete, the TrueNAS engineering team is focused on the development of the next release, “Fangtooth” in mid 2025. More information will be available at the end of 2024. Many thanks to those who submitted, and voted for, the Feature Requests that have already been adopted. If you have a specific feature or functionality that you feel would benefit TrueNAS, please feel free to submit it on our Community Forums, and vote for other suggested features to help us enhance 24.10 and beyond.

When Should I Migrate?

If you are deploying a new TrueNAS system, we recommend TrueNAS SCALE Dragonfish 24.04.2.2 for:

  • Added functionality over CORE
  • Vastly broader hardware support
  • Expanded App catalog (which will migrate to Electric Eel)
  • Sandboxes provide jail-like capabilities using systemd containers
  • Better performance on most workloads
  • Improved web UI makes managing TrueNAS easier than ever

Dragonfish users can easily update to Electric Eel RC.1 when desired, but at this point we only recommend it for early adopters. We recommend users review the TrueNAS Software Status page for advice.

If you’re ready to explore the Electric Eel Release Candidate, grab it from our downloads page now – and stay tuned for the upcoming full release!

The post Electric Eel is Now Feature-Complete appeared first on TrueNAS - Welcome to the Open Storage Era.

Enhanced Data Migration in Electric Eel

27 September 2024 at 00:05

One of the cornerstones of TrueNAS is how your data can be managed and manipulated. We believe that your data belongs to you, free from proprietary systems and vendor lock-in – part of our promise to what we call True Data Freedom.

While vendor lock-in can be deliberate and calculated on the part of the vendor, feeling locked-in to a storage ecosystem can often have more to do with simply fearing the time, effort, and pain of migrating the data to another vendor’s product, only to risk being stranded again in yet another different vendor’s ecosystem. While most vendors often provide tools to migrate to another one of their own products, they never make it simple to move to another vendor’s product.

To combat this, TrueNAS 23.10 introduced the Enterprise File Sync feature with Syncthing to enhance data mobility and simplify data migrations both onto – and off of – TrueNAS systems. TrueNAS 24.04 refined that feature set, and now TrueNAS 24.10 is poised to enhance it even further with the inclusion of SMB Alternate Data Stream (ADS) import. With the inclusion of SMB ADS migration, it will be easier than ever to migrate from a third-party NAS solution onto TrueNAS, or if you so please, off of TrueNAS onto something else.

With TrueNAS, the only way we want to lock you in is by providing an experience so good you don’t want to leave. And, if we can’t provide that, we provide the tools to help you do so.

What are Alternate Data Streams?

Have you ever downloaded a program or document from the internet, and received a warning message from your operating system that the file or some component was blocked because it was “untrusted” or “unsafe”?

Alternate Data Streams

This so-called “Mark of the Web” comes from the presence of an SMB Alternate Data Stream, in this case a flag attached to the file called “Zone.Identifier” that indicates the original URL of the file.

The Mark of the Web is just one of many Alternate Data Streams; with other applications choosing to store application data or metadata – ranging from simple timestamps to answer “when was this file last opened?” or more complex organization features such as MacOS file colors and tagging.

While TrueNAS has supported Alternate Data Streams when serving SMB shares for some time, Electric Eel now allows for the migration of these crucial pieces of metadata when importing data from third-party NAS solutions.

Migrate Data Easily From Any Compatible Third-Party NAS Solution

Using the Syncthing Enterprise application, TrueNAS 24.10 will have the ability to connect to a remote SMBv3 server directly from your TrueNAS installation. No plugin or service installation will be required on the source NAS server. A common identity service (such as Active Directory) must be used in order to synchronize security information and descriptors. If the two systems cannot use the same identity service, permissions will need to be updated after the migration. More information can be found on the TrueNAS Docs site under Third-Party Data Migration, and TrueNAS Enterprise customers with a valid support agreement can contact iXsystems for direct assistance.

Syncthing SMB migration between NAS systems

Using two copies of Syncthing Enterprise, TrueNAS can ensure that file consistency is maintained during the migration process, while keeping the source data available for use. Any changes made on the third-party system are automatically reflected on TrueNAS. No more manually running scripts or batch jobs, no more “pivot” systems in the middle of the data flow – simply enjoy the power and simplicity of TrueNAS and Syncthing working together.

Learn More

As we draw closer to the full release of TrueNAS 24.10, stay tuned for additional updates on new features and functionality that will arrive later this year. If you’re just starting out with your journey, you can download the current version of TrueNAS SCALE 24.04, and upgrade to 24.10 later this year.

For supported, Enterprise-ready solutions, check out our full line of TrueNAS systems, from the energy-efficient, highly-available H-series edge system to the performance flagship all-NVMe F-series. For help selecting and right-sizing the systems for your particular need, contact us directly to arrange a chat with one of our experts to learn more about how TrueNAS can help your organization.

The post Enhanced Data Migration in Electric Eel appeared first on TrueNAS - Welcome to the Open Storage Era.

❌
❌