What is ReFS in Windows?

ReFS is derived from Resilient File System, is a file system created by Microsoft for Windows OS. It is designed to overcome some of the limitations of the New Technology File System (NTFS). ReFS is designed to be more resilient against data corruption, perform better for certain workloads, and scale better for very large file systems.

History of ReFS

Microsoft introduced ReFS in September 2012, when it released Windows Server 2012 as an optional file system to account for larger data storage requirements and larger workloads. Microsoft added ReFS to desktop versions of Windows starting with Windows 8.1 in October 2013 but could only be used as part of the drive-pooling Storage Spaces feature. And is also included in Windows 10. Microsoft removed the ability to create ReFS volumes in Windows 10 in its 2017 Fall Creators Update, except for Windows 10 Pro for Workstations. But the OS will still read ReFS.

How ReFS works

ReFS is highly compatible with NTFS and was built on the base of NTFS for more consistency & better data integrity. If the OS detects a system error & data is lost/corrupted then ReFS can restore the lost/corrupted data. The ReFS also combats bit rot i.e. the corruption of bits in a file system with time, through disk scrubbing tasks that read and validate data. Microsoft had used part of the NTFS codebase to produce ReFS & added support for Win32 APIs so that existing APIs are compatible, giving ReFS support for several features, such as access control lists & file IDs.

In NTFS, there are different reasons for file metadata corruption, such as an abrupt system shutdown during power loss, or when the system only writes part of a block. To avoid data damage ReFS uses allocate-on-write technique, which reads & writes files in a single instruction. The allocate-on-write technique uses thin provision clones of the source databases to reduce the chances of corruption.

A B+ tree structure is used by ReFS for the management of data through metadata indexing. The tree consists of the root, internal nodes, and leaves. A B+ tree stores data in a branching pattern, where each node in the tree has an ordered list of keys and pointers to lower-level nodes or leaves. Records are stored at the leaf level, which allows for more branches. This process lowers disk I/O and provides better performance.

ReFS is not a direct replacement for NTFS. ReFS does not support some NTFS features, such as master file table (MFT), page file support and object IDs. Some applications will only work with features only available on NTFS. Many NTFS features were not part of the earlier version of NTFS, such as alternative streams, named streams and sparse files, which Microsoft added to ReFS in later updates.

Notable features and changes

1. ReFS is integrated with the Storage Spaces feature. If a mirrored Storage Space using ReFS is set up, Windows will detect file system corruption & automatically repair the problems by copying the alternate copy of the data to another drive. This feature is available on both Windows 10 and Windows 8.1.

   

2. If dated corruption is detected by ReFS and it doesn’t have an alternate copy it can restore from, the file system can remove the corrupted data from the drive immediately. A reboot of the system is not required or taking the drive offline, as one would do with NTFS.
3. When reading and writing ReFS doesn’t just check files for corruption. All files on the drive are regularly checked by an automated data integrity scanner to identify & fix data corruption, too, which is an auto-correcting file system. No need to run chkdsk at all.
4. The new file system is also resistant to data corruption in other ways, too. For example, when you update a file’s metadata—the file name, for example—the NTFS file system will directly modify the file’s metadata. If your computer fails or the power goes out during this process, there may be data corruption. When you update a file’s metadata, the ReFS file system will create a new copy of the metadata. ReFS will point the file towards the new metadata only when the new metadata is written, so, no risk of the file’s metadata to be corrupted. This is known as “copy-on-write”. The copy-on-write technique is also used by other modern file systems e.g. ZFS & BtrFS on Linux & Apple’s new APFS file system.
5. ReFS uses checksums for metadata—and it can optionally use checksums for file data, too. Whenever it reads or writes a file, ReFS examines the checksum to ensure its correct. This means the file system itself has a built-in way to detect data corruption on the fly.
6. In addition to its data integrity feature, ReFS scales to larger volume and file sizes, compared with NTFS in the long term, these are important improvements.
7. ReFS supports a maximum volume size of 262,144 exabytes (16 exbibytes), compared with 16 exabytes in NTFS. ReFS supports up to 32,768 characters in a file name, compared with the limit of 255 characters in NTFS.
8. Other ReFS features include improved performance with Hyper-V, support of integrity steams to use checksums to assess the state of data and data striping for performance similar to RAID.
9. Microsoft added support for alternate data streams in version 1.2 to allow ReFS to work with Microsoft SQL Server deployments. Other notable updates include the support for the data deduplication — a storage optimization feature — in ReFS version 3.2 with the release of Windows Server version 1709 in the Semi-Annual Channel in October 2017.
10. ReFS also discards the DOS-style 8.3 filenames e.g. C:\Program Files\ at C:\PROGRA~1\ on an NTFS volume can still be accessed for compatibility purposes older technologies. These legacy file names are not supported on ReFS.
11. ReFS isn’t just an improvement of NTFS. Instead, Microsoft is focusing on a few important optimizations that make ReFS perform much better in certain cases.
12. ReFS when used with Storage Spaces, supports “real-time tier optimization”. You could have a drive pool with both drives optimized for performance and drives optimized for capacity. ReFS will always write to the drives in the performance tier, maximizing performance. In the background, ReFS will automatically move large chunks of data to the slower drives for long term storage.
13. On Windows Server 2016, ReFS have been improved to offer better performance with certain VM i.e. virtual machine features. Microsoft’s own Hyper-V virtual machine software takes advantage of these (and in theory, other virtual machine software could support them if they wanted to) like ReFS supports block cloning, which accelerates virtual machine cloning and checkpoint-merging operations. To create a cloned copy of a virtual machine, ReFS only needs to create a new copy of metadata on the drive and point it at an existing data on the drive. That’s because, with ReFS, multiple files can point to the same underlying data on disk. When the virtual machine changes and new data is written to the drive, it’s written to a different location and the original virtual machine data is left on the drive. This makes the cloning process much faster and requires much less disk throughput.
14. There is a new “sparse VDL” feature added to ReFS that allows ReFS to quickly write zeros to a large file. This makes it much faster to create a new, empty, fixed-sized virtual hard disk (VHD) file. Whereas this might take 10 minutes with NTFS, it can take just a few seconds with ReFS.
15. You’ll find more details about ReFS and the specific features it supports on Microsoft’s website.

Drawbacks to ReFS

All these features sound pretty good, but you can’t just switch to ReFS from NTFS.

1. Windows cannot boot from ReFS and require NTFS.
2. ReFS consumes more system resources, compared with NTFS.
3. The bigger the disk array, the more RAM and IOPS ReFS uses for file integrity.
4. NTFS data can’t be converted to ReFS.
5. ReFS also omits other features NTFS includes, including file system compression and encryption, hard links, extended attributes, data deduplication, and disk quotas. But ReFS is compatible with many other features of NTFS e.g. while encryption of specific data at the file system level could not be performed, ReFS is compatible with full-disk BitLocker encryption.
6. In Windows 10 you cannot format old partitions as REFS, either. You can currently only use ReFS with Storage Spaces, where its reliability features help protect against data corruption. Volumes could be formatted as ReFS instead of NTFS on Windows Server 2016. You might want to do this for a volume you plan on storing virtual machines on, for example. However, you still can’t use ReFS on your boot volume. Windows can only boot from an NTFS drive.
7. A ReFS drive does not support the installation of any apps or programs. The reason behind this is the non-support of hard links in ReFS & the very few programs which allow installation on ReFS volumes but even those programs also experience problems while running.

How to Use ReFS

1. On Windows Server, you can choose to format some volumes as ReFS using the normal disk management tools, which may be particularly useful if you’re using virtual machines on those drives. But you can’t format your boot drive as ReFS, and you’ll lose access to some NTFS features.

   

2. ReFS functionality is now available as part of Windows 10 Pro for Workstations