If you thought that 7-Zip provided excellent compression ratios, then you might be in for a treat since Dropbox released their new DivANS compression algorithm as an open-source project. Developers claim that DivANS can compress data more densely than 7-Zip, Brotli or even Zstandard.
The code makes extensive use of the Rust systems programming language, which is sponsored by Mozilla. Rust was designed to be safe and concurrent, which makes it an excellent language for developing this sort of software in.
DivANS uses new vector intrinsic technology that’s provided by the Rust platform. Since it’s multi-threaded, it shouldn’t be at all slow either.
Another newer technology called intermediate representation gives developers the freedom to force different compression algorithms together and therefore build better optimizers over time.
Needless to say, the project is always looking for more community involvement from those who would like to volunteer their time toward producing a better compression algorithm.
DivANS got released under the Apache License, which should help to spread the code since the license is quite permissive. That being said, derivative compression code based on DivANS may not technically have to be distributed with the same license. This is somewhat unlike the copyleft licenses that accompany GPL-based software releases.
One of the most interesting aspects of the algorithm is related to its compilation. DivANS was written in Rust, but it got compiled against WebAssembly (WASM). This might help to explain how there’s already a browser demo available.
Considering the immense amount of data that Dropbox stores on behalf of clients, it could be easy to see the benefits of a new compression method from their point of view. Even a reduction by 1 or 2 percent of the total amount of data Dropbox stores would provide theoretically massive savings. It could also reduce the amount of time sent transmitting data over a network back to users.