While reading through some newsfeeds today, I came across a small comment by someone trying to compress a 700Mb mbox file full of spam emails. He said he was able to compress it down to 70Mb using a small utility called “rzip”. That’s quite a nice compression ratio, even beating out “bzip” which I thought was an awesome compressor. This is from the “rzip” site:
rzip is a compression program, similar in functionality to gzip or bzip2, but able to take advantage long distance redundencies in files, which can sometimes allow rzip to produce much better compression ratios than other programs. The original idea behind rzip is described in my PhD thesis (see http://samba.org/~tridge/), but the implementation in this version is considerably improved from the original implementation. The new version is much faster and also produces a better compression ratio.
The utility can be found at http://rzip.samba.org/