| 1 | =============================================================== |
|---|
| 2 | Announcing Blosc 1.0rc1 |
|---|
| 3 | A blocking, shuffling and lossless compression library |
|---|
| 4 | =============================================================== |
|---|
| 5 | |
|---|
| 6 | :Author: Francesc Alted i Abad |
|---|
| 7 | :Contact: faltet@pytables.org |
|---|
| 8 | :URL: http://blosc.pytables.org |
|---|
| 9 | |
|---|
| 10 | |
|---|
| 11 | What is new? |
|---|
| 12 | ============ |
|---|
| 13 | |
|---|
| 14 | Everything :-) This is the first public release of a project that |
|---|
| 15 | started more than a year ago and that, after very intensive testing |
|---|
| 16 | (several hundreds of TB compressed and decompressed without a glitch), |
|---|
| 17 | it is finally getting ready for public consumption. |
|---|
| 18 | |
|---|
| 19 | This is Release Candidate 1 for Blosc 1.0 release, so please test it |
|---|
| 20 | and report back any problem you may have with it. |
|---|
| 21 | |
|---|
| 22 | |
|---|
| 23 | What is it? |
|---|
| 24 | =========== |
|---|
| 25 | |
|---|
| 26 | Blosc [1]_ is a high performance compressor optimized for binary data. |
|---|
| 27 | It has been designed to transmit data to the processor cache faster |
|---|
| 28 | than the traditional, non-compressed, direct memory fetch approach via |
|---|
| 29 | a memcpy() OS call. Blosc is the first compressor (that I'm aware of) |
|---|
| 30 | that is meant not only to reduce the size of large datasets on-disk or |
|---|
| 31 | in-memory, but also to accelerate memory-bound computations. |
|---|
| 32 | |
|---|
| 33 | It uses the blocking technique (as described in [2]_) to reduce |
|---|
| 34 | activity on the memory bus as much as possible. In short, this |
|---|
| 35 | technique works by dividing datasets in blocks that are small enough |
|---|
| 36 | to fit in caches of modern processors and perform compression / |
|---|
| 37 | decompression there. It also leverages, if available, SIMD |
|---|
| 38 | instructions (SSE2) and multi-threading capabilities of CPUs, in order |
|---|
| 39 | to accelerate the compression / decompression process to a maximum. |
|---|
| 40 | |
|---|
| 41 | You can see some recent benchmarks about Blosc performance in [3]_ |
|---|
| 42 | |
|---|
| 43 | .. [1] http://blosc.pytables.org |
|---|
| 44 | .. [2] http://www.pytables.org/docs/CISE-12-2-ScientificPro.pdf |
|---|
| 45 | .. [3] http://blosc.pytables.org/trac/wiki/SyntheticBenchmarks |
|---|
| 46 | |
|---|
| 47 | |
|---|
| 48 | Download sources |
|---|
| 49 | ================ |
|---|
| 50 | |
|---|
| 51 | Please go to: |
|---|
| 52 | |
|---|
| 53 | http://blosc.pytables.org/sources/ |
|---|
| 54 | |
|---|
| 55 | and download the stable release from here. |
|---|
| 56 | |
|---|
| 57 | Blosc is distributed using the MIT license, see LICENSES/BLOSC.txt for |
|---|
| 58 | details. |
|---|
| 59 | |
|---|
| 60 | |
|---|
| 61 | ---- |
|---|
| 62 | |
|---|
| 63 | **Enjoy data!** |
|---|