source: trunk/README-threaded.txt @ 78

Revision 78, 1.5 KB checked in by faltet, 3 years ago (diff)

Updated docs after the merge of the threaded code.

Line 
1Blosc supports threads now
2==========================
3
4It just happened: Blosc can be run in threaded mode for both
5compressing and decompressing.  However, threaded Blosc doesn't work
6better than the serial version in all cases and the reason is that
7threads do add a non-negligible overhead (most specially, the cost of
8synchronization between them).
9
10In order to reduce the overhead of threads as much as possible, I've
11decided to implement a pool of threads (the workers) that are waiting
12for the main process (the master) to send them jobs (basically,
13compressing and decompressing small blocks of the initial buffer).
14
15Despite this and many other internal optimizations in the threaded
16code, it does not work faster than the serial version for buffer sizes
17around 64/128 KB or less.  This is for Intel Quad Core2 (Q8400 @ 2.66
18GHz) / Linux (openSUSE 11.2, 64 bit), but your mileage may vary (and
19will vary!) for other processors / operating systems.
20
21In contrast, for buffers larger than 64/128 KB, the threaded version
22starts to perform significantly better, being the sweet point at 1 MB
23(again, this is with my setup).  For larger buffer sizes than 1 MB,
24the threaded code slows down, but it is still considerably faster than
25serial code.
26
27This is why Blosc falls back to use the serial version for such a
28'small' buffers.  So, you don't have to worry too much about deciding
29whether you should set the number of threads to 1 (serial) or more
30(parallel).  Just set it to the number of cores in your processor and
31your are done!
32
33Francesc Alted
342010-05-03
Note: See TracBrowser for help on using the repository browser.