Re: [eigen] Parallel matrix multiplication causes heap allocation

[ Thread Index | Date Index | More Archives ]

On 2016-12-19 15:24, François Fayard wrote:

Did someone have a look at what blaze [1] does? They seem to be pretty advanced regarding parallelism -- they also parallelize "simple" things like vector addition, which if we trust their benchmarks [2] seems to be beneficial starting at something like 50000 doubles

You should not trust benchmarks, especially where they have been done by people who wrote the software :-)

Sure. They themselves write a small disclaimer, that they could not get Eigen's openmp support to run properly (I'm not sure, if we actually had a bug in beta1 regarding that).

Here is my attempt at this game, multiplying 10 000 x 10 000 "double" matrices on a Dual Xeon 2660-v4 (Haswell), 2x14 cores, Broadwell, 2400 Mhz memory, with the latest release of every single library (as of today) :
- MKL: 740 GFlops
- OpenBLAS: 540 GFlops
- Eigen: 440 GFlops
- Blaze: 440 GFlops
This machine is capable of: 2 x 14 (cores) x 2 (2 FMA ports) x 2 (FMA) x 4 (AVX2) x 2 (GHz) = 896 GFlops. Hyperthreading is turned off and the CPU frequency is blocked at 2 GHz.
On a MacBook Pro (2014), with a 4 core Haswell, I get:
- MKL: 170 GFlops
- Eigen: 130 GFlops

Interesting. And that confirms that we really should update our own benchmarks (and probably run blaze's benchmark suite for comparison, to see if it measures differently).
Can anyone spare his (up-to-date) CPU for a night?

For Blaze, for things such as vector addition, they also use streaming stores to speed up the process.

Might be interesting as well. Though I guess only useful for relatively large vectors. Otherwise, the overhead due to mfence probably destroys the benefit (Disclaimer: I never actually experimented with that).


 Dipl. Inf., Dipl. Math. Christoph Hertzberg

 Universität Bremen
 FB 3 - Mathematik und Informatik
 AG Robotik
 Robert-Hooke-Straße 1
 28359 Bremen, Germany

 Zentrale: +49 421 178 45-6611

 Besuchsadresse der Nebengeschäftsstelle:
 Robert-Hooke-Straße 5
 28359 Bremen, Germany

 Tel.:    +49 421 178 45-4021
 Empfang: +49 421 178 45-6600
 Fax:     +49 421 178 45-4150
 E-Mail:  chtz@xxxxxxxxxxxxxxxxxxxxxxxx

 Weitere Informationen:

Mail converted by MHonArc 2.6.19+