| Re: [eigen] Parallel matrix multiplication causes heap allocation |
[ Thread Index |
Date Index
| More lists.tuxfamily.org/eigen Archives
]
On 2016-12-19 15:24, François Fayard wrote:
Did someone have a look at what blaze [1] does? They seem to be pretty advanced regarding parallelism -- they also parallelize "simple" things like vector addition, which if we trust their benchmarks [2] seems to be beneficial starting at something like 50000 doubles
You should not trust benchmarks, especially where they have been done by people who wrote the software :-)
Sure. They themselves write a small disclaimer, that they could not get
Eigen's openmp support to run properly (I'm not sure, if we actually had
a bug in beta1 regarding that).
Here is my attempt at this game, multiplying 10 000 x 10 000 "double" matrices on a Dual Xeon 2660-v4 (Haswell), 2x14 cores, Broadwell, 2400 Mhz memory, with the latest release of every single library (as of today) :
- MKL: 740 GFlops
- OpenBLAS: 540 GFlops
- Eigen: 440 GFlops
- Blaze: 440 GFlops
This machine is capable of: 2 x 14 (cores) x 2 (2 FMA ports) x 2 (FMA) x 4 (AVX2) x 2 (GHz) = 896 GFlops. Hyperthreading is turned off and the CPU frequency is blocked at 2 GHz.
On a MacBook Pro (2014), with a 4 core Haswell, I get:
- MKL: 170 GFlops
- Eigen: 130 GFlops
Interesting. And that confirms that we really should update our own
benchmarks (and probably run blaze's benchmark suite for comparison, to
see if it measures differently).
Can anyone spare his (up-to-date) CPU for a night?
For Blaze, for things such as vector addition, they also use streaming stores to speed up the process.
Might be interesting as well. Though I guess only useful for relatively
large vectors. Otherwise, the overhead due to mfence probably destroys
the benefit (Disclaimer: I never actually experimented with that).
Christoph
--
Dipl. Inf., Dipl. Math. Christoph Hertzberg
Universität Bremen
FB 3 - Mathematik und Informatik
AG Robotik
Robert-Hooke-Straße 1
28359 Bremen, Germany
Zentrale: +49 421 178 45-6611
Besuchsadresse der Nebengeschäftsstelle:
Robert-Hooke-Straße 5
28359 Bremen, Germany
Tel.: +49 421 178 45-4021
Empfang: +49 421 178 45-6600
Fax: +49 421 178 45-4150
E-Mail: chtz@xxxxxxxxxxxxxxxxxxxxxxxx
Weitere Informationen: http://www.informatik.uni-bremen.de/robotik