Re: [eigen] Tracking performance regressions

[Benoit Jacob <jacob.benoit.1@xxxxxxxxx>]

Wow, that's great!

Thanks a lot --- performance regression testing has been most needed for a long time.

It already looks like my prefetch changes are bad for Haswell and we likely need to make them ARM-only.

Benoit

2015-02-20 13:42 GMT-05:00 Gael Guennebaud <gael.guennebaud@xxxxxxxxx>:

Hi,

I've started a very simple approach to help us tracking performance regression. The code is in bench/perf_monitoring/gemm, and some results are there:

http://download.tuxfamily.org/eigen/perf-monitoring/

These plots have been generated from a single command line:

$ CXX_FLAGS=" -mfma " ./run_gemm.sh

Currently this permits to see the performance evolution of matrix*matrix products for float, double, and complex<double> for some predefined matrix sizes configurations (gemm/settings.txt) and some predefined changesets (gemm/changesets.txt).

The idea is to add in gemm/changesets.txt any new changeset that might explicitly affect the performance of matrix products.

Then, by adjusting the file gemm/settings.txt, one can track its favorite size configurations.

Currently all changesets are tested for every run, but in the very near future I plan to make the script tests the new ones only by default with the option to update previous ones to reduce false positives.

Since the generated pdf files are very small, we could try to install a very simple webservice for sharing the plot files.

In the future, we could try to couple this with Mozilla performance monitoring tools:

https://wiki.mozilla.org/Buildbot/Talos

https://wiki.mozilla.org/Perfomatic

cheers,

gael