Re: [eigen] Tracking performance regressions

I've updated the plots to include recent changes in matrix-matrix products, and I also run the tests on a core2 and nehalem:

http://download.tuxfamily.org/eigen/perf-monitoring/

The situation on core2 is pretty bad, but such CPUs are obsolete now. Nehalem is pretty old too, but perfs are still ok, except for complex<double>.

gael

On Fri, Feb 20, 2015 at 7:42 PM, Gael Guennebaud <gael.guennebaud@xxxxxxxxx> wrote:

Hi,

I've started a very simple approach to help us tracking performance regression. The code is in bench/perf_monitoring/gemm, and some results are there:

http://download.tuxfamily.org/eigen/perf-monitoring/

These plots have been generated from a single command line:
$ CXX_FLAGS=" -mfma " ./run_gemm.sh

Currently this permits to see the performance evolution of matrix*matrix products for float, double, and complex<double> for some predefined matrix sizes configurations (gemm/settings.txt) and some predefined changesets (gemm/changesets.txt).

The idea is to add in gemm/changesets.txt any new changeset that might explicitly affect the performance of matrix products.

Then, by adjusting the file gemm/settings.txt, one can track its favorite size configurations.

Currently all changesets are tested for every run, but in the very near future I plan to make the script tests the new ones only by default with the option to update previous ones to reduce false positives.

Since the generated pdf files are very small, we could try to install a very simple webservice for sharing the plot files.

In the future, we could try to couple this with Mozilla performance monitoring tools:

https://wiki.mozilla.org/Buildbot/Talos
https://wiki.mozilla.org/Perfomatic

cheers,
gael