[eigen] tracking performance regression |

[ Thread Index | Date Index | More lists.tuxfamily.org/eigen Archives ]

*To*: eigen <eigen@xxxxxxxxxxxxxxxxxxx>*Subject*: [eigen] tracking performance regression*From*: Gael Guennebaud <gael.guennebaud@xxxxxxxxx>*Date*: Mon, 19 Dec 2016 19:44:26 +0100*Dkim-signature*: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=o2hYaj0O3gREy79theA9Giohu8Wpdn6kr/W2R0v2xao=; b=QHJMyEApNwunjZfruvAi8ehcbFjsrx6QUEQTgShaNey6tDBjSQxd8sZ1XPn8s5og9X 00mQs90gRKy9WTJfw4x7KVO++LbQorzR5X+J8beaAy7MJ7GzTSCNtgsB5FfnoyHABUsw Xf2GgZFpVtMfOFmUnIwjvzZERH4dQtw/yI4VzckJd15nN4MTMJLs/yk9na5FILRxMPQR 87VFNkJofhPzXk0e7NzgOTr/fwsju+QtK174TR3u9ZvQjSrEXUl5mTtBkojACm2FvmUk lryiwUfbGE/ZKRGix3oC244DIoRML4qzsS/1tdeA74Y8FBoxnHeIhJ1PO3mshP6ggXuS 9OZg==

Hi,

we currently have thousands of unit tests checking the correctness of the implementation, and, in some cases, the numerical robustness.

Checking performance, and in particular tracking performance regression, is also very important for a library like Eigen, but this task is way more tricky, and there is no tools to help us.

A while back, I've started to hack some scripts in bench/perf_monitoring, and a few weeks ago I've finally extended them such that they are now quite usable (disclaimer: some parts are really naive, there are plenty of shortcomings, but with proper inputs, they do the job!). You can see the outcome there:

These scripts do not really track performance regression, but rather generate performance charts for various tests (gemm, gemv, cholesky, etc.), with various predefined matrix sizes, and for the abscisse a selected set of changesets. We can then visually examine the behavior of Eigen over time. This strategy is quite convenient because it allows to easily add new test and look back in the history to check whether some previous state of Eigen was faster for the given problem.

See the above page for details on how to run and adjust these scripts.

If someone want to take the lead on this aspect, then you are very welcome. The bash scripts can be improved in numerous ways, starting from rewriting them in python, we could add more tests, and we could also imagine a small web-apps to ease the navigation across the different benchs, hardware, etc.

Cheers,

Gael.

**Messages sorted by:**[ date | thread ]- Prev by Date:
**Re: [eigen] Parallel matrix multiplication causes heap allocation** - Next by Date:
**Re: [eigen] Parallel matrix multiplication causes heap allocation** - Previous by thread:
**Re: [eigen] Parallel matrix multiplication causes heap allocation** - Next by thread:
**[eigen] On a flexible API for submatrices, slicing, indexing, masking, etc.**

Mail converted by MHonArc 2.6.19+ | http://listengine.tuxfamily.org/ |