[eigen] news on the refactoring of the expression template mechanism

[Gael Guennebaud <gael.guennebaud@xxxxxxxxx>]

Hi,

this email is mostly intended to people following and interested by
Eigen's internals.

As most of you probably know, since the releases of Eigen 3.0 we
started a significant refactoring of the expression template mechanism
under the code name "evaluators" aka "bug99" [1].

After we came-up with initial proof-of-concept, Jitse started its
implementation into Eigen's code base and I recently (a few months
back) started to seriously work on it again. To keep the default
branch safe, these recent efforts are done in a separate repository
[2]. This refactoring is much more work than what I expected, but I've
reached a state where I think that the rest of the refactoring should
be quite mechanical. So I think it's time to describe the new design
and to discuss it before going too far.

To this end I prepared a pdf [3] that summarize the new concepts and
their true implementation inside Eigen. This pdf focuses on the
technical implementation details of the concepts, so for those who are
not familiar at all with expression templates, have a look at the
slides 39 to 78 of this presentation [4].
(Disclaimer: I choose to make slides to present the new design because
I initially thought that I would make a lot of diagrams. So finally
that's not an ideal format and I probably has better to simply update
the initial working note page [5] but that's all I got for today).

The main purpose here is to start discussions and get some early feedbacks.

A few remarks that are not in the slides:

- Currently, the two mechanisms co-exist via conditional compilations
(EIGEN_ENABLE_EVALUATORS and EIGEN_TEST_EVALUATORS). This means the
code is a bit messy. Maybe it's OK to already do a big cleaning pass
since we have a separate branch anyway?

- Currently I only focused on the Core and LU modules, and even for
these modules, the refactoring is not complete (e.g., visitor, deeper
cleaning in products is needed, image() and kernel() have to ported,
etc.). Other modules are broken.

- All unit-tests making exclusive uses of the Core and LU modules
compile and run fine with EIGEN_TEST_EVALUATORS.

- There are many TODO and FIXME notes in the code for which feedback
would be welcome.

I'll try to come up with a more precise DONE versus TODO list later,
but as pointed out in the pdf, there are still a few black areas, so I
don't have myself a precise idea of what's really DONE.

Gael

[1] http://eigen.tuxfamily.org/bz/show_bug.cgi?id=99
[2] https://bitbucket.org/ggael/eigen-evaluators
[3] https://dl.dropboxusercontent.com/u/260133/eigen/Eigen3.3_ExprTemplExplained.pdf
[4] http://downloads.tuxfamily.org/eigen/eigen_CGLibs_Giugno_Pisa_2013.pdf
[5] (old stuff)
http://eigen.tuxfamily.org/index.php?title=Working_notes_-_Expression_evaluator