Re: [eigen] news on the refactoring of the expression template mechanism

On Sat, Apr 12, 2014 at 2:07 PM, Jitse Niesen <jitseniesen@xxxxxxxxx> wrote:

On Fri, 21 Feb 2014, Gael Guennebaud wrote:

As most of you probably know, since the releases of Eigen 3.0 we
started a significant refactoring of the _expression_ template mechanism
under the code name "evaluators" aka "bug99" [1].
[...]

This refactoring is much more work than what I expected, but I've reached a state where I think that the rest of the refactoring should be quite mechanical. So I think it's time to describe the new design and to discuss it before going too far.

I had a good look and it seems to work.

thank you for having a look at it!

With products you managed to do the most difficult part. The next question for me is whether it works with sparse matrices; but since you managed to do diagonal matrices we can have some confidence.

It's a bit hard to tell though because there is still quite some evaluation logic in the _expression_ objects, so it's quite possible that there are still gaps in the evaluators even in the parts of the Core and LU modules that appear at first sight to be done.

I've started to migrate the sparse module too and also to remove the coeff* from the "ExpressionImpl" classes. Ideally, we would remove these "ExpressionImpl" classes, but I'm still not 100% sure that we never have to specialize the API, for instance for sparse block... We'll see.

To this end I prepared a pdf [3] that summarize the new concepts and
their true implementation inside Eigen. [...]

The main purpose here is to start discussions and get some early feedbacks.

The pdf all seems right to me, but I realize it's the details that are important here and only actually programming it will make clear whether the design is okay or not.

- Currently, the two mechanisms co-exist via conditional compilations
(EIGEN_ENABLE_EVALUATORS and EIGEN_TEST_EVALUATORS). This means the
code is a bit messy. Maybe it's OK to already do a big cleaning pass
since we have a separate branch anyway?

Indeed, how to continue this? I can see two possibilities:

1. To make sure we get the design right, we can continue in the evaluator branch with the Core and LU modules, make sure that everything works, get rid of the conditional compilations, remove the dead code from the _expression_ objects (e.g., implement expr.coeff() in terms of evaluators), refactor, etc. Once we are confident with the design we can continue with the other modules.

I think that we must also completely migrate the SparseCore module before being sure we got the design right.

2. The alternative would be to get all (supported?) modules in working order with EIGEN_TEST_EVALUATORS, at least in that the tests pass, and then merge the branches. At that point we will probably see a lot of issues when people start to use evaluators because our tests are not comrehensive.

The problem with 1 is the code duplication due to the two separate branches (conditional compilation in the same branch has more or less the same issue). It will be difficult to merge the two branches and make sure that all bug fixes from one branch are ported to the other branch , and the further the branches diverge the more difficult it will be.

Approach 2 avoids this issue, but has the problem that when we want to change the design for evaluators, we have to change it in all modules (not just Core and LU), which will be much more work.

Of course there are more possibilities, but I feel this is the main choice to make. What do you guys think?

Note that EIGEN_TEST_EVALUATORS is not only used to make use of the novel code, but also to remove dead code. So my feeling is to continue with EIGEN_TEST_EVALUATORS until the SparseCore module is migrated and all supposedly dead code are hidden by EIGEN_TEST_EVALUATORS.

gael