Re: [eigen] Levenberg-marquardt and dogleg minimizer

On Fri, Feb 20, 2009 at 2:53 PM, Timothy Hunter <tjhunter@xxxxxxxxxxxx> wrote:

Hello Keir,
thank you this post, it was a very interesting reading. I was trying to find a good way to represent the gradient and the hessian of a function for a convex solver, and I like your way of separating the jacobian from the function object itself.

I have a failed attempt which is not posted going down the other path :)

I have a few suggestions:
- in the Jacobian class, instead of having a boolean as a second template type, you may want a integer, so that different methods can be implemented in the future

This is a good idea, and as a bonus, will make calling code clearer, i.e.

NumericJacobian<MyFunc, CENTRAL_DIFFERENCE> instead of NumericJacobian<MyFunc, false>

I will make this change.

- in a function, the typenames XMatrixType and FMatrixType are not so obvious. Maybe InputType and ReturnType?

I agree that FMatrixType and XMatrixType are not great, but I am not convinced InputType and ReturnType are better. Arguably the return type is return type of the minimization, which is the same type as the parameters (x in this case) and not the type of f(x). Note that inside the LevenbergMarquardt class, I typedef things to Parameters and FVec. Again, not optimal. I will think about this more.

I'm open to better ideas.

Thanks for your suggestions!

Keir

Best regards

Timothy Hunter

On Thursday 19 February 2009 09:12:10 Keir Mierle wrote:
> The API for the LM implementation in libmv is more or less ready; since I
> plan to include it in Eigen at some point, I thought I'd send it along:
>
> http://code.google.com/p/libmv/source/browse/trunk/src/libmv/numeric/levenb
>erg_marquardt.h
> http://code.google.com/p/libmv/source/browse/trunk/src/libmv/numeric/levenb
>erg_marquardt_test.cc
>
> Mostly, I am looking for a review from an API perspective; also, I need to
> support sparse matrices at some point but don't have any experience with
> them; perhaps Gael could comment here as he is the sparse expert. In
> particular, I will need to support sparse Jacobians and solving sparse
> normal equations.
>
> I also have an implementation of Powell's dogleg method, which can be much
> faster than LM:
>
> http://code.google.com/p/libmv/source/browse/trunk/src/libmv/numeric/dogleg
>.h
> http://code.google.com/p/libmv/source/browse/trunk/src/libmv/numeric/dogleg
>_test.cc
>
> Keir

--
Timothy Hunter
Student (Stanford University)

T. 404 421 3075