Re: [eigen] LeastSquares, Pseudo Inverse & Geometry

[Jitse Niesen <jitse@xxxxxxxxxxxxxxxxx>]

On Mon, 25 Jan 2010, Thomas Capricelli wrote:

> I feel uncomfortable with the name of the module "LeastSquares"
> http://eigen.tuxfamily.org/dox-devel/group__LeastSquares__Module.html
>
> It's nothing more than one function (and one variant) computing linear 
> regression, which is I think a very special case of least squares. The 
> API has a 'geometry' feeling more than an optimization or statistical 
> one. (that's what i'm interested about, you'll have guessed).
>
> What the (currently still unsupported) NonLinearOptimization module does 
> is also least squares, in the non linear case.
>
> Though even in the linear case, this is different from 
> http://en.wikipedia.org/wiki/Least_squares, which also is something that 
> I need. I dont really understand how far it's different, but at least 
> the computation is different : the current LeastSquares module uses 
> eigenvalue, while this article (and the stuff I do) uses pseudo-inverse.

I seem to remember that the LeastSquares module does total least squares, 
and not ordinary least squares. At least, that is what I claimed in 
http://listengine.tuxfamily.org/lists.tuxfamily.org/eigen/2009/05/msg00158.html

In two dimensions, both ordinary least squares (OLS) and total least 
squares (TLS) fit a line y = ax+b through some data (x_i,y_i). OLS chooses 
the line such that sum_i (ax_i+b-y_i)^2 is minimized; that is the sum of 
the squares of the distance along the y-direction between the data points 
and the line. TLS minimizes the sum of the squares of the distance between 
the data points and the line perpendicular to the line.

My memory is hazy and I'm not sure I remember correctly, but 
perhaps this puts you in the right direction.

Cheers,
Jitse