On Tue, Feb 3, 2009 at 5:31 AM, Benoit Jacob
<jacob.benoit.1@xxxxxxxxx> wrote:
2009/2/3 Keir Mierle <mierle@xxxxxxxxx>:
> Is there a reason that 1e-11 is used for double precision instead of the
> more conventional 2.2e-16? And same for float precision; why 1e-5 instead of
> 1.2e-7 for floats?
Let me ask the question the converse way: is there a reason why the
"epsilon" 2.2e-16 should play any specific role here?
If coarser comparisons are needed, then they should be available as different names. Machine precision should indeed be e for which 1 + e = 1.
Matrix operations involve many, many floating point operations. The
imprecision accumulates and at the end of a matrix operation one
cannot hope to be withing the "epsilon" of the correct result.
Here's an example. Invert a random 1000x1000 matrix of double's,
uniformly picked in the interval [-1,1]. Check your computation:
double error = (matrix * matrix.inverse() - Identity ).norm();
If you used full pivoting LU, you'll get an error of the order of 1e-13
If you used partial pivoting LU, you'll get an error of the order of 1e-11
Now Eigen's internal choice of 1e-11 isn't meant for that kind of use
case; it's more meant for smaller internal operations like the ones
the full LU does in order to determine the rank. So the imprecision
doesn't get that big, but still it is much bigger than the epsilon
2.2e-16.
Certainly. But then there should be a different function available for machine epsilon.
In Eigen's unit tests, an even far coarser choice of precision is used
in order to check the result of large end-user operations for sanity.
Our unit tests generally don't aim to test precision, except for a few
of them (we should have more) which can compare precision to other
libraries such as GSL if they are found. (It's really hard to say
abstractly "this computation must have this precision").
Cheers,
Benoit