Re: [eigen] Signed or unsigned indexing

Hi Benoit,

Thanks for your notes on undefined behaviour which are quite interesting, especially the "switch" one that looks scary. But we always come back to those "nasal deamons" and these stories around the big bad wolf that you hear a lot but you never really get in real life. Yes, with undefined behaviour, anything can happen, but I believe that it should be seen from a practical point of view:

-1- Do we know of any disaster (such as Ariane 5) that could have been avoided using signed or unsigned integers?

-2- Do we know of any security flaw that could have been avoided using signed or unsigned integers?

-3- Do we know of any bugs that could have been avoided using signed or unsigned integers?

For the third question, my answer is yes. I've heard of so many bugs caused by a misuse of unsigned integers that could have been avoided by using signed integers.

Now, I understand that you are concerned about the answers to question 1 and 2. I understand why people could be scared of "undefined behaviour". But I would like to get some concrete examples of things that really went wrong because of the usage of signed integers. I've never seen any, but I have to admit that I don't have any experience in security. If you look at the CERT C coding standard, they are equally concerned about warping behaviour of unsigned integers and undefined behaviour of signed integers.

As a side note, I would be more concerned about the fact that Eigen does not use any error reporting mechanism. For instance, I have no idea what could happen when I ask a Cholesky decomposition on a matrix that is symmetric but not positive. And it is not something that could be easily checked before asking for the decomposition.

François

On 20 Jan 2017, at 20:50, Benoit Jacob <jacob.benoit.1@xxxxxxxxx> wrote:

Hi Francois,

There is no question that unsigned overflow can already be very dangerous, leading to crashes etc.

"Undefined behavior" is its own class of danger, though. When the compiler sees undefined behavior, it can do absolutely anything... an old version of GCC was emitting code to run some some video game, to illustrate that point.

here is a little 'study' of the actual behavior of compilers on undefined behavior that I made while I was working at a Web browser vendor:

https://raw.githubusercontent.com/bjacob/builtin-unreachable-study/master/notes

That was actually a study of the effect of using __builtin_unreachable. But any kind of undefined behavior is in principle equivalent to that.

As you can see there, it's pretty crazy the optimizations that GCC does when it takes undefined behavior seriously, e.g.
- removing 'ret' instructions at the end of functions, allowing code to continue running past the end of a function's code!
- removing bounds checks on switch statements implemented as jump-tables, thus allowing to jump to unintended addresses!

Benoit