Re: [AD] ex3d crash

["Bertrand Coconnier" <bcoconni@xxxxxxxxxx>]

This is the third patch to poly3d.c and I hope it is the last one :-) Well,
I cleaned up poly3d.c (I removed those stupid tests on _polygon_mode : the
file is now readable. After all, I can barely imagine who may want to
disable the subpixel and subtexel accuracy since it now works). But the real
bug comes from round-off errors. Subpixel and subtexel accuracy is far more
sensitive to round-off errors than less accurate routines. Moreover, the
GRGB asm function can generate out of range access to the rgb_map array if
the RGB values are overflowed (and by the way it generates a segmentation
fault). The bug didn't occur with the C version of the libary because
makecol8() is protected against overflowed RGB values.
It is not possible to easily remove those round-off errors since numbers
can't be stored with an infinite accuracy. So I have rewritten the asm
function in order to protect it against overflow (it now truncates the extra
bits of the RGB values).
The bug should now be fixed. Can you test the patch and tell me if it works
?

Bertrand.

Attachment: ex3dbug.diff
Description: Binary data