Re: [eigen] Alignment of derived matrix class

[Benoit Jacob <jacob.benoit.1@xxxxxxxxx>]

2010/3/6 Piotr Trojanek <piotr.trojanek@xxxxxxxxx>:
> I have send a problem report to QNX:
>
> http://community.qnx.com/sf/discussion/do/listPosts/projects.core_os/discussion.newcode.topc12472
>
> The Eigen alignment should be disable until QNX resolve this issue (or
> somebody will find a reasonable
> workaround to add in Eigen code).
>
> Thanks once again for your support!

You're welcome!

In Eigen 2.0, all alignment and all vectorization is disabled on QNX.

In the development branch, static alignment (aka alignment attributes)
is disabled on QNX. But heap alignmnet is kept and vectorization is
thus still enabled for dynamic-size matrices (like MatrixXd). Same for
SunCC, GCC3, etc. Likewise, the "Unaligned Array Assert" page now
tells the user to define EIGEN_DONT_ALIGN_STATICALLY which still
allows to vectorize dynamic-size matrices.

Benoit

>
> On Sat, Mar 6, 2010 at 11:06, Piotr Trojanek <piotr.trojanek@xxxxxxxxx>
> wrote:
>>
>> Thanks Benoit!
>>
>> Unfortunately the target architecture is plain old x86, but... I think
>> that finally I have traced this issue!
>>
>> It looks like alignment is not done for a threaded code (which is actually
>> exactly what happen in my
>> robotic code - the assertion fails in a thread created with
>> boost::thread). Please take a look at a simple
>> test code attached. It works perfect on Linux (x86, g++/icpc) and Solaris
>> (sparc, g++), but the assertion
>> for a BAD variable fails on QNX (x86, QCC/g++).
>>
>> Quick googling points to this old post for FreeBSD:
>>
>> http://lists.freebsd.org/pipermail/freebsd-threads/2004-November/002730.html
>> and similar for OpenSolaris:
>> http://defect.opensolaris.org/bz/show_bug.cgi?id=10932
>>
>> The problem seems, that gcc alignment works according to initial thread's
>> stack alignment.
>> On QNX this alignment is done for a main thread (and main() routine), but
>> not for a threads created
>> with pthread_create(). I believe this is a fault of QNX's C library, not
>> the compiler.
>>
>> I will try to dig into this issue and hope to find a solution (QNX code is
>> opensourced) and hopefully
>> send at least a problem report (or hopefully a patch) to QNX.
>>
>> Piotr
>>
>> On Sat, Mar 6, 2010 at 06:35, Benoit Jacob <jacob.benoit.1@xxxxxxxxx>
>> wrote:
>>>
>>> Just a BIG HUGE question...... what CPU architecture are you on...?
>>>
>>> We just discussed that GCC alignment attributes don't work on ARM....
>>> so if you are on ARM it is probably the same problem, not QNX's fault!
>>>
>>> Benoit
>>>
>>> 2010/3/5 Piotr Trojanek <piotr.trojanek@xxxxxxxxx>:
>>> > Yes, this is Eigen 2.0 branch, exactly the lastest release 2.0.12 with
>>> > all
>>> > my std:: patches
>>> > backported from main 2.0 branch.
>>> >
>>> > I have just completed one more test:
>>> >
>>> > 10) New alpha version of QNX, recently (just few days ago) pre
>>> > released,
>>> >
>>> > http://community.qnx.com/sf/wiki/do/viewPage/projects.community/wiki/SDP_6.5.0
>>> > which ships with gcc-4.4.2 based compiler does not solve the issue.
>>> >
>>> > Answering your suggestion - no I do not care about vectorization and I
>>> > can
>>> > surely live
>>> > with EIGEN_DONT_ALIGN. Even without vectorization Eigen is next century
>>> > product
>>> > comparing to the hand-crafted student's matrix libraries we have been
>>> > using
>>> > for a years :-)
>>> > The main advantage of Eigen is template-basing, which is what we need
>>> > for
>>> > real-time
>>> > robot control. Vectorization and performance are much less important.
>>> >
>>> > My proposal is as follows - it seems, that I am the only user of Eigen
>>> > on
>>> > QNX.
>>> > I will just add EIGEN_DONT_ALIGN into my framework's Makefile. If
>>> > anyone
>>> > double-report
>>> > my issues - I suggest to add #if (defined __QNXNTO__) test in Eigen
>>> > headers
>>> > to disable vectorization.
>>> >
>>> > At the beginning I was thinking, that the issue is caused by broken
>>> > inheritance,
>>> > but apparently it is caused by compiler or the OS.
>>> >
>>> > Thanks all for support!
>>> >
>>> > Piotr
>>> >
>>> > On Fri, Mar 5, 2010 at 15:34, Benoit Jacob <jacob.benoit.1@xxxxxxxxx>
>>> > wrote:
>>> >>
>>> >> This is Eigen 2.0 right?
>>> >>
>>> >> Piotr, do you care about vectorization in your case? If not, the
>>> >> simple fix is to disable alignment altogether (EIGEN_DONT_ALIGN) when
>>> >> QCC is detected. Acceptable to you?
>>> >>
>>> >> A more subte, yet possible, fix is to disable stack alignment
>>> >> (attributes) but keep heap alignment which will still allow to
>>> >> vectorize MatrixXf (but not Matrix4f). I'd only do that in the 2.0
>>> >> branch if that's really important to you...
>>> >>
>>> >> In any case, if QCC doesn't honor align attributes, we can't vectorize
>>> >> fixed size types like Matrix4f.
>>> >>
>>> >> Benoit
>>> >>
>>> >> 2010/3/5 Piotr Trojanek <piotr.trojanek@xxxxxxxxx>:
>>> >> > I have found this page:
>>> >> > http://eigen.tuxfamily.org/dox/WrongStackAlignment.html,
>>> >> > which seems to be based on this post:
>>> >> > http://forum.kde.org/viewtopic.php?f=74&t=61935
>>> >> > but it did not solves the issue:
>>> >> >
>>> >> > 4) my code compiled with -mstackrealign segfaults just at the
>>> >> > beginning
>>> >> > (backtrace points to some std::string internals)
>>> >> >
>>> >> > 5) option -mincoming-stack-boundary=2 is unrecognized by QCC (I
>>> >> > guess it
>>> >> > comes from gcc-4.4, while current version
>>> >> > of QCC compiler is based on gcc-4.3.1)
>>> >> >
>>> >> > The following options does not align data and fails with the
>>> >> > alignment
>>> >> > assert, line from gdb backtrace included:
>>> >> >
>>> >> > 6) -mpreferred-stack-boundary=2: ei_matrix_array (this=0x7f42e28)
>>> >> >
>>> >> > 7) -mpreferred-stack-boundary=3: ei_matrix_array (this=0x7f42e0c)
>>> >> >
>>> >> > 8) -malign-double: ei_matrix_array (this=0x7f42dc4)
>>> >> >
>>> >> > 9) both variants of forced alignment fails:
>>> >> > EIGEN_ALIGN_128 Eigen::Matrix<double, 6, 1> servo_real_kartez_pos;
>>> >> > __attribute__((aligned(16))) Eigen::Matrix<double, 6, 1>
>>> >> > servo_real_kartez_pos;
>>> >> >
>>> >> > It looks like QCC does only does alignment when it likes... (?).
>>> >> >
>>> >> > On Fri, Mar 5, 2010 at 14:10, Gael Guennebaud
>>> >> > <gael.guennebaud@xxxxxxxxx>
>>> >> > wrote:
>>> >> >>
>>> >> >>
>>> >> >> On Fri, Mar 5, 2010 at 1:58 PM, Piotr Trojanek
>>> >> >> <piotr.trojanek@xxxxxxxxx>
>>> >> >> wrote:
>>> >> >>>
>>> >> >>> Gael, thanks for your suggestions!
>>> >> >>>
>>> >> >>> I have verified the following:
>>> >> >>>
>>> >> >>> 1)  the code that breaks assertion is even the most simple variant
>>> >> >>> (so
>>> >> >>> this has nothing to do with inheritance):
>>> >> >>>
>>> >> >>> * Eigen::Matrix<double, 6, 1> servo_real_kartez_pos;
>>> >> >>
>>> >> >> not good!
>>> >> >>
>>> >> >> so what about:
>>> >> >>
>>> >> >> EIGEN_ALIGN_128 Eigen::Matrix<double, 6, 1> servo_real_kartez_pos;
>>> >> >>
>>> >> >> ??
>>> >> >>
>>> >> >> and if that does not work:
>>> >> >>
>>> >> >> __attribute__((aligned(16))) Eigen::Matrix<double, 6, 1>
>>> >> >> servo_real_kartez_pos;
>>> >> >>
>>> >> >>
>>> >> >> (paranoia)
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >>>
>>> >> >>> 2) for 100% right variant of macro is used:
>>> >> >>> #define EIGEN_ALIGN_128 __attribute__((aligned(16)))
>>> >> >>> (added #error to all the other variants).
>>> >> >>>
>>> >> >>
>>> >> >> ok
>>> >> >>
>>> >> >>>
>>> >> >>> 3) my code works just fine with -DEIGEN_DONT_ALIGN switch.
>>> >> >>>
>>> >> >>
>>> >> >> yes, in that case the assertions are removed.
>>> >> >>
>>> >> >>>
>>> >> >>> 4) alignment attribute is honored by QCC compiler (which uses
>>> >> >>> gcc-4.3.1
>>> >> >>> as a backend),
>>> >> >>> which I have verified with the following result of a test program
>>> >> >>> attached:
>>> >> >>>
>>> >> >>> # ../a.out
>>> >> >>> aligned: 8047b50
>>> >> >>> notaligned: 8047b98
>>> >> >>> In function nonaligned -- align.cc:24
>>> >> >>> (reinterpret_cast<std::size_t>(array) & 0xf) ==0 -- assertion
>>> >> >>> failed
>>> >> >>> Abort (core dumped)
>>> >> >>>
>>> >> >>> I will appreciate any idea... It looks like QCC compiler does
>>> >> >>> align
>>> >> >>> the
>>> >> >>> data only when it wants (?!).
>>> >> >>
>>> >> >> that's weird. And what if in this simple example you include Eigen,
>>> >> >> and
>>> >> >> using EIGEN_ALIGN_128 instead of the attribute ?
>>> >> >>
>>> >> >> and then, what if you replace the member array by a
>>> >> >> Eigen::Matrix<double,
>>> >> >> 6, 1> ?
>>> >> >>
>>> >> >> gael
>>> >> >>
>>> >> >>
>>> >> >>>
>>> >> >>> Piotr
>>> >> >>>
>>> >> >>> On Fri, Mar 5, 2010 at 09:35, Gael Guennebaud
>>> >> >>> <gael.guennebaud@xxxxxxxxx>
>>> >> >>> wrote:
>>> >> >>>>
>>> >> >>>> Hi Piotr,
>>> >> >>>>
>>> >> >>>> this sounds really strange to me. To help to understand the cause
>>> >> >>>> of
>>> >> >>>> the
>>> >> >>>> issue have you tried, or could you try the following:
>>> >> >>>>
>>> >> >>>> replace B servo_real_kartez_pos; by each of the following:
>>> >> >>>>
>>> >> >>>> * Eigen::Matrix<double, 6, 1> servo_real_kartez_pos;
>>> >> >>>> * A servo_real_kartez_pos;
>>> >> >>>> * EIGEN_ALIGN_128 B servo_real_kartez_pos;
>>> >> >>>>
>>> >> >>>> Check that sizeof(B)==sizeof(Eigen::Matrix<double, 6, 1>)
>>> >> >>>>
>>> >> >>>> gael
>>> >> >>>>
>>> >> >>>> On Fri, Mar 5, 2010 at 12:45 AM, Piotr Trojanek
>>> >> >>>> <piotr.trojanek@xxxxxxxxx> wrote:
>>> >> >>>>>
>>> >> >>>>> Dear Eigen experts,
>>> >> >>>>>
>>> >> >>>>> while porting some robotics matrix code to Eigen2 I have
>>> >> >>>>> encountered
>>> >> >>>>> the following problem
>>> >> >>>>> on QNX platform, but I do not think it is QNX specific.
>>> >> >>>>>
>>> >> >>>>> I have the following class hierarchy, which I have created using
>>> >> >>>>> guidelines from:
>>> >> >>>>> http://www.ros.org/wiki/eigen#Creating_typedefs_for_Eigen_types
>>> >> >>>>>
>>> >> >>>>> // Base class (with not copy-pasted robot-specific methods)
>>> >> >>>>> class A : public Eigen::Matrix<double, 6, 1> {
>>> >> >>>>> public:
>>> >> >>>>>     // Copy constructor from any Eigen matrix type
>>> >> >>>>>     template<typename OtherDerived>
>>> >> >>>>>     A(const Eigen::MatrixBase<OtherDerived>& other)
>>> >> >>>>>         : BaseClass(other)
>>> >> >>>>>     {}
>>> >> >>>>>
>>> >> >>>>>     // Reuse assignment operators from base class
>>> >> >>>>>     using BaseClass::operator=;
>>> >> >>>>>
>>> >> >>>>>     EIGEN_MAKE_ALIGNED_OPERATOR_NEW
>>> >> >>>>> };
>>> >> >>>>>
>>> >> >>>>> // Derived class type, which will hold additional methods
>>> >> >>>>> class B : public A {
>>> >> >>>>> public:
>>> >> >>>>>     template<typename OtherDerived>
>>> >> >>>>>     B(const Eigen::MatrixBase<OtherDerived>& other)
>>> >> >>>>>         : Ft_v_vector(other)
>>> >> >>>>>     {}
>>> >> >>>>>
>>> >> >>>>>     EIGEN_MAKE_ALIGNED_OPERATOR_NEW
>>> >> >>>>> };
>>> >> >>>>>
>>> >> >>>>> When I run this code on Linux - it works just fine.
>>> >> >>>>> When I run this on QNX - it fails with Unalligned Array Assert..
>>> >> >>>>> The
>>> >> >>>>> backtrace is here:
>>> >> >>>>>
>>> >> >>>>> #0  0xb033dad1 in SignalKill () from
>>> >> >>>>> /usr/qnx641/target/qnx6/x86/lib/libc.so.3
>>> >> >>>>> #1  0xb032c82e in raise () from
>>> >> >>>>> /usr/qnx641/target/qnx6/x86/lib/libc.so.3
>>> >> >>>>> #2  0xb032ab08 in abort () from
>>> >> >>>>> /usr/qnx641/target/qnx6/x86/lib/libc.so.3
>>> >> >>>>> #3  0xb032ad29 in __assert () from
>>> >> >>>>> /usr/qnx641/target/qnx6/x86/lib/libc.so.3
>>> >> >>>>> #4  0x080534ec in ei_matrix_array (this=0x7f42db4) at
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>> /opt/qnx641/target/qnx6/mrlib/include/eigen2/Eigen/src/Core/MatrixStorage.h:43
>>> >> >>>>> #5  0x0805350f in ei_matrix_storage (this=0x7f42db4) at
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>> /opt/qnx641/target/qnx6/mrlib/include/eigen2/Eigen/src/Core/MatrixStorage.h:79
>>> >> >>>>> #6  0x0809936f in
>>> >> >>>>>
>>> >> >>>>> Matrix<Eigen::CwiseNullaryOp<Eigen::ei_scalar_constant_op<double>,
>>> >> >>>>> Eigen::Matrix<double, 6, 1, 2, 6, 1> > > (
>>> >> >>>>>     this=0x7f42db4, other=@0x7f42ccc) at
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>> /opt/qnx641/target/qnx6/mrlib/include/eigen2/Eigen/src/Core/Matrix.h:405
>>> >> >>>>> #7  0x080981a4 in A (this=0x7f42db4) at mrmath/ft_v_vector..cc:28
>>> >> >>>>> #8  0x0809820f in B (this=0x7f42db4) at
>>> >> >>>>> mrmath/ft_v_vector.cc:185
>>> >> >>>>> #9  0x08078293 in
>>> >> >>>>>
>>> >> >>>>> mrrocpp::edp::common::manip_effector::compute_servo_joints_and_frame
>>> >> >>>>> (this=0x80d7fc0) at edp_e_manip.cc:55
>>> >> >>>>>
>>> >> >>>>> The code, that causes the abort is just creating local,
>>> >> >>>>> temporary
>>> >> >>>>> variable of class B:
>>> >> >>>>> B servo_real_kartez_pos;
>>> >> >>>>>
>>> >> >>>>> I have checked, that QNX compiler honours EIGEN_ALIGN_128
>>> >> >>>>> statement
>>> >> >>>>> (I
>>> >> >>>>> am 100% sure,
>>> >> >>>>> checked with gdb for very simple structure alignment with and
>>> >> >>>>> without
>>> >> >>>>> this statement).
>>> >> >>>>>
>>> >> >>>>> The question is - where is my mistake? Is it correct to make
>>> >> >>>>> inherit
>>> >> >>>>> in
>>> >> >>>>> the above way?
>>> >> >>>>> The problem seems to be, that in frames #8 to #4 "this" is not
>>> >> >>>>> aligned
>>> >> >>>>> to 128bit boundary.
>>> >> >>>>> I have tried adding EIGEN_ALIGN_128 and virtual inheritance, but
>>> >> >>>>> this
>>> >> >>>>> did not solve the problem.
>>> >> >>>>>
>>> >> >>>>> Thanks in advance!
>>> >> >>>>>
>>> >> >>>>> --
>>> >> >>>>> Piotr Trojanek
>>> >> >>>>> Robot Control and Recognition Systems Team,
>>> >> >>>>> Institute of Control & Computation Engineering,
>>> >> >>>>> Warsaw University of Technology
>>> >> >>>>>
>>> >> >>>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> --
>>> >> >>> Piotr Trojanek
>>> >> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Piotr Trojanek
>>> >> >
>>> >>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Piotr Trojanek
>>> >
>>>
>>>
>>
>>
>>
>> --
>> Piotr Trojanek
>
>
>
> --
> Piotr Trojanek
>