Re: [eigen] segment fault in "Eigen::internal::gemm_pack_rhs" when call MatrixXd multiplication on a memory limited 32 bit platform

[ Thread Index | Date Index | More lists.tuxfamily.org/eigen Archives ]


Hi,

just speculating, but it could be a stack-overflow problem.
Can you check what your stack-limit is? Call this in your console: `ulimit -s`. The default for most linuxes today is 8192KiB, if it is smaller than that, this might be the reason. Is your matrix multiplication called inside a function, which already is relatively deep in your call-stack (with some functions having relatively big objects on the stack -- you can walk through your stack-trace and check the value of rsp/esp at each stage to find possible culprits).

If that is indeed the reason, you can try to reduce the space Eigen is allowed to allocate for temporaries on the stack, by defining EIGEN_STACK_ALLOCATION_LIMIT before including Eigen. (The default is 131072 == 128 KiB; maybe try 8192 bytes, then increase it until you get crashes again). This could significantly reduce the performance, since all temporaries above that size will be allocated on the heap.

Cheers,
Christoph



On 2017-07-29 07:38, extr15 wrote:
Hi Gael, thanks for your reply, I will try to reproduce a example but it's difficult.
as for your question:
1. it seems crash for matrix size bigger than about 100*100.
2. my program is multi-threaded.
3. the eigen version is 3.2.4, eigen-eigen-10219c95fe65, maybe I should try the latest stable release.
4. I don't think I can try it with valgrind, because the system has only 512MB memory, even running a gdb is not that easy.


Thank you again.



--

Best Regards!
--
extr15

At 2017-07-25 16:44:43, "Gael Guennebaud" <gael.guennebaud@xxxxxxxxx> wrote:

Hi,


since you cannot reproduce with a self-contained example this is gonna be difficult to fix. Some question to help isolate the issue:



1 - Does it crash for any problem size or for specific sizes only?


2 - Is your program multi-threaded?


3 - What is your exact version of Eigen? (lines are not matching the latest 3.3.4 stable release)


4 - You could try to run it within a memory debugger like valgrind.




Gael


On Thu, Jul 20, 2017 at 6:09 PM, extr15 <extr15@xxxxxxx> wrote:

Hello, it seems I can't log in "http://eigen.tuxfamily.org/bz/?GoAheadAndLogIn=1"; as I am a new user, so I write this mail to ask for your help, thanks!
I have a project which runs OK on my 64bit laptop( ubuntu), but when I run this program on a 32 bit linux platform, it will crash with the output:
"
*** Aborted at 2926 (unix time) try "date -d @2926" if you are using GNU date ***
PC: @   0x206384 Eigen::internal::gemm_pack_rhs<>::operator()()
segment fault
"
I use gdb to debug, and the "bt" output:
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Program received signal SIGSEGV, Segmentation fault.
[Switching to LWP 1138]
0x0020b65c in Eigen::internal::gemm_pack_rhs<double, int, 2, 0, false, false>::operator() (blockB=0xb6e3dc00, rhs=rhs@entry=0x1cda940, rhsStride=1071644672,
     rhsStride@entry=106, depth=depth@entry=106, cols=106,
     this=<optimized out>, offset=0, stride=0)
     at eigen3/Eigen/src/Core/products/GeneralBlockPanelKernel.h:1236
1236eigen3/Eigen/src/Core/products/GeneralBlockPanelKernel.h: No such file or directory.
(gdb) #0  0x0020b65c in Eigen::internal::gemm_pack_rhs<double, int, 2, 0, false, false>::operator() (blockB=0xb6e3dc00, rhs=rhs@entry=0x1cda940,
     rhsStride=1071644672, rhsStride@entry=106, depth=depth@entry=106,
     cols=106, this=<optimized out>, offset=0, stride=0)
     at eigen3/Eigen/src/Core/products/GeneralBlockPanelKernel.h:1236
#1  0x00212664 in Eigen::internal::general_matrix_matrix_product<int, double, 0, false, double, 0, false, 0>::run (rows=3, cols=<optimized out>,
     depth=<optimized out>, _lhs=<optimized out>, lhsStride=3, _rhs=0x1cda940,
     rhsStride=106, res=0x1e39ac0, resStride=3, alpha=1, blocking=..., info=0x0)
     at eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:171
#2  0x0021cacc in Eigen::internal::gemm_functor<double, int, Eigen::internal::general_matrix_matrix_product<int, double, 0, false, double, 0, false, 0>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::internal::gemm_blocking_space<0, double, double, -1, -1, -1, 1, false> >::operator() (info=0x0,
     cols=<optimized out>, col=0, rows=<optimized out>, row=0,
     this=<optimized out>)
     at eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:221
#3  Eigen::internal::parallelize_gemm<true, Eigen::internal::gemm_functor<double, int, Eigen::internal::general_matrix_matrix_product<int, double, 0, false, dou---Type <return> to continue, or q <return> to quit---ble, 0, false, 0>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::internal::gemm_blocking_space<0, double, double, -1, -1, -1, 1, false> >, int> (
     transpose=false, cols=<optimized out>, rows=<optimized out>, func=...)
     at eigen3/Eigen/src/Core/products/Parallelizer.h:96
#4  Eigen::GeneralProduct<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 5>::scaleAndAddTo<Eigen::Matrix<double, -1, -1, 0, -1, -1> > (alpha=<optimized out>, dst=..., this=0xb6e546a4)
     at eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:421
#5  Eigen::ProductBase<Eigen::GeneralProduct<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 5>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1> >::scaleAndAddTo<Eigen::Matrix<double, -1, -1, 0, -1, -1> > (alpha=<optimized out>, dst=...,
     this=0xb6e546a4)
     at eigen3/Eigen/src/Core/ProductBase.h:118
#6  Eigen::ProductBase<Eigen::GeneralProduct<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 5>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1> >::evalTo<Eigen::Matrix<double, -1, -1, 0, -1, -1> > (dst=..., this=0xb6e546a4)
     at eigen3/---Type <return> to continue, or q <return> to quit---Eigen/src/Core/ProductBase.h:109
#7  Eigen::ProductBase<Eigen::GeneralProduct<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 5>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1> >::operator Eigen::Matrix<double, -1, -1, 0, -1, -1> const& (this=0xb6e546a4)
     at eigen3/Eigen/src/Core/ProductBase.h:127
#8  Eigen::ProductBase<Eigen::GeneralProduct<Eigen::GeneralProduct<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 5>, Eigen::Transpose<Eigen::Matrix<double, -1, -1, 0, -1, -1> const>, 5>, Eigen::GeneralProduct<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 5>, Eigen::Transpose<Eigen::Matrix<double, -1, -1, 0, -1, -1> const> >::ProductBase (a_rhs=..., a_lhs=..., this=<optimized out>)
     at eigen3/Eigen/src/Core/ProductBase.h:98
#9  Eigen::GeneralProduct<Eigen::GeneralProduct<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 5>, Eigen::Transpose<Eigen::Matrix<double, -1, -1, 0, -1, -1> const>, 5>::GeneralProduct (rhs=...,
     lhs=..., this=<optimized out>)
     at eigen3/Eigen/src/Core/products/GeneralMatrixMatrix.h:391
#10 Eigen::MatrixBase<Eigen::GeneralProduct<Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, 5> >::operator*<Eigen::Transpose---Type <return> to continue, or q <return> to quit---<Eigen::Matrix<double, -1, -1, 0, -1, -1> const> > (other=..., this=0xb6e546a4)
     at eigen3/Eigen/src/Core/GeneralProduct.h:595
-----------------------------------------------------------------------------------------------------------------------------------------------------------------


and I find this is called by a matrix multiplication like:
" MatrixXd cov2= H.transpose() * cov * H;"
when I comment out this line and the segment fault will gone.
also, if I use opencv's matrix multiplication here and no segment fault will happen, so I think it's a bug of eigen.


the rows of H may be 6~200, and the cols of H is 3. I write a simple code to test but no segment fault happen, here's my simple code:
"
   MatrixXd H(2*num, 3);
   MatrixXd cov = MatrixXd::Zero(2*num, 2*num);
   H = MatrixXd::Random(2*num, 3);
   cov = MatrixXd::Random(2*num, 2*num);
   cout<<"H"<<endl<<H<<endl;
   cout<<"cov"<<endl<<cov<<endl;
   MatrixXd cov2= H.transpose() * cov * H;
   cout<<cov2<<endl;
"


I have tried to use gdb to debug, but no more info I can get.
Could you please have a look or tell me how to debug this problem? thanks!


some info about my platform:

uname -a
Linux  3.4.39 #375 SMP PREEMPT Mon Jul 17 09:07:31 UTC 2017 armv7l GNU/Linux


free
               total        used        free      shared  buff/cache   available
Mem:         524288       21000      270188        1372      233100      491107
Swap:             0           0           0


df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                    5.0M    528.0K      4.5M  10% /
/dev/root                 5.8M      5.8M         0 100% /rom
tmpfs                   250.9M      1.3M    249.5M   1% /tmp
/dev/mtdblock5            5.0M    528.0K      4.5M  10% /overlay
overlayfs:/overlay        5.0M    528.0K      4.5M  10% /
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/mmcblk0p1            7.4G      3.1G      4.3G  42% /mnt/SDCARD
/dev/by-name/UDISK        1.1M    196.0K    892.0K  18% /mnt/UDISK
/dev/by-name/boot         2.0M      1.7M    322.0K  84% /boot
/dev/by-name/boot-res
                           8.0M     39.0K      7.9M   0% /boot-res


Thanks again!
--

Best Regards!
--
extr15










--
 Dipl. Inf., Dipl. Math. Christoph Hertzberg

 Universität Bremen
 FB 3 - Mathematik und Informatik
 AG Robotik
 Robert-Hooke-Straße 1
 28359 Bremen, Germany

 Zentrale: +49 421 178 45-6611

 Besuchsadresse der Nebengeschäftsstelle:
 Robert-Hooke-Straße 5
 28359 Bremen, Germany

 Tel.:    +49 421 178 45-4021
 Empfang: +49 421 178 45-6600
 Fax:     +49 421 178 45-4150
 E-Mail:  chtz@xxxxxxxxxxxxxxxxxxxxxxxx

 Weitere Informationen: http://www.informatik.uni-bremen.de/robotik



Mail converted by MHonArc 2.6.19+ http://listengine.tuxfamily.org/