Re: [eigen] Preventing Memory Allocation for Large GEMM

[Gael Guennebaud <gael.guennebaud@xxxxxxxxx>]

Hi,

to disable multi-threading, you can compile with -DEIGEN_DONT_PARALLELIZE, but this won't be enough to discard all memory allocations because GEMM needs some workspace to restructure the blocks of y and z. If x and y specifies maximal sizes at compile-time (or size at compile time), then the workspace will be allocated on the stack. Otherwise, Eigen's will try to call alloca if available and that the require memory is not too large. In a last resort it will call malloc.

gael

On Tue, Jun 21, 2016 at 11:35 PM, Graham Neubig <neubig@xxxxxxxxxxx> wrote:

Hi All,

I have a question about GEMM on Eigen. We're creating a library where we'd like to prevent additional memory allocation, so we've been testing with EIGEN_NO_MALLOC. However, whether Eigen is allocating memory seems to be inconsistent for GEMM, with no memory allocated for smaller multiplications, but memory allocated for larger multiplications. I'm running something similar to the following command, and when the size gets large enough (50 or 100 or so) it starts dying.

x.noalias() += y.transpose() * z;

Looking at the stack trace included at the end of the message, it seems the Eigen is parallelizing the GEMM, which is allocating memory. Is there any way to prevent this parallelization to ensure that we work within the memory already allocated?

Graham

#5  0x00000001002ae571 in Eigen::internal::aligned_malloc(unsigned long) ()

#6  0x0000000100139fb0 in Eigen::internal::general_matrix_matrix_product<long, float, 1, false, float, 0, false, 0>::run(long, long, long, float const*, long, float const*, long, float*, long, float, Eigen::internal::level3_blocking<float, float>&, Eigen::internal::GemmParallelInfo<long>*) ()

#7  0x0000000100139ae5 in Eigen::internal::gemm_functor<float, long, Eigen::internal::general_matrix_matrix_product<long, float, 1, false, float, 0, false, 0>, Eigen::Transpose<Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > const>, Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >, Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >, Eigen::internal::gemm_blocking_space<0, float, float, -1, -1, -1, 1, false> >::operator()(long, long, long, long, Eigen::internal::GemmParallelInfo<long>*) const ()

#8  0x0000000100139956 in void Eigen::internal::parallelize_gemm<true, Eigen::internal::gemm_functor<float, long, Eigen::internal::general_matrix_matrix_product<long, float, 1, false, float, 0, false, 0>, Eigen::Transpose<Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > const>, Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >, Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >, Eigen::internal::gemm_blocking_space<0, float, float, -1, -1, -1, 1, false> >, long>(Eigen::internal::gemm_functor<float, long, Eigen::internal::general_matrix_matrix_product<long, float, 1, false, float, 0, false, 0>, Eigen::Transpose<Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > const>, Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >, Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >, Eigen::internal::gemm_blocking_space<0, float, float, -1, -1, -1, 1, false> > const&, long, long, bool) ()

#9  0x0000000100137b1d in void Eigen::internal::generic_product_impl<Eigen::Transpose<Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > const>, Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >, Eigen::DenseShape, Eigen::DenseShape, 8>::scaleAndAddTo<Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > >(Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >&, Eigen::Transpose<Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > const> const&, Eigen::Map<Eigen::Matrix<float, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > const&, float const&) ()