Re: [eigen] Re: tests taking forever when run with -j8 and OpenMP is en

In the world of scientific computing, your code is usually assumed to be the only process on the system, so this is normally a fairly safe assumption to make. In fact, the OpenMP library has an API which allows you to ask how many cores are on the physical CPU you are running on, which usually allows you to leverage the entire machine. I think for 99% of the users you encounter who are using Eigen and care about this sort of thing, they will want you to utilize the memory hierarchy as if you have the whole machine cache (L1-L3) at your disposal.

On Sat, Jul 3, 2010 at 6:47 PM, Benoit Jacob <jacob.benoit.1@xxxxxxxxx> wrote:

Ah, just one thing. It remains that I only get this phenomenon when
using OpenMP. Is it really the case that with OpenMP enabled we rely
on the availability of more cache memory? That would be a bug.

Benoit

2010/7/3 Benoit Jacob <jacob..benoit.1@xxxxxxxxx>:

> No, I'm not reporting about a bug here!
>
> But I just stumbled across something interesting. With OpenMP enabled,
> when I run
>
> ctest -j8
>
> that is, 8 tests run simultaneously, the tests using large matrix
> product take above of 2 minutes each to complete.
>
> When I run only 1 test at a time (which is the default for ctest),
> they complete very quickly.
>
> Of course the reason is that running 8 tests at a time on my 4-core
> Core i7 is a recipe for making them fight for the CPU's cache memory.
>
> This is an illustration that setting cache sizes is ultimately
> something that only the user can do; the defaults we're using can only
> rely on rather unsafe assumptions about competing processes.
>
> Benoit
>