Re: [eigen] 3.0.4 coming soon, testing appreciated

[Gael Guennebaud <gael.guennebaud@xxxxxxxxx>]

Hi Rhys,

it seems that you compiled the tests without optimizations as they
should not take more than 1s. You can check by running cake again and
look at the following lines:

-- ************************************************************
-- ***    Eigen's unit tests configuration summary          ***
-- ************************************************************
-- 
-- Build type:        Release
....

Also comparing the running times of different runs does not make sense
because the involved matrix sizes are different for each run (unless
you specify a unique seed, e.g.: ./product_trmm_1 s0).

gael

On Wed, Nov 23, 2011 at 9:43 PM, Rhys Ulerich <rhys.ulerich@xxxxxxxxx> wrote:
>>>> I see
>>>>   icpc 12.0.4 20110427
>>>>   icpc 12.1.0 20111011
>>>> both build all the 'make check' targets.  I noticed some terribly long
>>>> individual test times for those two builds...
>
>>> Can you profile this?
>>> Is Eigen 3.0.3 performing better?
>
>> I'm building the outer product of {Intel 11.1 latest,
>> Intel 12.1 latest} and {3.0.3 release, 3.0 tip} ...
>> to get better test case timing info.
>
> Here's gross timing information from a release build with
> -fno-omit-frame-pointer executing 'EIGEN_REPEAT=100 ./check.sh
> product_trmm_.' on a quiet x86_64 Linux system.  Notice the repeat
> count of 100.  I picked those tests as they popped up as slow when I
> initially ran make check on a login node earlier in this discussion.
>
> Versions are icpc (ICC) 11.1 20100806 and icpc (ICC) 12.1.0 20111011.
>
> Test project /g/g10/ulerich1/Build/build-3.0.3-intel-11.1
> 1/4 Test #215: product_trmm_1 ...................   Passed   14..28 sec
> 2/4 Test #216: product_trmm_2 ...................   Passed   17..99 sec
> 3/4 Test #217: product_trmm_3 ...................   Passed   75..14 sec
> 4/4 Test #218: product_trmm_4 ...................   Passed   36..13 sec
>
> Test project /g/g10/ulerich1/Build/build-3.0.4-intel-11.1
> 1/4 Test #215: product_trmm_1 ...................   Passed   19..55 sec
> 2/4 Test #216: product_trmm_2 ...................   Passed   24..53 sec
> 3/4 Test #217: product_trmm_3 ...................   Passed   87..97 sec
> 4/4 Test #218: product_trmm_4 ...................   Passed   32..09 sec
>
> Test project /g/g10/ulerich1/Build/build-3.0.3-intel-12.1
> 1/4 Test #215: product_trmm_1 ...................   Passed   10..07 sec
> 2/4 Test #216: product_trmm_2 ...................   Passed   16..13 sec
> 3/4 Test #217: product_trmm_3 ...................   Passed   79..36 sec
> 4/4 Test #218: product_trmm_4 ...................   Passed   35..73 sec
>
> Test project /g/g10/ulerich1/Build/build-3.0.4-intel-12.1
> 1/4 Test #215: product_trmm_1 ...................   Passed   12..57 sec
> 2/4 Test #216: product_trmm_2 ...................   Passed   19..36 sec
> 3/4 Test #217: product_trmm_3 ...................   Passed   86..88 sec
> 4/4 Test #218: product_trmm_4 ...................   Passed   27..58 sec
>
> Good news: Across the board, Intel 12.1 does better than Intel 11.1 on
> 3.0 tip (what I called 3.0.4 above).
>
> Bad news: On all of product_trmm_{1,2,3} the performance is worse on
> 3.0 tip than it was on 3.0.3 on both Intel 11.1 and 12.1.  In
> particular, compare product_trmm_{1,2} on Intel 11.1.
>
> I can do some more detailed profiling on product_trmm_{1,2,3} using
> Intel 11.1.  Any preference for with which one I start?
>
> - Rhys
>
>
>