Re: [eigen] Windows Testing Issues

[ Thread Index | Date Index | More lists.tuxfamily.org/eigen Archives ]


Hi!

On 12/06/2019 02.59, Rob Conde wrote:
Total Build Time: ~7 hours

[...]

So a great improvement there...although 7 hours is still very long. The eigen docs quote '3 hours', but I'm unsure if that is using multiple cores and/or when that number was captured? This, and the former run, were all done serially.

I think the 3 hours were for a serial build, but that section has not been updated for years, so the number is quite outdated.


However, there are more failures:

   *   [...]  (Exit code 0xc0000409)
   *   [...]  (SEGFAULT)

Some searching tells me that "Exit code 0xc0000409" means STATUS_STACK_BUFFER_OVERRUN, i.e., it looks like Eigen uses too much stack space for some of the tests (this could also explain the segmentation faults). You can either decrease EIGEN_STACK_ALLOCATION_LIMIT (this defaults to (128*1024), but that is per temporary/per matrix, i.e., with some mild recursion this could easily exceed a few MiB). Or you can tell your compiler to increase the stack-size. According to https://docs.microsoft.com/en-us/cpp/build/reference/f-set-stack-size the corresponding option is "/F". I guess on 64bit systems there is nothing wrong with simply setting it to 16MiB or higher (this should just be virtual memory until it is actually used).

You can also run each test manually from the command-line to see more details when it crashes.


I should also note that in this test, and the former VS2017 test, I was using the default CMake settings...so C++11 tests were not enabled/tested. Although, many cxx11_tensor_xxxxx tests DID run...which I find confusing. I need to go back and time on gcc as well for reference.

That is expected behavior. Admittedly, the naming is not 100% accurate here: With C++11 disabled, some of the cxx11_tensor-tests run in "emulated c++11 mode" (mostly these are overloads for functions/classes with 1-5 parameters instead of variadic templates -- and re-implementations of simple classes like std::array).


Cheers,
Christoph



Rob
________________________________
From: Christoph Hertzberg <chtz@xxxxxxxxxxxxxxxxxxxxxxxx>
Sent: Tuesday, June 11, 2019 8:08 AM
To: eigen@xxxxxxxxxxxxxxxxxxx
Subject: Re: [eigen] Windows Testing Issues

Hi,

thanks for investigating this!

These build-times are really far from great, to say it mildly ...

If you have time to further investigate this, you could start removing
parts of each (or some) test until build-times get reasonable (i.e.,
cancel the build if it takes more than a few minutes -- depending on
your patience). If that works reasonably well, we can further split the
current tests into smaller ones.

On the other hand, if you find very small tests which take a lot to
build, this could help the MSVC maintainers to look for issues in their
compiler, or it might help us to find workarounds for this.


Also, if you have time, could you check the current development branch
("default" in hg) and the latest 3.2 branch if there were any large
regressions or progressions in build-time (some tests were newly
introduced or split/renumbered between releases, though)

Cheers,
Christoph

On 11/06/2019 01.58, Rob Conde wrote:
Ok - I ran 3.3.7 through the testing.

    *   cmake ..\eigen_3.3.7 -G "Visual Studio 15 Win64" -Thost=x64
       *   I later realized that VS also uses the 32-bit toolchain by default, but I doubt it has a big effect here
    *   ctest -C Release -D Experimental

The complete build took about 22 hours. I flagged anything that took longer than ten minutes (I calculated build time as the modified time of the test minus the modified time of the previous test).

    *   array_reverse_8 (16 mins)
    *   geo_quaternion_2 (25 mins)
    *   geo_quaternion_4 (30 mins)
    *   mixingtypes_5 (12 mins)
    *   product_small_3 (11 mins)
    *   product_small_4 (23 mins)
    *   sparse_extra_1 (98 mins)
    *   sparse_extra_2 (215 mins)
    *   sparse_extra_3 (105 mins)
    *   sparse_product_1 (108 mins)
    *   sparse_product_2 (256 mins)
    *   sparse_product_3 (104 mins)

Not sure if that list forms a pattern. I will continue investigating when i get a chance.

The actual tests took...12 mins. Only 1 failed.

Rob
________________________________
From: Rob Conde <rob.conde@xxxxxxxxxxxxxxxx>
Sent: Monday, June 10, 2019 9:53 AM
To: eigen@xxxxxxxxxxxxxxxxxxx
Subject: Re: [eigen] Windows Testing Issues

Hey Edward,
     I did try that quickly at one point for a single test but I don't remember seeing a big difference, but I will try to do something more comprehensive. When I get a chance (sigh) I will also see if I can distill a simple example which i can report to the visual studio team. At this point I haven't spent much time trying to find a pattern in the things that take a long time to compile, if there is one, but that would be useful too I think.

Rob
________________________________
From: Edward Lam <edward@xxxxxxxxxx>
Sent: Monday, June 10, 2019 9:03 AM
To: eigen@xxxxxxxxxxxxxxxxxxx
Subject: Re: [eigen] Windows Testing Issues

Hi Rob,

On 6/10/2019 7:57 AM, Rob Conde wrote:
This has been running for about 11 hours now (still on the build part).
Obviously this is a visual studio issue, but I'm not sure when it was
introduced. We're not seeing anything like this in the our usage of
Eigen...although to a smaller degree it might be hard to tease out of the larger
picture.

It might be worthwhile trying out the latest Visual Studio 2019 just to see if
there's any behaviour change.

-Edward




--
   Dr.-Ing. Christoph Hertzberg

   Besuchsadresse der Nebengeschäftsstelle:
   DFKI GmbH
   Robotics Innovation Center
   Robert-Hooke-Straße 5
   28359 Bremen, Germany

   Postadresse der Hauptgeschäftsstelle Standort Bremen:
   DFKI GmbH
   Robotics Innovation Center
   Robert-Hooke-Straße 1
   28359 Bremen, Germany

   Tel.:     +49 421 178 45-4021
   Zentrale: +49 421 178 45-0
   E-Mail:   christoph.hertzberg@xxxxxxx

   Weitere Informationen: http://www.dfki.de/robotik
    -------------------------------------------------------------
    Deutsches Forschungszentrum für Künstliche Intelligenz GmbH
    Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany

    Geschäftsführung:
    Prof. Dr. Jana Koehler (Vorsitzende)
    Dr. Walter Olthoff

    Vorsitzender des Aufsichtsrats:
    Prof. Dr. h.c. Hans A. Aukes
    Amtsgericht Kaiserslautern, HRB 2313
    -------------------------------------------------------------





--
 Dr.-Ing. Christoph Hertzberg

 Besuchsadresse der Nebengeschäftsstelle:
 DFKI GmbH
 Robotics Innovation Center
 Robert-Hooke-Straße 5
 28359 Bremen, Germany

 Postadresse der Hauptgeschäftsstelle Standort Bremen:
 DFKI GmbH
 Robotics Innovation Center
 Robert-Hooke-Straße 1
 28359 Bremen, Germany

 Tel.:     +49 421 178 45-4021
 Zentrale: +49 421 178 45-0
 E-Mail:   christoph.hertzberg@xxxxxxx

 Weitere Informationen: http://www.dfki.de/robotik
  -------------------------------------------------------------
  Deutsches Forschungszentrum für Künstliche Intelligenz GmbH
  Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany

  Geschäftsführung:
  Prof. Dr. Jana Koehler (Vorsitzende)
  Dr. Walter Olthoff

  Vorsitzender des Aufsichtsrats:
  Prof. Dr. h.c. Hans A. Aukes
  Amtsgericht Kaiserslautern, HRB 2313
  -------------------------------------------------------------




Mail converted by MHonArc 2.6.19+ http://listengine.tuxfamily.org/