Re: [eigen] Improved dashboard submissions...

[Hauke Heibel <hauke.heibel@xxxxxxxxxxxxxx>]

We had a short chat on IRC. It seems to work - I will do some more
testing when I find the time.

On Tue, Feb 15, 2011 at 1:03 PM, Hauke Heibel
<hauke.heibel@xxxxxxxxxxxxxx> wrote:
> It is not really working over here.
>
> Typing "nmake experimental" does not build the tests but it just tries
> to run them. The same is true for the visual studio generators - it
> does not build the test and just tries to run them.
>
> For nmake the following works (assuming we had just a single test 'meta')
>
> nmake meta
> nmake experimental
>
> Using ctest does not help either - again, the tests are not being build.
>
> I spammed the dashboard a bit, just for testing purposes.
>
> http://eigen.tuxfamily.org/CDash/index.php?project=Eigen&date=
>
> ... and today's dashboard does again show what I am talking about with
> the submissions from machine "I0009939-X64".
>
> - Hauke
>
> On Tue, Feb 15, 2011 at 11:40 AM, Gael Guennebaud
> <gael.guennebaud@xxxxxxxxx> wrote:
>> Hi,
>>
>> I've just committed a change allowing ctest to build the unit tests
>> even though they are not in the "all" target. In clear I enforced
>> ctest to build the "buildtests" target. The workaround is very tricky,
>> but here that works.
>>
>> gael
>>
>> On Tue, Feb 15, 2011 at 9:28 AM, Eamon Nerbonne <eamon@xxxxxxxxxxxx> wrote:
>>> As a bit of anecdotal evidence; when I built the tests (just once), I
>>> avoided running and submitting tests because it wasn't clear exactly how to
>>> do so nor whether that was welcome.  A batch file somewhere may work well;
>>> but it's not immediately obvious whether it's maintained, whether you need
>>> an account on some dashboard, whether people are supposed to run the tests
>>> and store the results and submit that file or the script does that for you,
>>> etc.
>>>
>>> Integrating it into the cmake/make targets makes it clearer that the target
>>> should Just Work.
>>>
>>> --eamon@xxxxxxxxxxxx - Tel#:+31-6-15142163
>>>
>>>
>>> On Tue, Feb 15, 2011 at 07:59, Hauke Heibel <hauke.heibel@xxxxxxxxxxxxxx>
>>> wrote:
>>>>
>>>> I want to clarify again while trying to be short.
>>>>
>>>> Dashboard submissions should be possible with
>>>>
>>>>  "make experimental"
>>>>
>>>> on all systems (or "make nightly", etc.). The reasons are numerous. I
>>>> bet, we will get many more users who submit do the dashboard. This
>>>> means faster detection of regressions. It could lead to a better
>>>> coverage of more systems. It would prevent "empty" dashboard
>>>> submissions... and I could continue to fill the list.
>>>>
>>>> The only thing standing in the way of this feature is that we exclude
>>>> the testing targets from "all" to prevent accidentally building all
>>>> tests. I still do not quite get, why we are doing this. You will never
>>>> have to wait for ever but at most until you hit "ctrl break" or
>>>> whatever shell escape sequence works on your system to abort such a
>>>> build.
>>>>
>>>> In case you prefer to discuss this on the bug tracker as a feature I
>>>> can add a request.
>>>>
>>>> Finally, I want to say, that I would love to see this feature now
>>>> since it will for sure improve our 3.0 release while being easy to
>>>> integrate - we can hardly break anything.
>>>>
>>>> - Hauke
>>>>
>>>> On Sun, Feb 13, 2011 at 12:53 PM, Hauke Heibel
>>>> <hauke.heibel@xxxxxxxxxxxxxx> wrote:
>>>> > On Sun, Feb 13, 2011 at 12:44 PM, Jitse Niesen <jitse@xxxxxxxxxxxxxxxxxx>
>>>> > wrote:
>>>> >> On Sun, 13 Feb 2011, Hauke Heibel wrote:
>>>> >> I don't know ctest/cdash that well. From your comments, I get the
>>>> >> impression
>>>> >> that in the Windows IDE you get automatically an Experimental (or
>>>> >> something
>>>> >> like that) target which submits the tests to the dashboard. Any idea
>>>> >> how it
>>>> >> is supposed to work with Unix make?
>>>> >
>>>> > If I am not totally wrong you would simply "make Experimental" and
>>>> > cmake would build and submit the tests. No need to configure or touch
>>>> > anything extra besides running CMake once.
>>>> >
>>>> >> That has been there from before my time, but isn't it because otherwise
>>>> >> all
>>>> >> the test programs are built when you do 'make' or 'make install' and
>>>> >> that
>>>> >> takes a very long time?
>>>> >
>>>> > Right, there was something like that. I just don't get why you would
>>>> > ever type "make" if you don't intend to build everything? That was
>>>> > probably just a precaution.
>>>> >
>>>> > Maybe we could agree upon allowing this type of convenient dashboard
>>>> > submissions on Windows by default.
>>>> > (EIGEN_LEAVE_TEST_IN_ALL_TARGET==true for win32 systems).
>>>> >
>>>> > - Hauke
>>>> >
>>>> > p.s. I attached a new patch which also works with nmake. Benoit, by
>>>> > the way, our scripts do properly propagate the SSE flags for nmake.
>>>> >
>>>>
>>>>
>>>
>>>
>>
>>
>>
>