Re: [eigen] Improved dashboard submissions...

[Hauke Heibel <hauke.heibel@xxxxxxxxxxxxxx>]

It is not really working over here.

Typing "nmake experimental" does not build the tests but it just tries
to run them. The same is true for the visual studio generators - it
does not build the test and just tries to run them.

For nmake the following works (assuming we had just a single test 'meta')

nmake meta
nmake experimental

Using ctest does not help either - again, the tests are not being build.

I spammed the dashboard a bit, just for testing purposes.

http://eigen.tuxfamily.org/CDash/index.php?project=Eigen&date=

.... and today's dashboard does again show what I am talking about with
the submissions from machine "I0009939-X64".

- Hauke

On Tue, Feb 15, 2011 at 11:40 AM, Gael Guennebaud
<gael.guennebaud@xxxxxxxxx> wrote:
> Hi,
>
> I've just committed a change allowing ctest to build the unit tests
> even though they are not in the "all" target. In clear I enforced
> ctest to build the "buildtests" target. The workaround is very tricky,
> but here that works.
>
> gael
>
> On Tue, Feb 15, 2011 at 9:28 AM, Eamon Nerbonne <eamon@xxxxxxxxxxxx> wrote:
>> As a bit of anecdotal evidence; when I built the tests (just once), I
>> avoided running and submitting tests because it wasn't clear exactly how to
>> do so nor whether that was welcome.  A batch file somewhere may work well;
>> but it's not immediately obvious whether it's maintained, whether you need
>> an account on some dashboard, whether people are supposed to run the tests
>> and store the results and submit that file or the script does that for you,
>> etc.
>>
>> Integrating it into the cmake/make targets makes it clearer that the target
>> should Just Work.
>>
>> --eamon@xxxxxxxxxxxx - Tel#:+31-6-15142163
>>
>>
>> On Tue, Feb 15, 2011 at 07:59, Hauke Heibel <hauke.heibel@xxxxxxxxxxxxxx>
>> wrote:
>>>
>>> I want to clarify again while trying to be short.
>>>
>>> Dashboard submissions should be possible with
>>>
>>>  "make experimental"
>>>
>>> on all systems (or "make nightly", etc.). The reasons are numerous. I
>>> bet, we will get many more users who submit do the dashboard. This
>>> means faster detection of regressions. It could lead to a better
>>> coverage of more systems. It would prevent "empty" dashboard
>>> submissions... and I could continue to fill the list.
>>>
>>> The only thing standing in the way of this feature is that we exclude
>>> the testing targets from "all" to prevent accidentally building all
>>> tests. I still do not quite get, why we are doing this. You will never
>>> have to wait for ever but at most until you hit "ctrl break" or
>>> whatever shell escape sequence works on your system to abort such a
>>> build.
>>>
>>> In case you prefer to discuss this on the bug tracker as a feature I
>>> can add a request.
>>>
>>> Finally, I want to say, that I would love to see this feature now
>>> since it will for sure improve our 3.0 release while being easy to
>>> integrate - we can hardly break anything.
>>>
>>> - Hauke
>>>
>>> On Sun, Feb 13, 2011 at 12:53 PM, Hauke Heibel
>>> <hauke.heibel@xxxxxxxxxxxxxx> wrote:
>>> > On Sun, Feb 13, 2011 at 12:44 PM, Jitse Niesen <jitse@xxxxxxxxxxxxxxxxx>
>>> > wrote:
>>> >> On Sun, 13 Feb 2011, Hauke Heibel wrote:
>>> >> I don't know ctest/cdash that well. From your comments, I get the
>>> >> impression
>>> >> that in the Windows IDE you get automatically an Experimental (or
>>> >> something
>>> >> like that) target which submits the tests to the dashboard. Any idea
>>> >> how it
>>> >> is supposed to work with Unix make?
>>> >
>>> > If I am not totally wrong you would simply "make Experimental" and
>>> > cmake would build and submit the tests. No need to configure or touch
>>> > anything extra besides running CMake once.
>>> >
>>> >> That has been there from before my time, but isn't it because otherwise
>>> >> all
>>> >> the test programs are built when you do 'make' or 'make install' and
>>> >> that
>>> >> takes a very long time?
>>> >
>>> > Right, there was something like that. I just don't get why you would
>>> > ever type "make" if you don't intend to build everything? That was
>>> > probably just a precaution.
>>> >
>>> > Maybe we could agree upon allowing this type of convenient dashboard
>>> > submissions on Windows by default.
>>> > (EIGEN_LEAVE_TEST_IN_ALL_TARGET==true for win32 systems).
>>> >
>>> > - Hauke
>>> >
>>> > p.s. I attached a new patch which also works with nmake. Benoit, by
>>> > the way, our scripts do properly propagate the SSE flags for nmake.
>>> >
>>>
>>>
>>
>>
>
>
>