Re: [eigen] Bugs on bugzilla

[Benoit Jacob <jacob.benoit.1@xxxxxxxxx>]

Thanks for starting this conversation. It makes me very happy and hopeful for the future of Eigen, as I have been thinking --- with regret --- for a long time about what I/we may have done wrong in growing the Eigen developer community.

Eigen was sort of my first serious FOSS project. I thought of it purely in terms of solving technical problems. I didn't understand the social consequences of making the codebase more complex.

If I could go back in time and change one thing, I would have us care more and earlier about easing outside contributions. Case in point: a long time ago, around the 3.0 release, we considered writing a lot of internal documentation, but then we thought that we should rather fix bug 99 first as it would cause deep changes in these internals. We made a mistake there: we should have stuck to plans of internal documentation back then.

It's not too late, and it makes me very, very happy to see this topic brought up now.

Thanks!

Benoit

2014-05-29 13:30 GMT-04:00 Mark Borgerding <mark@xxxxxxxxxxxxxx>:

Kolja, Jitse, others,  Thanks for tackling this.  My response is bottom posted for flow.

On 05/29/2014 07:07 AM, Kolja Brix wrote:

Dear Jitse,

On 08.04.2014 09:48, Jitse Niesen wrote:

I talked with Christoph on IRC earlier today about the amount of bugs
and other issues on bugzilla. The issue is that the number is growing
all the time; there are almost 300 open at the moment.

That in itself is perhaps not that much a problem though a bit
overwhelming. The real danger is that important bugs slip through.
Especially bad is when users submit a patch and nobody looks at it,
which does happen.

What can we do about this?

Why are there so many bugs / issues? Some issues are feature requests
that no developer is really interested in. Some are bugs on exotic
platforms that no developer has access to. Some bugs need discussion
(e.g., bug 662 on input operators).

But the main problem is a lack of manpower; or to put it another way,
the library is too big for the number of people working on it. This
is compounded by the fact that some parts of the code base (e.g.,
product, I think) are only really understood by one guy (Gael). So
improving the process and the bug tracker will not solve everything.

Christoph suggested that we have an IRC meeting to go through the
bugs and discuss them so to prioritize / downgrade / close /
delegate. Another suggestion is to enable voting on bugzilla to help
 us prioritize the issues.

Any other ideas?

thank you for pointing out the current situation. There are a few more
observations:

In the Eigen source code simple or straight forward algorithms are often
well-documented. But unfortunately, the documentation of much more
advanced and complicated parts of the code (e.g. the template core part)
is very sparse, what might discourage newcomers. Of course, this is a
natural effect, but we should try to go against that. I know that
currently, core parts are rewritten for the evaluators, it would be
great if the concepts of the new code would be easier to understand.

In fact, Gael's presentation slides from Eurographics/CGLibs in Pisa
available on <http://eigen.tuxfamily.org/index.php?title=Publications>
helped me a lot to understand more on the architectural concepts behind
Eigen, but the link to this web page and the PDF file is not so easy to
find.


Therefore, possible strategies to attract more new contributors - and I
am quite sure that many users would like to contribute if the learning
curve was not so steep - could include the following issues:

* An improvement of the documentation in the code would be a great step
forward. In particular the documentation of the essential core parts and
the template mechanisms underlying the library would be very helpful.
Moreover, the concepts behind Eigen and its API need to be documented in
more detail, preferably outside the code.

* A review and a classification of the bugs possibly is also a good
idea. Maybe we can also rate them by how hard it probably is to fix
them [from "easy" (standard C++ knowledge is enough) via "advanced"
(additional knowledge on numerics or template metaprogramming necessary)
to "for experts only" (deep insight into Eigen internal structures
required)]. Then new contributers can get started with easy ones and
grow with the tasks.

* By setting up a kind of mentoring program for bug fixing new
contributors could learn much faster from current experts.

I agree with JItse's characterization that bugzilla has many "feature requests that no developer is really interested in",  and "bugs on exotic platforms".
I view this as a symptom of the difficulty of contributing.  If the learning curve was gentler, the reporting users would be empowered to enhance or fix things themselves.

I echo Kolja's sentiment -- if we reduce the barriers to entry, bug reports on bugzilla will more often come with test cases and patches.

Eigen is wonderfully easy to *use*. I love it! I think I speak for many Eigen users when I say I am so thankful to all the great developers who have worked so hard to make it!  I am proud of my (small) part in helping make it.
... but Eigen internals are quite daunting sometimes (all for good reasons, I'm sure).

Adding to Kolja's ideas: How about a tutorial aimed at Eigen *internal* developers?  e.g. Walk a user through a truly simple, developer task in the span of a half-hour.  Maybe create a algorithm for something that is understandable and easy to code.  Gram-Schmidt or Power Iteration come to mind.  I'd stress that the cmake test environment is not necessary, at least for beginners.

I suspect the average Eigen user (who might become an occasional contributor) is an engineer or scientist, but not a computer scientist or a "language lawyer". He or she is competent in c++, but not comfortable with esoteric language features. 

Musing on the "Drake Equation"-type probabilities -- the intersection of circumstance, knowledge, and skills needed to be a truly effective Eigen contributor (a set in which I am not an element)  is very small indeed.

1. desire to help
2. time to help ( this is my greatest obstacle in the last few years )
3. Linear Algebra skill : high to expert
   
4. C++ skill: expert level ( comfort with metaprogramming, partial specialization, etc)    <-- I suspect this shrinks the most!
   
5. amenability to (possibly new) tools: CMake, mercurial, and doxygen-style commenting.

Multiply the probabilities and you start to see why there are few who rise to the elite rank: Gael, Benoit, Thomas, Desire,  Jitse, Hauke, Chen-Pang, Pavel. (developers who have contributed more than 10,000 lines of code according to hg churn).

-- Mark