Re: [eigen] Dropping C++14 compatibility discussion

[Rob McDonald <rob.a.mcdonald@xxxxxxxxx>]

I don't have access to the machine, so I can't answer the detailed questions.

My application isn't super intensive -- but it can be used to generate input files for much more expensive computations and optimization studies.  Consequently, it sometimes gets run on big iron.

My user who recently had problems because my app developed a dependency on std::regexp -- which isn't really supported by gcc 4.8.  The problem is less the compiler and more the standard libraries installed on the machine.  He is not building on the big machine -- he builds on a local vm on a laptop and then transfers the binary up.  They also prefer to do their local setup on images that are as similar as possible to the computation environment.  Perhaps they could set up local compilers and libraries, but that is a much bigger hassle than using system installed standard libraries.

The machine is not at TACC.  It has nearly 250k cores and 1PB memory.  It was installed when RHEL 7 was brand new.  I have no idea how many cores the users are using -- a small fraction no doubt.

Rob

On Tue, May 5, 2020 at 8:01 PM Jeff Hammond <jeff.science@xxxxxxxxx> wrote:

Is the primary compiler on said supercomputer really /usr/bin/g++? How many nodes are users running on when they build apps with this compiler?

My guess is most of the users of that system use mpicxx or something similar from modules that provides a trivial solution to this problem. If you’re referring to TACC, I’m sure they’ll recommend a bunch of compilers besides GCC-4.8.

RHEL has devtoolset for a reason. All major HPC systems have modules for a dozen or more compilers that don’t live in /usr. Eigen shouldn’t hold itself back for people who refuse to help themselves.

Jeff

On Tue, May 5, 2020 at 3:21 PM Rob McDonald <rob.a.mcdonald@xxxxxxxxx> wrote:

I certainly understand the need to move forward.  However, conservatism certainly has value.  Particularly when you advertise yourself as a production library (not a research or proof of concept toy).

Just as an example, the main national supercomputer for a major US government research agency is running RHEL 7.  That means gcc 4.8.x and the standard libraries that go with it.  That means C++11 and most of the stuff that goes with it -- but not everything.

RHEL 8 was only released 364 days ago (depending on your time zone).  It is certainly not unreasonable for installed high value production computing assets like the above machine to still be running RHEL 7.  While it would be great if RHEL moved compilers forward more quickly -- that isn't how it works.

The fact is -- Eigen development is slow.  The code has reached a very stable state.  The last major release was in 2018.  These facts are probably good for most users of Eigen.

If permission to jump to some advanced version means someone is going to go through and wholesale clean up the code and move it forward -- who is that someone?  Where is this development energy coming from?  Users making a request doesn't mean there is anyone to get the job done.

Eigen 3.x should remain pretty much the way it is.  I hope/assume that any big jump like this would start by creating an Eigen 4.x branch/fork on GitHub and moving forward from there -- hopefully the energetic people calling for this change are the ones who will step forward to do the work.

Rob

On Tue, May 5, 2020 at 3:01 PM Janek Kozicki (yade) <jkozicki-yade@xxxxxxxxx> wrote:

I suggest to move forward, and use C++17. There is no reason to
stick with old libraries.

If you prefer to keep for a year or two the ability to compile with
C++14, then place such obsolete code inside some #ifdef #else #endif,
so that more efficient code is generated with C++17 and less efficient
code when someone insists on compiling with C++14.

In yade[1] we have moved to C++17 when building packages for ubuntu
focal 20.04 or debian buster or bullseye. The C++14 is used only when
building packages for older systems such as ubuntu xenial 16.04.

Yade can compile with eigen using C++17 without any problems. Which
is good. For example the coinor library had some problems with C++17:
and older version worked only with C++17.

best regards
Janek

[1] https://yade-dem.org/doc/ and https://gitlab.com/yade-dev/trunk



Rasmus Munk Larsen said:     (by the date of Tue, 5 May 2020 14:14:19 -0700)

> I agree with you, but it seems the maintainers are very busy with other
> projects, and I am guessing they do not have time for major initiatives to
> clean up Eigen. We have dropped support for c++03 in the Tensor library,
> but a lot of things could be cleaned up if we more fully embraced c++11 or
> c++14. At this point, I only see this happening slowly, or if somebody from
> the open source community is willing to help.
>
> On Tue, May 5, 2020 at 2:05 PM Patrik Huber <patrikhuber@xxxxxxxxx> wrote:
>
> > Hi all,
> >
> > I'm a quite sad to see this post got no replies. I can only add that I
> > fully agree and think along the very same lines. I would love to see Eigen
> > move forward, move to C++14 in the 3.5 release, and drop the old cruft.
> > It's 2020 now, 6 years after C++14.
> >
> > Best wishes,
> > Patrik
> >
> > On Wed, 8 Apr 2020 at 17:20, Martin Beeger <martin.beeger@xxxxxxxxx>
> > wrote:
> >
> >> Hallo!
> >>
> >> This is continuation of the discussion from last year about
> >> compatibility, but I wanted to bring it up again, because I have some
> >> new data from my experience.
> >>
> >> Our company uses Eigen extensively and has a larger ecosystem built on
> >> it, as do many of the voices on this list. We are in the embedded world,
> >> so we are often hindered to adapt quickly to new tools, very much like
> >> the HPC community has been.
> >> But we managed to move to C++14 with our codebase in 2019. When doing
> >> that, we somewhat monitored performance and compile time during the
> >> adaption.
> >>
> >> When we starting compiling our C++98 codebase as is with C+14,
> >> performance already went up slightly. C++14 does silently move or elide
> >> copies, and modern compiler have better optimizers, and the quality of
> >> implementation of  STL types got better. So if you are at all bound by
> >> performance of STL types, I would really recommend using a new compiler
> >> in C++14 mode even with your old code.
> >>
> >> This was the obvious part. Another part was less obvious. We profiled
> >> compile time, which didn't change much after the switch and started to
> >> look into stuff which was expensive to compile. And then selectively in
> >> places, where the compile time was bad we simplified the code using
> >> C++14 features to improve compile time. This allowed us with very
> >> localized changes to cut our compile time by almost in half, while, at
> >> the same time, making the code in question often a lot simpler and new
> >> features (e.g. for performance improvements) easier to implement.
> >>
> >> This is also the experience which was observed with other template heavy
> >> codebases like boost::mpl in comparison to hana and other stuff. The
> >> gains in simplicity and compile time, especially from constexpr and
> >> lambda features are not minor. They can often cut your code in half and
> >> more than double the compile time.
> >>
> >> The problem is, I am at a point where its hard to do much more to
> >> improve the compile time of my codebase significantly, because most of
> >> the compile time is brought in by 2 libraries: boost.test and Eigen. We
> >> will most likely at some point abandon boost.test due to this, like many
> >> others have already (which slowed the development and improvement of
> >> boost.test further, while the alternatives got better and getting into
> >> this downwards spiral). I would be vary happy if I am not forced to
> >> abandon Eigen after the great 10 years we had with this library. In
> >> order to avoid this I pulled a lot of tricks like explicit template
> >> instatiations, tricks to reduce includes, even pimpl-like encapsulation
> >> at performance cost to isolate from the problem, but that gets you only
> >> so far.
> >>
> >> I may well be that my use case is special, but I strongly assume that
> >> new users which today want to adopt Eigen and have to look into its
> >> internals (as you inevitable need to do at some points), will see how
> >> its written and quickly run for alternatives. This amount of macros,
> >> boilerplate and similar stuff will be an argument against this great
> >> library some day and this day may already have come.
> >>
> >> The important part is here: Eigen compile time and internal _expression_
> >> template engine code readability it was great by 2010s standards, it was
> >> ok by 2015 standards, it is borderline by 2020 standards, and unless
> >> something changes, it will be unacceptable by 2025 standards, unless the
> >> Eigen library moves along.
> >>
> >> As C++ users, we do care about backwards compatibility greatly and that
> >> is even good for me, but we should not go the C way and care about too
> >> ancient compilers. The C++ comittee doesn't (that why int is required to
> >> be 2s complement in C++20), so Eigen library maintainers IMHO should
> >> follow.
> >>
> >> What are the chances to get Eigen 3.4 out of the door with C++98 support
> >> and drop it on the devel branch afterwards and jump to C++14?
> >> What is the Eigen promises about how old yours compilers may be?
> >> Can we explicitly agree on a statement like: We vow to support up to 3
> >> year old compilers (or 5 years)?
> >>
> >> If we could agree on clear and conservative rules like we will go 3
> >> years back or 5 years back and state these on the Eigen front page,
> >> Eigen user may look at our codebase and be much more willing to accept
> >> older standards code, knowing that it will improve over time and that
> >> the user gets some useful guarantees about the future in return.
> >> There is agrument to be made that if you use a 5+ years old compiler,
> >> you really do not care about performance, something Eigen uses generally
> >> care about. So you are not in a targeted user group of Eigen. Little HPC
> >> clusters do not at all offer any way to install a more recent compilers
> >> that 5 years (this has changed a lot from a decade ago) and even in the
> >> embedded world, vendors tend to drop support or upgrade for platforms
> >> with more than 5 years old toolchains too (this has also very much
> >> changed from 10 years ago).
> >>
> >> Eigen users should be able to get a clear answer on the question when we
> >> drop C++98 (of if). That belongs on the front page IMHO.
> >>
> >> Kind regards,
> >> Martin
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>


--
--
Janek Kozicki, PhD. DSc. Arch. Assoc. Prof.
Gdańsk University of Technology
Faculty of Applied Physics and Mathematics
Department of Theoretical Physics and Quantum Information
--
http://yade-dem.org/
http://pg.edu.pl/jkozicki (click English flag on top right)

--

Jeff Hammond
jeff.science@xxxxxxxxx
http://jeffhammond.github.io/