Re: [eigen] Bitbucket is dropping its Mercurial support!

Hi all,

I'll add a few points to consider, from the viewpoint of a very infrequent contributor, and both github and gitlab user...

- hg is a significant hurdle when coming from the git world. I spent quite a bit of time trying to come up with git equivalent to things in order to submit a few minor patches that would have been super easy for me using just git (and github or gitlab). I guess a majority of developers now use git (>90% according to atlassian), and would have the same hurdles to contribution.

- I have a number of projects on both github and gitlab, I find both to be pretty good and fairly intuitive. I *feel* better about gitlab, but still use github for its mac and windows builds.

- From the open source perspective, gitlab is of course preferable and prevents github / microsoft lock-in, and allows a realistic path to self-hosting, and other "freedom" (define as you like.)

- Github is on very fast infrastructure, and is noticeably snappier than gitlab, but the slight lagginess of gitlab isn't (imho) bad, and I think not a significant factor in deciding between them (and it's nowhere near as slow as bitbucket). One imagines furthermore that gitlab will improve.

- Gitlab has integrated CI, whereas github has circleci and travisci (etc). Gitlab's CI is great, but the freely available runners are somewhat more limited than what's available with circle/trav (gitlab lacking free macos & windows, last time I checked - but maybe they'd donate a paid plan? https://about.gitlab.com/pricing/index.html#gitlab-com ). Where this situation will be in 1-5 years is unknowable, regardless I'd imagine that the best way to run Eigen continuous integration is on dedicated infrastructure, or scatter/gather to it, since a large part is (or should be) regression tests of compilation speed and algorithmic speed. And running performance tests on VMs is too noisy.

Cheers

Michael

On Sat, Aug 24, 2019 at 12:30 PM David Tellenbach <david.tellenbach@xxxxxxxxxxxxx> wrote:

Hi all,

just some thoughts about some points you've made:

b) Fixing internal links inside commit messages ("grafted from ...", "fixes error introduced in commit ...")
Maybe I've forgot something crucial but doing something like

for branch in $(hg branches | awk '{print $1}'); do
hg update -C $branch > /dev/null
echo "$branch $(hg log -v | egrep "bitbucket.org" | wc -l)"
done

gives me

Branch Links
------ ------
default 9
3.3 9
3.2 9
find-module-imported-target 9
fix_cuda_clang 9
CUDA_9.0 9
patch-1 9
3.1 9
3.0 9
2.0 9

If we also consider closed branches (what we probably should) I also get 9 links (using hg branches -c). A short manual look at the links shows that they are all the same 9 (remains to be checked!). That said this is something we can easily fix manually.

Another point are links inside the codebase that point to bitbucket.

Following the same logic as above I use

hg grep "bitbucket.org"

and get 11 links (all seem to be the same). Again something fixable manually.

c) Fixing external links to the repository. Most notably, any links from our bugtracker will eventually fail (even if we stayed with bitbucket, the hashes won't match). I doubt that we could set up any automatic forwarding for that.
This might be by far the most complicated point since a lot (the majority?) of all issues contain links to commits. If desired I can find a concrete number but I doubt that it will be very...motivating. I also doubt that Bitbucket will provide any functionality to redirect links to other Git providers but I could image that there could be some workaround if we decide to migrate to Bitbucket Git. Something we should keep in mind before choosing a new provider.

Any third-party which relies on our main repository will need to change as well (not directly "our" problem, but we need to give a reasonable amount of time for everyone to migrate to whatever will be our future official repository).
It's currently unclear for me what exactly will happen with the hg repo but I guess it will be archived or something similar.. In this case we can link to the new repo on the README page. I don't have any further ideas regarding this but also think we should migrate somewhat fast.

I see essentially three options:
1. Migrate to another mercurial provider
2. Convert to git, stay at bitbucket
3. Convert to git, migrate to another provider

1. We could migrate to Tuxfamily and keep mercurial. As you said this would imply we have to handle pull requests separately which is possible. As you surly know LLVM does exactly that by using Phabricator. However this would fix some of the issues above but links to bitbucket would remain a problem. Another downside of mercurial is that only very few projects are using it and contributing would be much easier in the case of git.

2. The only reason I see for this is the one I mentioned above: If there is (or will be) any support to redirect bitbucket links it will most likely only work if we stay at bitbucket. Compared with other code hosting services I find bitbucket (not mercurial) to be really complicated and not intuitive.

3. In an ideal world this would be my absolute preference (not very surprising). Regarding the choice of a service I want to make the personal point that I would rather migrate to Gitlab than to Github because it is as least as good as Github and I think that diversity of tools and providers is crucial for open source. In the long run we could even think about migrating issues to Gitlab and installing test runners (this is another story).

Thanks,
David

On 21. Aug 2019, at 14:53, Christoph Hertzberg <chtz@xxxxxxxxxxxxxxxxxxxxxxxx> wrote:

Hello Eigen users and contributers!

As some may have noticed, bitbucket/atlassian is "sunsetting" its mercurial support:

https://bitbucket.org/blog/sunsetting-mercurial-support-in-bitbucket

If they stick to their timeline, we will have to migrate until June 1st, 2020. That means we still have time, but if we do nothing, things will break ...

Converting the repository itself to git should not be a bigger issue -- and if we do this we could as well migrate to a more mainstream provider (i.e., github).

I think the main problems for migration are:
a) Migrating open pull-requests (for historical reasons, the closed/merged ones should probably be archived as well)
b) Fixing internal links inside commit messages ("grafted from ...", "fixes error introduced in commit ...")
c) Fixing external links to the repository. Most notably, any links from our bugtracker will eventually fail (even if we stayed with bitbucket, the hashes won't match). I doubt that we could set up any automatic forwarding for that.
d) Any third-party which relies on our main repository will need to change as well (not directly "our" problem, but we need to give a reasonable amount of time for everyone to migrate to whatever will be our future official repository).

Smaller issues (relatively easy to fix or not as important):
e) Change links from our wiki (to downloads)
f) Change URLs for automated doxygen generation and for unit-tests
g) Automatic links from the repository to our bugtracker (currently "Bug X" automatically links to http://eigen.tuxfamily.org/bz/show_bug.cgi?id=X)
h) Change hashes in bench/perf_monitoring/changesets.txt

I probably missed a few things ...

I see essentially three options:
1. Migrate to another mercurial provider
2. Convert to git, stay at bitbucket
3. Convert to git, migrate to another provider

Honestly, I see no good reason for option 2. And the only real reason I see for option 1 would be that it safes a lot of hassle with b) and h) -- also perhaps it would simplify c) (e.g., we could easily crawl through our bugzilla-database and just replace some URLs).

Any opinions on this? Preferences for how to proceed, or other alternatives?
Does anyone have experience with migrating from hg to git? Or migrating between providers? Especially, also dealing with the issues listed above.
Does anyone see issues I forgot?

Cheers,
Christoph

--
Dr.-Ing. Christoph Hertzberg

Besuchsadresse der Nebengeschäftsstelle:
DFKI GmbH
Robotics Innovation Center
Robert-Hooke-Straße 5
28359 Bremen, Germany

Postadresse der Hauptgeschäftsstelle Standort Bremen:
DFKI GmbH
Robotics Innovation Center
Robert-Hooke-Straße 1
28359 Bremen, Germany

Tel.: +49 421 178 45-4021
Zentrale: +49 421 178 45-0
E-Mail: christoph.hertzberg@xxxxxxx

Weitere Informationen: http://www.dfki.de/robotik
-------------------------------------------------------------
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH
Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany

Geschäftsführung:
Prof. Dr. Jana Koehler (Vorsitzende)
Dr. Walter Olthoff

Vorsitzender des Aufsichtsrats:
Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
-------------------------------------------------------------