Re: [eigen] Bitbucket migration

[Christoph Hertzberg <chtz@xxxxxxxxxxxxxxxxxxxxxxxx>]

Very nice indeed! Thanks Gael and Joseph!

I guess the main decision left is to decide on a schedule when to 
officially switch to gitlab and if/how long we will keep the bitbucket 
repository as a mirror (or essentially as a snapshot).
Once bitbucket closes hg-support (or sooner), we can replace our current 
main-repo by an otherwise empty git repository which just links to the 
new repository (or make a git-mirror, but I don't see any benefits here).
Also, we should probably host the "last official" state of the mercurial 
repository somewhere (as read-only).

Christoph


On 18/09/2019 00.30, Gael Guennebaud wrote:
> Thanks to Joseph's work, and after fighting with Gitlab.com's super
> aggressive spam filter, I finally managed to get the following Gitlab.com
> project as a demo of what could be the outcome of a full migration:
>
> https://gitlab.com/ggael/eigen-migration6
>
> This project includes:
>   - a git repo with bug ids, commit hashes, and pull-request links updated
> in commit message.
>   - a migration of all our bugzilla entries and comments with similar
> updates as the commit messages.
>
> Some examples:
> - attachments, links to commits and bugs:
> https://gitlab.com/ggael/eigen-migration6/issues/1305
> - link to PR:
> https://gitlab.com/ggael/eigen-migration6/issues/1306#note_218167101
> - migration of our "3.x" bug entries as milestones:
> https://gitlab.com/ggael/eigen-migration6/-/milestones/6
> - link to PR from commits:
> https://gitlab.com/ggael/eigen-migration6/commit/97f1e1d89ff759186f4a9c866f7ea3afa7b4c325
> - link to bug from commits:
> https://gitlab.com/ggael/eigen-migration6/commit/c06e6fd115d747c42a2b2ea029c53bbdf41276d6
>
> The bugzilla to gitlab migration script is there:
> https://gitlab.com/ggael/bztogl (adapted from
> https://gitlab.gnome.org/World/bztogl)
>
> PS1: If you notice many extra newlines in issue descriptions/comments,
> that's normal, I already fixed this shortcoming.
>
> What do you think ?
>
> gael
>
>
>
> On Wed, Sep 11, 2019 at 6:03 PM Gael Guennebaud <gael.guennebaud@xxxxxxxxx>
> wrote:
>
> > To prepare the migration from bitbucket, I started to play a bit with its
> > API to see what could be done. So far I've quickly draft two (ugly) python
> > scripts to archive the forks and pull-requests. Since this is a one shot
> > for us, I did not cared about robustness, safety, generality, beauty, etc.
> >
> > You can see them there :
> > https://gitlab.com/ggael/bitbucket-migration-tools and contribute!
> >
> > ** Forks **
> >
> > You can see the summary of the fork script there:
> > http://manao.inria.fr/eigen_tmp/archive_forks_log.html
> >
> > The hg clones (history+checkout) represents 20GB, maybe 12GB if we remove
> > the checkouts. Among the 460 forks, 214 seems to have no change at all
> > (according to "hg out") and could be dropped. I don't know yet where to
> > host them though.
> >
> > This script can be ran incrementally.
> >
> >
> > ** Pull-Requests **
> >
> > You can find the output of the pull-requests script there:
> > http://manao.inria.fr/eigen_tmp/pullrequests/
> >
> > There is a short summary, and then for each PR a static .html file plus
> > diff/patch files, and other details. For instance, see:
> > http://manao.inria.fr/eigen_tmp/pullrequests/OPEN/686/pr686.html
> >
> > Currently this script cannot be ran incrementally. You have to run it just
> > before closing the respective repository!
> >
> > Also, this script does not grab inline comments. Only the main discussions
> > is archived. Those can be obtained by iterating over the "activity" pages,
> > but I don't think that's worth the effort because they would be difficult
> > to exploit anyway.
> >
> >
> > ** hg to git **
> >
> > As discussed in the other thread, if we switch from hg to git, then all
> > hashes will have to be updated. Generating a map file is easy, and thus
> > updating the links/hashes in bug comments and PR comments should not be too
> > difficult (we only have to figure out the right regex to catch all
> > variants).
> >
> > However, updating the hashes within the commit messages will require to
> > rewrite the whole history in a careful order. Does anyone here feels brave
> > enough to write such a script? If not, I guess we could live with an online
> > php script doing the hash conversion on demand. I don't think we'll have to
> > follow such hashes so frequently.
> >
> > cheers,
> > gael

--
 Dr.-Ing. Christoph Hertzberg

 Besuchsadresse der Nebengeschäftsstelle:
 DFKI GmbH
 Robotics Innovation Center
 Robert-Hooke-Straße 5
 28359 Bremen, Germany

 Postadresse der Hauptgeschäftsstelle Standort Bremen:
 DFKI GmbH
 Robotics Innovation Center
 Robert-Hooke-Straße 1
 28359 Bremen, Germany

 Tel.:     +49 421 178 45-4021
 Zentrale: +49 421 178 45-0
 E-Mail:   christoph.hertzberg@xxxxxxx

 Weitere Informationen: http://www.dfki.de/robotik
  -------------------------------------------------------------
  Deutsches Forschungszentrum für Künstliche Intelligenz GmbH
  Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany

  Geschäftsführung:
  Prof. Dr. Jana Koehler (Vorsitzende)
  Dr. Walter Olthoff

  Vorsitzender des Aufsichtsrats:
  Prof. Dr. h.c. Hans A. Aukes
  Amtsgericht Kaiserslautern, HRB 2313
  -------------------------------------------------------------