CoEPP GitLab

• GitLab is an opensource tool which provides a web interface similar to GitHub to manage git repositories.
• Similarly to GitHub, it provides (see GitLab Features for the Community Edition for details):
  1. Activity Stream.
  2. File Browser.
  3. Git Powered Wiki.
  4. Code review.
  5. Issue Management.
  6. Code snippets.
  7. Web Hooks.

• The web interface is accessible through CoEPP GitLab URL.

• GitLab is integrated with CoEPP central authentication system, meaning that CoEPP users can transparently start to use the tool.

• Non CoEPP users can also have access to the tool but their account has to be explicitly created by the GitLab administrator. If an external CoEPP user needs access to the tool, please send a request to rc@coepp.org.au with the following information:
  1. First and last names.
  2. Institution.
  3. Preferred login name.
  4. Institutional email.

• After successful login, users are redirected to the GitLab dashboard.

• The Dashboard consists on a main menu (highlighted in blue) and of a secondary menu (highlighted in red). The secondary menu options change according to the main menu option.

• The user should start by customizing his settings, which can be done through the Profile Settings option of the main menu.

• The secondary menu options are:
  1. Profile: Allows the user to insert a general description about himself and details about his/her social networking accounts (skype, twitter, linkedin, …).
  2. Account: Allows the user to change his username (do not change it unless you know what you are doing). It also allows the user to change his GITLAB token which can be used for atom feeds or the API.
  3. Emails: Allows the user to add secondary email addresses.
  4. Notifications: Allows the user to define different notification levels triggered by different project activities. The following options are available:
     • Disabled (the user will not received email alerts).
     • Participating (the user will receive email alerts on his commits or issues assigned to him).
     • Watch (the user will receive all notifications from projects in which he participates.
  5. SSH Keys: Allow the user to manage his ssh pub keys.
  6. Design: Customize the layout of GitLab.
  7. Group: List the groups the user belongs to.
  8. History: Access the latest activity history.

• To seamlessly perform commits and checkouts from private projects, we strongly suggest that users generate a ssh key pair, and add their ssh public key to their profile under the SSH Keys option.

• Use the New Project button on the main menu, and provide the following information:
  1. Provide a Project name
  2. Import from an existing repository
  3. Provide a brief Project description
  4. Define the visibility for the project:
     • Private (Project access must be granted explicitly for each user)
     • Internal (Project can be cloned by any logged in user)
     • Public (Project can be cloned without authentication).

• Once the user hits the Create Project button, GitLab provides details of what exact commands have to be executed to initialize the git repository (in the user's Linux box)

• For the example showed above, the flow of commands would be:

$ git config --global user.name "Goncalo Borges"

$ git config --global user.email "goncalo@XXXXX"

$ mkdir myfirstproject; cd myfirstproject

$ git init
Initialized empty Git repository in /suphys/goncalo/GITLABTUTORIAL/myfirstproject/.git/

$ echo "This is my first project in GITLAB" > README.md

$ git add README.md

$ git commit -m "first commit"
[master (root-commit) 3e33a5f] first commit
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 README.md

$ git remote add origin git@gitlab.coepp.org.au:goncalo/myfirstproject.git

$ git push -u origin master
The authenticity of host 'gitlab.coepp.org.au (144.6.225.162)' can't be established.
RSA key fingerprint is 18:7a:15:cd:00:c2:50:c8:f9:a7:9f:07:97:ba:b8:38.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'gitlab.coepp.org.au,144.6.225.162' (RSA) to the list of known hosts.
Counting objects: 3, done.
Writing objects: 100% (3/3), 254 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To git@gitlab.coepp.org.au:goncalo/myfirstproject.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin.

• Once the project git repository has been initialized, the user can proceed with the normal git activities, such as new commits.

$ git pull
Already up-to-date.

$ vi prime.py

$ git add prime.py

$ git commit -m "second commit"
[master 8a96740] second commit
 1 files changed, 20 insertions(+), 0 deletions(-)
 create mode 100644 prime.py

$ git push
Counting objects: 4, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 553 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To git@gitlab.coepp.org.au:goncalo/myfirstproject.git
   3e33a5f..8a96740  master -> master

• Once a project is created, the user can explore its settings and functionalities.

• To access to specific features of a given project:
  1. Hit the Dashboard button in the main menu (the top-left fox “gitlab” icon).
  2. Select Projects in the secondary menu.
  3. Select the project you want to explore.

• Once a project is selected, the user may explore the following options:
  1. Project: Entry point for the project. Allows to access to the activity log and to the basic project description provided in the README.md file. The SSH and HTTP endpoints for the project repository are also displayed.
  2. Files: Allows to browse through the project files.
  3. Commits: Allows to browse through the project commits.
  4. Network: Provides a graphical view of the repository topology with branches and commits.
  5. Graphs: Graphical view of the number of commits to the project repository.
  6. Issues: Interface to create labels, issues and milestones associated with the project.
  7. Merge Requests: Manage merge requests in a similar way has done in “Pull Requests” in GitHub.
  8. Wiki: Capability to create Wiki pages associated to the project.
  9. Snippets: Capability to create extracts of source code associated to the project.
  10. Settings: Project settings.

• The Issues, Merge Requests, Wiki and Snippets functionalities can be turned on / off in the project settings.

• Project settings allow the Project owner to access to a set of management configurations. There are 5 categories of management configurations:
  1. Project.
  2. Members.
  3. Deploy Keys.
  4. Web Hooks.
  5. Services.
  6. Protected Branches.

• A detailed explanation of the most significant options follows.

• Allows project owner to change Project Visibility, Default Branch or which GitLab functionalities (Issues, Merge Requests, Wiki and Snippets) should be enabled for the project.

• Also allows to archive the project (freeze of the project since it becomes accessible in Read-Only mode).

• Allows the Project owner to add / remove users to the Project.

• Users can be added with different roles: Guest, Reporter, Developer and Master. The definition of the permissions attributed to each role are defined here: GitLab User/Role permissions.

• Allows the owner to define the execution of actions or scripts, accessible via a predefined URL, and triggered by specific GitLab events such as Push, Tah push, Merge Requests and Issues.

• Allow the owner to keep stable branches secure and force developers to use Merge Requests. Protected branches are designed to:
  1. prevent pushes from everybody except masters.
  2. prevent anyone from force pushing to the branch.
  3. prevent anyone from deleting the branch.

• By default, the master branch is protected. If you want other user (different from the owner or master) to be able to modify the master branch, you will have to unprotect it.

• GitLab groups allow you to group projects into directories and give users to several projects at once.

• When you create a new project in GitLab, the default namespace for the project is the personal namespace associated with your GitLab user. When you transfer or create a project within a group, the name space changes. Group project URLs are prefixed with the group namespace.

$ git clone git@gitlab.coepp.org.au:my-first-group/myfirstproject.git

• The advantage of groups is that you can automatically set user roles / permissions to all projects belonging to the same group. If you need a more fine grained permission setting as a function of project, you should tune it in the project configuration. Therefore, members of a group may only view projects they have permission to access.

• Existing projects may be moved into a group.

• For more information on Groups, check GitLab Project Groups.

• GitLab provides a browsable placeholder for several git projects.
• However, it may well happen that users already have their data is other kind of repositories, like SVN.
• It is then necessary to migrate the data from a SVN repo to a git project.
• The instructions on how to migrate a single SVN repository to a git project will follow.

• In this exercise, we are assuming that we want to migrate a SVN repo like file:import/sydpp1/svnsydpp/atlas/CommonArea/SUSY/SUSY_ToolsSel to a git project under a user namespace.

• Start by creating the SUSY_ToolsSel git project in GitLab under your user namespace.
  • This represents the remote git repo where your 'new' SUSY_ToolsSel git project will be pushed to.
  • For details on how to create a project in GitLab, please see the Project Creation section at CoEPP GitLab

• Login to sui.coepp.org.au machine.

• Run the following command to create a 'authors-transform.txt' file from svn logs since git needs authors to have a valid name and email.

 
$  svn log -q file:////import/sydpp1/svnsydpp/atlas/CommonArea/SUSY/SUSY_ToolsSel | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > authors-transform.txt

• Once the 'authors-transform.txt' file is created, you will need to edit it to insert the correct emails.

# Original file
$ cat authors-transform.txt
saavedra = saavedra <saavedra>
swasnik = swasnik <swasnik>

# Changed file
$ cat authors-transform.txt
saavedra = saavedra <saavedra@physics.usyd.edu.au>
swasnik = swasnik <swasnik@physics.usyd.edu.au>

• Clone the Subversion repository using git-svn
  • This will do the standard git-svn transformation (using the authors-transform.txt file you previously created)
  • The git repository will be placed in the '~/temp' folder inside your home directory.
  • The following instruction is different depending of you have a well defined trunk directory in your SVN repo or not (in some cases, we have found that users do not created a proper SVN trunk structure).

 
# For a well defined trunk
$ git svn clone file:////import/sydpp1/svnsydpp/atlas/CommonArea/SUSY/SUSY_ToolsSel --no-metadata -A authors-transform.txt --stdlayout ~/temp

or

# If you do not have a trunk definition in your SVN repo 
$ git svn clone file:////import/sydpp1/svnsydpp/atlas/CommonArea/SUSY --no-metadata -A authors-transform.txt --trunk=SUSY_ToolsSel ~/temp

• Create a bare git repo, Push the newly created git project to a bare git repo and make its default branch match svn’s “trunk” branch name.

$ git init --bare ~/SUSY_ToolsSel.git
$ cd ~/SUSY_ToolsSel.git
$ git symbolic-ref HEAD refs/heads/trunk

• Push the temp repository to the new bare repository. At the end, you can safely delete the ~/temp directory

$ cd ~/temp
$ git remote add bare ~/SUSY_ToolsSel.git
$ git config remote.bare.push 'refs/remotes/*:refs/heads/*'
$ git push bare
$ cd ..
$ rm -Rf temp

• Rename “trunk” branch to “master” in the bare repo.

$ cd ~/SUSY_ToolsSel.git
$ git branch -m trunk master

• git-svn makes all of Subversions tags into very-short branches in Git of the form “tags/name”. You’ll want to convert all those branches into actual Git tags using:

$ git for-each-ref --format='%(refname)' refs/heads/tags | cut -d / -f 4 | while read ref; do   git tag "$ref" "refs/heads/tags/$ref";   git branch -D "tags/$ref"; done

• Try to clone your recently created bare git repo

$ cd ..; git clone ~/SUSY_ToolsSel.git SUSY_ToolsSel_clone
$ ll SUSY_ToolsSel_clone
total 8
drwxr-xr-x  2 goncalo people 4096 Apr 21 01:28 grid
drwxr-xr-x  2 goncalo people   75 Apr 21 01:28 localbatch
drwxr-xr-x 10 goncalo people 4096 Apr 21 01:28 python

• Push your bare git repo to the remote GitLab project.
  • You will need to remove references to the original remote project (with 'git remote rm origin') and add a reference to the remote project you created in GitLab

$ cd SUSY_ToolsSel_clone
$ git remote rm origin
$ git remote add origin git@gitlab.coepp.org.au:goncalo/SUSY_ToolsSel.git
$ git push -u origin master
$ git push --all
$ git push --tags

• The project should now be available in GitLabL
  • You should be able to browse files via https://gitlab.coepp.org.au/goncalo/SUSY_ToolsSel/tree/master
  • You should be able to clone it via git clone git@gitlab.coepp.org.au:goncalo/SUSY_ToolsSel.git SUSY_ToolsSel