git

As svn is considered well-known, I will explain git compared to that.

First of git only works in repositories, not repositories and working copies like svn. This tutorial has the right consistency for my taste.

proxy

If you are behind a proxy the easiest way to make git use it is :

git config --global http.proxy http://proxyuser:proxypwd@proxy.server.com:8080
#git config --global https.proxy https://proxyuser:proxypwd@proxy.server.com:8080

The second line was commented on purpose, because curl is used under the hood and that does not support https proxies, see below

unsupported protocol scheme

Curl doesn't support HTTPS proxy so you get this message to make you aware of that so that you don't think you're actually are using a HTTPS proxy. Previously curl would just ignore non-supported schemes in the proxy URL and that mislead people into believing things about what they accomplished. Now, curl will tell you when you try to use a proxy method that curl doesn't support. Like HTTPS. If you instead specify http:// to the proxy, you'll see that it works (like before) and you don't any longer rely on curl's silent downgrade.

create a development branch

First take a look at what branches are currently active (-verbose and -all)

> git branch -va
develop                f0c99b7 case sensitive ?!
* master                 f0c99b7 case sensitive ?!
remotes/origin/HEAD    -> origin/master
remotes/origin/develop f0c99b7 case sensitive ?!
remotes/origin/master  f0c99b7 case sensitive ?!

The -v show the id and last comment, the -a adds the remote branches.

To create a develop branch or check it out if it already exists :

cd existing-local-copy
git checkout -b develop

It will switch you to the development branch, but if it is create new it will be local only. To get it to the remote server :

git push -u origin develop

create a feature branch

bitbucket has special directories for features, hotfixes etc that you can best use !

For instance look at https://bitbucket.org/keesklopt/backend/branches/ On this page you see the current remote branches but also a dropdown menu for

• branches : Merged All Active
• Branch Type : Release Hotfix Feature Bugfix

The last ones also show which 'directory' to use and for features it says 'feature/'

So if you want to adhere to that structure, name your feature not 'newbells' but 'feature/newbells' and it fits better in the bitbucket GUI.

Normally you branch new features from develop, so you use a command like this :

git checkout feature/newbells develop

merging back into develop

git checkout develop

Or.. move back to the develop branch to operate from there..

git merge --no-ff feature/newbells

This incorporates the newbells into develop preserving some history. Without --no-ff (no fast forward) will most likely never mention the feature branch just develop changes.

deleting feature branch

git branch -D feature/newbells

troubleshooting

double pull needed

This happens often, git pull fails the first time, but succeeds the seconds. Two reasons i recently detected :

• unable to resolve reference
• "this probably means a git process crashed in this repository earlier"

A possible solution is to cleanup the repository with :

git gc --prune=now # cleanup unnecessary file and optimize
git remote prune origin # Deletes all stale remote-tracking branches

But of course that's hard to test since the second time works anyway. So report here if you see the occurrences go down after doing this.

password problem

If you have problems with git asking for username password every time.

type

git remote -v

And see if the protocol used is git:, if its https: it will keep asking. So switch to git, or just clone it again :

git clone git@bitbucket.org:keesklopt/project

Make sure you have imported your public key to bitbucket site first.

username problem

This basically occurs when you checked out the project like this :

remote -v 
origin  https://bitbucket.org/keesklopt/doc (fetch)
origin  https://bitbucket.org/keesklopt/doc (push)

You should have entered your username@ as well, you can switch in-place with this command :

git remote set-url origin https://keesklopt@bitbucket.org/keesklopt/doc
git pull
Password for 'https://keesklopt@bitbucket.org': 

caching credentials

If you are not on a site that can use git with you id, you will have to type your password, but sometimes git asks for you username as well.

git config --global credential.helper "cache --timeout=3600"

This will tell git to cache your type username and password for 1 hour, if you leave out the timeout option it will default to 15 minutes. During this time you won't have to type username or password.

changing typo's

If you commit with a nasty typo, there is a way of altering the commit message before pushing it :

git commit --amend

tagging the master branch

git tag -a 1.2 git push -u --tags

config

The configuration is accessible through a readable (editable) file in .git, .git/config. It looks somewhat like this :

[core]
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
[remote "origin"]
    url = https://keesklopt@bitbucket.org:/keesklopt/backend
    fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
    remote = origin
    merge = refs/heads/master
[branch "develop"]
    remote = origin
    merge = refs/heads/develop

If you use this command in the same directory:

git config --list

You get :

user.email=kees@klopt.org
user.name=Kees Klop
push.default=simple
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=https://keesklopt@bitbucket.org:/keesklopt/backend
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
branch.develop.remote=origin
branch.develop.merge=refs/heads/develop

Also note the url, it contains keesklopt@ in the url. If you need to type your username each time you check in, it probably has the form with username:

url = https://bitbucket.org:/keesklopt/backend

Note that with the https scheme you always need to provide a password !

Weirdly though, this still can go wrong when you have .com instead of .org

url = https://bitbucket.com:/keesklopt/backend

This DOES work, but asks for you username !!

clean start

I needed this when i added a file with password to the repository and pushed it to gitlab. All guides that show how to remove it again failed. This is possibly because i did not enable forced pushes and misread the error message.

The script used :

FILE=ldapsrv_test.py
git filter-branch -f \
    --prune-empty \
    --tag-name-filter cat \
    --tree-filter 'rm -f $FILE' \
    -- --all
# this gives a warning : 
git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
git reflog expire --expire=now --all && git gc --prune=now --aggressive

But that is besides the point since i also wanted to know how to completely start again because of bad comments etc. Here is the way to restart with the code you have :

-- Remove the history from 
rm -rf .git

-- recreate the repos from the current content only
git init
git add .
git commit -m "Initial commit"

-- push to the github remote repos ensuring you overwrite history
git remote add origin git@github.com:<YOUR ACCOUNT>/<YOUR REPOS>.git
git push -u --force origin master

And here i got much the same error on the push command. So here is how to change the protected branch:

• Choose your project
• Repository -> Branches you see a 'protected' block beside the branch.
• Here is an option "Protected branches can be managed in " project settings"
• Expand Protected branches.
• Un-protect the branch you need

remote example

One particular problem was at SSC-ICT where you do not have access to the repository on production machines.

Here is a picture of the locations and operations :

Lets imagine i changed the version number on both sides.

The number was 0.46 , i changed it on production to 0.48 and to 0.50 on development.

|------------- |------------ |------------ |development |repository |production |0.50 |0.46 |0.48

In the picture it is quit clear what we need to do when merging this change.

We commit and push it on development which make it 0.50 in the repository, and we copy the repository to the shielded repository.

Now it would look more like this within the shielded site :

|------------- |---------------- |------------------------ |socstl repo |socportal repo |socportal working copy |0.50 |0.46 |0.48

So doing a diff on production will say :

+0.48
-0.46

But we would want to know the difference between development :

+0.48
-0.50

So according to the image above a 'git fetch' would do the trick. BUT you still compare against the local repository with git diff, you need to :

git diff origin/master  <path>

Now this reports the changes you have locally against the remote copy. So you can read this as :

compared to the remote branch you have to delete the msg_ok() line and insert the “OK” line to get this local copy.

Example :

git diff origin/master app/mod_scan/controllers.py
diff --git a/app/mod_scan/controllers.py b/app/mod_scan/controllers.py
index df17384..f107d77 100644
--- a/app/mod_scan/controllers.py
+++ b/app/mod_scan/controllers.py
@@ -113,7 +113,7 @@ def scan_scenter_results_download(scanid):
    Args:
       scanid(str): download for this scan id
    """
-   stat = core_util.msg_ok()
+   stat = "Ok"

If that seems a little backward, you can reverse the diff with -R :

git diff -R origin/master <file>
-   stat = "Ok"
+   stat = core_util.msg_ok()

Now it show how to get from your copy to the repository.

To traverse all of the files you either commit the changes without -a because that means all !! Or you choose for the remote version and just checkout that file.

To see which files differ you can use :

git diff --name-only

And whenever you commit a file or checkout a file this list will become shorter until you are done.

reverting

There are several levels :

• If you just want to undo uncommitted changes, just checkout the file.
• If you added a file (not committed) , do git reset
• If you committed, do git revert
• If you want to remove untracked files (e.g., new files, generated files): git clean -f
• Or untracked directories (e.g., new or automatically generated directories): git clean -fd

sub modules

There will always be modules that are better maintained as separate repositories but still needed as a subdirectory under a larger repository. Sub modules do just that.

I will describe the klopt example that uses several submodules. The structure for this projects is :

└── klopt
    ├── 3pty
    ├── backend
    ├── bag
    ├── deploy
    ├── frontend
    └── readme.rst

To arrive at this, you first create the top repository 'klopt' and make a readme.rst there.

mkdir klopt
cd klopt
vim readme.rst
git init 
git add readme.rst
# now add all of the repositories you want as submodules
git submodule add git@bitbucket.org:keesklopt/3pty
git submodule add git@bitbucket.org:keesklopt/backend
git submodule add git@bitbucket.org:keesklopt/frontend
git submodule add git@bitbucket.org:keesklopt/deploy
git submodule add git@bitbucket.org:keesklopt/bag
git commit -a 
# ok here you also should add a new bitbucket repo and :
git remote add origin git@bitbucket.org:keesklopt/klopt.git
git push -u origin master

Each time you add a submodule it checks out the complete tree, and each time you will see that the file .gitmodules gets extended with another section. However if you clone this repository back, it will be empty.

git clone git@bitbucket.org:keesklopt/klopt.git
tree # will be rather empty

You can either check it out recursively or one by one

git submodule init
git submodule update
# or clone it directly like :
git clone --recurse-submodules git@bitbucket.org:keesklopt/klopt.git

It is possible to alter code in the submodules, but i got problems with (detached HEAD). So probably only use this to checkout installations, and work on the separate repositories as normal.

bare repositories

A bare repository does not contain a working tree but only a copy of the repo and is used as a central storage. Normally you get one when creating a repo at a provider like bitbucket. But sometimes you cannot do that for instance when working at customers that do not want sources to be on-line.

You cannot use a normal repository for this because it won't let you push to it. So you need to init with the --bare option and then just only use it for storage, locally or via ssh (tried) or even http or git(not tried).

You can recognize a bare repo:

• It has the data files/dirs in plain view (config,HEAD,objects.. etc)
• It does not have any sources visible

Whereas a normal repo or checkout (working tree) :

• The data files/dirs are hidden under .git/
• It does have a source tree.

You can best start with an empty bare repository because it won't let you add anything anyway :

cd myproject
git init --bare
empty Git repository in ...
git add Readme.txt
fatal: This operation must be run in a work tree.

So you just need to provide a directory and init --bare it and then use it as a remote, even if it's local.

git remote -v
origin  /home/klopk/projects/repo (fetch)
origin  /home/klopk/projects/repo (push)

So even if on the same disk, this still works when push and pulling code.

useful commands

I will start Here a collection of commands for certain recurring tasks.

change url

git remote set-url origin git@192.168.1.204:kees/bhv.git

Before and after :

git remote -v 
origin  git@gitlab.uvt.nl:lis-dev/bhv (fetch)
origin  git@gitlab.uvt.nl:lis-dev/bhv (push)
git remote set-url origin git@192.168.1.204:kees/bhv.git
git remote -v 
origin  git@192.168.1.204:kees/bhv.git (fetch)
origin  git@192.168.1.204:kees/bhv.git (push)

pull overwriting local changes

I generally do this after git reports local changes, and i remove those files. But it can be done much quicker.

git reset --hard
git pull

So the first command can also be seen as : (next chapter)

reset all local changes

git reset --hard

reset a branch to the master

You have worked on a branch for a while and now you need some new features to finish. Normally to be sure I create a new branch, them compare with meld and move all changes to the new branch, remove the old branch and continue.

Surely that must be easier, so here are the steps.

git diff # optional, and fix anything you don't need
git commit -a # commit your changes before merge
git merge origin/master # probably some complaints if you did not commit
git diff(tool) origin/master
git commit -a
git push # if you approved this all

The first diff can just help you cleanup some stuff before merging. Otherwise just commit and do it afterwards.

The last difftool command let's you compare the result and normally you don't have to fix anything. If you forgot something, fix it there.