GIT: Rewriting commit history with git filter-branch

48.1. Using the git filter branch command (filter-branch)

The git filter-branch command allows you to rewrite the Git commit history. This can be done for selected branches and you can apply custom filters on each revision. This creates different hashes for all modified commits. This implies that you get new IDs for all commits based on any rewritten commit.

The command allows you to filter for several values, e.g., the author, the message, etc. 

Using the filter-branch command is dangerous as it changes the Git repository.  It changes the commit IDs and reacting on such a change requires explicit action  from the developer, e.g., trying to rebase the stale local branch onto the  corresponding rewritten remote-tracking branch.

For example, you can use git filter-branch if you want to remove a file which contains a password from the Git history. Or you want to remove huge binary files from the history. To completely remove such files, you need to run the filter-branch command on all branches.

48.2. filter-branch examples

The following command extracts a directory from a Git repository and retains all commits for this subfolder.

git filter-branch --prune-empty --subdirectory-filter FOLDER-NAME  BRANCH-NAME

The following command replaces the email address of one author from all commits.

git filter-branch -f \
--env-filter 'if [ "$GIT_AUTHOR_NAME" = "Lars Vogel" ]; then \
GIT_AUTHOR_EMAIL="lars.vogel@gmail.com"; fi' HEAD)

GIT: Git commit and other hooks

50.1. Usage of Git hooks

Git provides commit hooks, e.g., programs which can be executed at a pre-defined point during the work with the repository. For example, you can ensure that the commit message has a certain format or trigger an action after a push to the server.

These programs are usually scripts and can be written in any language, e.g., as shell scripts or in Perl, Python etc. You can also implement a hook, for example, in C and use the resulting executables. Git calls the scripts based on a naming convention.

50.2. Client and server side commit hooks

Git provides hooks for the client and for the server side. On the server side you can use the pre-receive and post-receive script to check the input or to trigger actions after the commit. The usage of a server commit hook requires that you have access to the server. Hosting providers like GitHub or Bitbucket do not offer this access.

If you create a new Git repository, Git creates example scripts in the .git/hooks directory. The example scripts end with .sample. To activate them make them executable and remove the .sample from the filename.

50.3. Restrictions

Not all Git server implementations support server side commit hooks. For example Gerrit (a Git server which also provides the ability to do code review) does not support hooks in this form. Also Github and Bitbucket do not support server hooks at the time of this writing.

Local hooks in the local repository can be removed by the developer.

GIT: Commit history of a repository or certain files

Gitk can be used to visualize the history of a repository of certain files.

In some cases simply using git blame is not sufficient in order to see all details of certain changes. You can navigate to the file location in the target git repository and use the gitk [filename] command to see all commits of a file in a clear UI.

In this screenshot we can see all commits of the ShowViewHandler.java by using the gitk ShowViewHandler.javacommand:

On Linux you can easily install gitk by using the sudo apt-get install gitk command in a terminal.

GIT: Analyzing the commit history with git log

17.1. Using git log

The git log command shows the history of the Git repository. If no commit reference is specified it starts from the commit referred to by the HEAD pointer.

git log

git log HEAD~10 

git log COMMIT_REF 

	shows the history of commits starting from the HEAD~10 commit
	shows the history of commits starting from the COMMIT_REF commit

17.2. Helpful parameters for git log

The following gives an overview of useful parameters for the git log command.

git log --oneline  
git log --abbrev-commit 
git log --graph --oneline 
git log --decorate 

	--oneline - fits the output of the git log command in one line. --online is a shorthand for "--pretty=oneline --abbrev-commit"
	--abbrev-commit - the log command uses shorter versions of the SHA-1 identifier for a commit object but keeps the SHA-1 unique. This parameter uses 7 characters by default, but you can specify other numbers, e.g., --abbrev-commit --abbrev=4.
	graph - draws a text-based graphical representation of the branches and the merge history of the Git repository.
	decorate - adds symbolic pointers to the log output

17.3. View the change history of a file

To see changes in a file you can use the -p option in the git log command.

git log -- [file_reference] 

git log -p -- [file_reference]  

git log --follow -p -- [file_reference] 

	- shows the list of commits for this file
	- the -p parameter triggers that the diffs of each commit is shown
	- --follow allow include renames in the log output

17.4. Configuring output format

You can use the --pretty parameter to configure the output.

# command must be issued in one line, do not enter the line break
git log --pretty=format:'%Cred%h%Creset %d%Creset %s %Cgreen(%cr)
 %C(bold blue)<%an>%Creset' --abbrev-commit

This command creates the output.

17.5. Filtering based on the commit message via regular expressions

You can filter the output of the git log command to commits whose commit message, or reflog entry, respectively, matches the specified regular expression pattern with the --grep=<pattern> and --grep-reflog=<pattern> option.

For example the following command instructs the log command to list all commits which contain the word "workspace" in their commit message.

git log --oneline --grep="workspace" 

Greps in commit message for "workspace", oneline parameter included for better readability of the output

There is also the --invert-grep=<pattern> option. When this option is used, git log lists the commits that don’t match the specified pattern.

17.6. Filtering the log output based on author or committer

You can use the --author=<pattern> or --committer=<pattern> to filter the log output by author or committer. You do not need to use the full name, if a substring matches, the commit is included in the log output.

The following command lists all commits with an author name containing the word "Vogel".

git log --author="Vogel"

GIT: Commit references

4.1. Predecessor commits, parents and commit references

Each commit has zero or more direct predecessor commits. The first commit has zero parents, merge commits have two or more parents, most commits have one parent.

In Git you frequently want to refer to certain commits. For example, you want to tell Git to show you all changes which were done in the last three commits. Or you want to see the differences introduced between two different branches.

Git allows addressing commits via commit reference for this purpose.

A commit reference can be a simple reference (simple ref), in this case it points directly to a commit. This is the case for a commit hash or a tag. A commit reference can also be symbolic reference (symbolic ref, symref). In this case it points to another reference (either simple or symbolic). For example HEAD is a symbolic ref for a branch, if it points to a branch. HEAD points to the branch pointer and the branch pointer points to a commit.

4.2. Branch references and the HEAD reference

A branch points to a specific commit. You can use the branch name as reference to the corresponding commit. You can also use HEAD to reference the corresponding commit.

4.3. Parent and ancestor commits

You can use ^ (caret) and ~ (tilde) to reference predecessor commit objects from other references. You can also combine the ^ and ~ operators. 

The Git terminology is parent for ^ and ancestor for ~.

4.4. Using caret and tilde for commit references

[reference]~1 describes the first predecessor of the commit object accessed via [reference]. [reference]~2 is the first predecessor of the first predecessor of the [reference] commit. [reference]~3 is the first predecessor of the first predecessor of the first predecessor of the [reference] commit, etc.

[reference]~ is an abbreviation for [reference]~1.

For example, you can use the HEAD~1 or HEAD~ reference to access the first parent of the commit to which the HEADpointer currently points.

[reference]^1 also describes the first predecessor of the commit object accessed via [reference].

For example HEAD^ is the same as HEAD~ and is the same as HEAD~3.

The difference is that [reference]^2 describes the second parent of a commit. A merge commit typically has two predecessors. HEAD^3 means ‘the third parent of a merge’ and in most cases this won’t exist (merges are generally between two commits, though more is possible).

[reference]^ is an abbreviation for [reference]^1.

4.5. Commit ranges with the double dot operator

You can also specify ranges of commits. This is useful for certain Git commands, for example, for seeing the changes between a series of commits.

The double dot operator allows you to select all commits which are reachable from a commit c2 but not from commit c1. The syntax for this is "c1..c2". A commit A is reachable from another commit B if A is a direct or indirect parent of B.

Think of c1..c2 as all commits as of c1 (not including c1) until commit c2.

For example, you can ask Git to show all commits which happened between HEAD and HEAD~4.

git log HEAD~4..HEAD

This also works for branches. To list all commits which are in the "master" branch but not in the "testing" branch, use the following command.

git log testing..master

You can also list all commits which are in the "testing" but not in the "master" branch.

git log master..testing

4.6. Commit ranges with the triple dot operator

The triple dot operator allows you to select all commits which are reachable either from commit c1 or commit c2 but not from both of them.

This is useful to show all commits in two branches which have not yet been combined.

# show all commits which
# can be reached by master or testing
# but not both
git log master...testing




GIT: The details of the commit objects

3.1. Commit object (commit)

Conceptually a commit object (short:commit) represents a version of all files tracked in the repository at the time the commit was created. Commits know their parent(s) and this way capture the version history of the repository.

3.2. Technical details of a commit object

This commit object is addressable via a hash ( SHA-1 checksum ). This hash is calculated based on the content of the files, the content of the directories, the complete history of up to the new commit, the committer, the commit message, and several other factors.

This means that Git is safe, you cannot manipulate a file or the commit message in the Git repository without Git noticing that corresponding hash does not fit anymore to the content.

The commit object points to the individual files in this commit via a tree object. The files are stored in the Git repository as blob objects and might be packed by Git for better performance and more compact storage. Blobs are addressed via their SHA-1 hash.

Packing involves storing changes as deltas, compression and storage of many objects in a single pack file. Pack files are accompanied by one or multiple index files which speedup access to individual objects stored in these packs.

A commit object is depicted in the following picture.

The above picture is simplified. Tree objects point to other tree objects and file blobs. Objects which didn’t change between commits are reused by multiple commits.

3.3. Hash and abbreviated commit hash

A Git commit object is identified by its hash (SHA-1 checksum). SHA-1 produces a 160-bit (20-byte) hash value. A SHA-1 hash value is typically rendered as a hexadecimal number, 40 digits long.

In a typical Git repository you need fewer characters to uniquely identify a commit object. As a minimum you need 4 characters and in a typical Git repository 5 or 6 are sufficient. This short form is called the abbreviated commit hash or abbreviated hash. Sometimes it is also called the shortened SHA-1 or abbreviated SHA-1.

Several commands, e.g., the git log command can be instructed to use the shortened SHA-1 for their output.