Sort directories by number of files contained

Here is a one-liner which sorts all the directories under ‘/’ by the number of files contained:

find / -type d -exec sh -c 'ls -a1 "$1" | wc -l' \"{}\" {} \; -print | \
sed  '$!N;s/\n/ /' | sort -gr

I recently needed this on a linux server where the inode quota was exceeded. Through the one-liner I discovered where most of the inodes where consumed.

Tabs in vim

You can open several files in tabs from the command line with vim, just type:

vim -p file1 file2 file3

And each file specified on the command line will be opened in a new tab.

To open a file in a new tab inside vim use :tabe file

To close a tab do :tabc

You can quickly switch between tabs typing gt or gT in normal mode. To switch to a particular tab prepend the tab number to gt, i.e. type #gt, where #gt is the tab number.

Avoid the need to escape parenthesis, brackets… in vim regexes

Vim has a so called “very magic” mode for regexes which allows you to use parenthesis, brackets, the alternative separator (i.e. ‘|’), pluses, etc. with their special meaning but without the need to escape those characters.

(see :help /\v)

Example:

Let’s say you have the following in your buffer:

12345aaa678
12345bbb678
12345aac678

If you execute

:%s/\d\{5\}\(\D\+\)\d\{3\}/\1/

you will get

aaa
bbb
aac

but it required a lot of backslash escaping in the regex. You can avoid the need to escape parenthesis, curly braces, pluses, etc. using vim’s “very magic” mode for regexes. The following would do exactly the same as the previous substitution command but with fewer escaping required:

:%s/\v\d{5}(\D+)\d{3}/\1/

How to view realmedia streams in ubuntu hardy


$ sudo wget http://www.medibuntu.org/sources.list.d/hardy.list -O \
/etc/apt/sources.list.d/medibuntu.list

$ sudo aptitude update

$ sudo aptitude install medibuntu-keyring

$ sudo aptitude install w32codecs

$ mplayer -playlist http://example.com/video.rm

Where http://example.com/video.rm is the url to a realmedia stream.

Make a comit the new root of its branch

This is a quick note on how to make a commit the new root commit of its branch, i.e. chop a branch off of it’s previous history at a specific point.

Imagine we have the following commits:

root -- o -- o -- o -- o -- newroot -- A -- B -- C -- D <-- branch

And what we want to do is drop everything before newroot so that we get the following:

                            newroot' -- A' -- B' -- C' -- D' <-- branch

You can accomplish this doing the following:

Create the file .git/info/grafts [*] and insert just one line with the SHA1 of the commit you want to be the new root of your branch. In the example we would put the SHA1 of newroot into .git/info/grafts.

If you now do a git log or gitk you will see that those commands will display newroot as the root of your branch. But nothing will have actually changed in your repository. You can delete .git/info/grafts and the output of git log or gitk will be as before. To actually create a new root you will have to run git-filter-branch, with no arguments:

$ git filter-branch

git-filter-branch will use the .git/info/grafts file to actually apply the change to the repository.

What git-filter-branch will do is create a new commit object with no parents and whose contents will be the same as the commit whose SHA1 you put into .git/info/grafts. As the root commit will have changed, in consequence, new commits will also be created for all commits following the commit the new root commit was created from. Using the example, git filter-branch would create newroot’ from newroot and it will also create A’, B’, C’ and D’ from A, B, C and D.

WARNING: Running git-filter-branch will rewrite history, which can be a problem.

[*] From the gitrepository-layout manual page:

info/grafts

This file records fake commit ancestry information, to pretend the set of parents a commit has is different from how the commit was actually created. One record per line describes a commit and its fake parents by listing their 40-byte hexadecimal object names separated by a space and terminated by a newline.

Synchronize git repositories between deskop and laptop

I sometimes work on my laptop and sometimes on my desktop computer. This is a little demonstration showing how I synchronize the repository on the laptop with the repository on the desktop.

Let’s say I originally created my git repository on the laptop:

david@laptop$ mkdir ~/repo
david@laptop$ cd !$
david@laptop$ git init
david@laptop$ echo "hello world" > hello.txt
david@laptop$ git add hello.txt
david@laptop$ git commit -m 'Added hello world'

To continue to work on my desktop I’ll just clone the repository on my desktop:

david@desktop$ git clone ssh://192.168.0.42/home/david/repo
david@desktop$ cd repo

Let’s say I do some commits on the desktop now:

david@desktop$ for i in $(seq 5); do echo $i >> hello.txt; \
git commit -am "added $i"; done

Now I can simply push them back to the laptop:

david@desktop$ git push

Let’s go back to the laptop now:

david@laptop$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#	modified:   hello.txt
#

We notice it say’s the file has been modified. The reason is that the previous push from the laptop didn’t update the working directory, as a diff shows:

david@laptop$ git diff --cached
diff --git a/hello.txt b/hello.txt
index 1c8a100..3b18e51 100644
--- a/hello.txt
+++ b/hello.txt
@@ -1,6 +1 @@
 hello world
-1
-2
-3
-4
-5

So we’ll just checkout the version that has been pushed previously:

david@laptop$ git checkout -f HEAD

Let’s do some work on the laptop:

david@laptop$ git checkout -b dev
david@laptop$ echo "this is a new file" > new.txt
david@laptop$ git add new.txt
david@laptop$ git commit -m 'Added new file'
david@laptop$ git checkout master
david@laptop$ echo "6" >> hello.txt
david@laptop$ git commit -am 'Added 6'

Now to push those changes to the desktop we have to add the desktop as a remote.

david@laptop$ git remote add desktop ssh://192.168.0.12/home/david/repo
david@laptop$ git push desktop

This will push the latest changes in the master branch. To push the dev branch:

david@laptop$ git push desktop dev

Now let’s rebase dev onto master:

david@laptop$ git rebase master dev

With the rebase we went from:

hello -- added 1 -- added 3 -- added 4 -- added 5 -- added 6   <-- master
                                              \
                                              added new file  <-- dev

To:

hello -- added 1 -- added 3 -- added 4 -- added 5 -- added 6   <-- master
                                                         \
                                                         added new file  <-- dev

If we now try to push from the laptop to the desktop it will fail:

david@laptop$ git push desktop dev
david@192.168.0.12's password: 
To ssh://192.168.0.12/home/david/repo
 ! [rejected]        dev -> dev (non-fast forward)
error: failed to push some refs to 'ssh://192.168.0.12/home/david/repo'

The reason is that it is not a fast forward push, so what we have to do to push anyway is force the push:

david@laptop$ git push desktop dev -f

In summary, if you have done some changes which result in a non-fast forward push you can push those change anyway using the ‘-f’ switch.

WARNING: Non-fast forward pushes are disabled by default to prevent you from losing changes that are on the remote repository and that are not in your local repository. If you do accidentally lose some changes on the remote end, you can log on to the remote machine, use git reflog to find the commit that was the HEAD before the forced push was executed and establish that commit as the HEAD again using git reset –hard.

Now let’s see what happens if you try to do a non-fast forward pull. First we’ll have to do some changes that will result into a non-fast forward merge. For instance, let’s squash some commits using interactive rebase:

david@laptop$ git checkout master
david@laptop$ git log --pretty=oneline
3003e24db3933cdcf0b337efee105bb8d5fab162 Added 6
addeee530032c74354b119fd0afd5370ebcec744 added 5
94c3719981b10553b3dba4d34c56c4c383769bee added 4
8ed90f1eb2bafc66a238b2fff3a85e579f782b3c added 3
c499db8d72aab61eb275d38df72bcaddb3a5e263 added 2
5fd3b26ffcdd611e6f58e5ec4a7ed8ce34ffe546 added 1
0efc99fff3e258961c321b529930bc56c5d91c39 Added hello world
david@laptop$ git rebase -i master~6

Your editor will be opened with the following contents:

pick 5fd3b26 added 1
pick c499db8 added 2
pick 8ed90f1 added 3
pick 94c3719 added 4
pick addeee5 added 5
pick 3003e24 Added 6

# Rebase 0efc99f..3003e24 onto 0efc99f
#
# Commands:
#  pick = use commit
#  edit = use commit, but stop for amending
#  squash = use commit, but meld into previous commit
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#

Now let’s squash some commits:

pick 5fd3b26 added 1
pick c499db8 added 2
pick 8ed90f1 added 3
squash 94c3719 added 4
squash addeee5 added 5
squash 3003e24 Added 6

# Rebase 0efc99f..3003e24 onto 0efc99f
#
# Commands:
#  pick = use commit
#  edit = use commit, but stop for amending
#  squash = use commit, but meld into previous commit
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.

Save and quit and your editor will open again for a new commit message. Edit the new commit message, save and quit. Now the log should look similar to this:

david@laptop$ git log --pretty=oneline
857596e692c7fd98d019dae2630dc320f5c95177 added 3 added 4 added 5 Added 6
c499db8d72aab61eb275d38df72bcaddb3a5e263 added 2
5fd3b26ffcdd611e6f58e5ec4a7ed8ce34ffe546 added 1
0efc99fff3e258961c321b529930bc56c5d91c39 Added hello world

Now let’s go back to the desktop and see what happens if we do a pull now:

david@desktop$ git co -f master
david@desktop$ git pull
david@desktop$ git log --pretty=oneline
c4b80bae8d6a86373fac45e16c89608a6009e8fc Merge branch 'master' of ssh://192.168.0.42/home/david/repo
857596e692c7fd98d019dae2630dc320f5c95177 added 3 added 4 added 5 Added 6
3003e24db3933cdcf0b337efee105bb8d5fab162 Added 6
c499db8d72aab61eb275d38df72bcaddb3a5e263 added 2
addeee530032c74354b119fd0afd5370ebcec744 added 5
5fd3b26ffcdd611e6f58e5ec4a7ed8ce34ffe546 added 1
94c3719981b10553b3dba4d34c56c4c383769bee added 4
8ed90f1eb2bafc66a238b2fff3a85e579f782b3c added 3
0efc99fff3e258961c321b529930bc56c5d91c39 Added hello world

As you can see the pull doesn’t rewrite the history, i.e. the log looks different now on the laptop and on the desktop. I find that confusing, I’d prefer having the same commit history on all my machines. Let’s undo that last pull:

david@desktop$ git reset --hard HEAD^
david@desktop$ git log --pretty=oneline
3003e24db3933cdcf0b337efee105bb8d5fab162 Added 6
addeee530032c74354b119fd0afd5370ebcec744 added 5
94c3719981b10553b3dba4d34c56c4c383769bee added 4
8ed90f1eb2bafc66a238b2fff3a85e579f782b3c added 3
c499db8d72aab61eb275d38df72bcaddb3a5e263 added 2
5fd3b26ffcdd611e6f58e5ec4a7ed8ce34ffe546 added 1
0efc99fff3e258961c321b529930bc56c5d91c39 Added hello world

What I’d like to do is something equivalent to a push –force but with a pull. The solution is using the plus sign with pull:

WARNING: The same caution as to a push –force applies, you can loose changes that are on your local repository but not on the remote repository when using a pull with the plus sign. Although, like in the previous warning, you can probably use git reflog and git reset –hard to get those changes back.

david@desktop$ git pull origin +master:master

The first master references the remote master branch and the second references the local master branch. If we now look at the commit history we can see that it is the same as on the laptop:

david@desktop$ git log --pretty=oneline
857596e692c7fd98d019dae2630dc320f5c95177 added 3 added 4 added 5 Added 6
c499db8d72aab61eb275d38df72bcaddb3a5e263 added 2
5fd3b26ffcdd611e6f58e5ec4a7ed8ce34ffe546 added 1
0efc99fff3e258961c321b529930bc56c5d91c39 Added hello world

In summary, the equivalent pull operation to a “git push –force” is a pull using the plus sign.

Finally one more WARNING: do not force push to repositories from which other people have already pulled as that may cause problems for them. See problems with rewriting history in the git manual.