How to get file sizes recursively in a directory and list them in ascending order

When cleaning up unused files from your linux computer, I find it useful to know which of my files use most of the space so that I can easily determine which ones to remove.

The most basic command to find file size of a given file is:
du -sm myfile.txt

If you are in a folder with multiple files and want to get file sizes of all the files, the command is:
du -sm *

The above command does not get file sizes recursively. However, if you are happy with it and want to sort the file sizes in ascending order:
du -sm * | sort -n -k 1

Now, this is little useful as it does not do recursive file size listing. To do that:
find . -type f -exec du -sm {} \; -print | sort -n -k 1

If you are looking for a particular type of file (eg. jar files):
find . -name "*.jar" -type f -exec du -sm {} \; -print | sort -n -k 1

Enjoy!

Unix/Linux command to check if Apache Tomcat is running

So I've come across this problem quite a few times. Normal way to do this is:

ps -ef | grep tomcat

This works most of the times. If tomcat is running, it gives between 1 and 2 lines back but if not, it gives anywhere between 0 and 1 lines back. A much cleaner use of the above command would be with wc -l:

ps -ef | grep tomcat | wc -l

However, this doesn't solve the actual problem as along with the tomcat process, it also gives you the process of command "grep tomcat".

Here's the command to solve this problem. You can use either of the two below commands:

ps -ef | grep tomcat | grep -v "grep tomcat" | wc -l

ps -ef | grep tomca[t] | wc -l

The first command explicly says that once you get a list of all processes containing the word tomcat, ignore lines containing words "grep tomcat". And then the usual, pipe it to word count and output the number of lines.

The second one, however, tricks the grep into using a regular expression and ignoring itself. This is because the actual output containing "grep tomca[t]" will have the square brackets which obviously won't match the actual regular expression.

Changing keyboard layout in Ubuntu Server (Linux, How to)

So I recently installed Ubuntu Server in a virtual machine and I had problems with my keyboard layout. I use UK layout while the default that comes with Ubuntu Server is US layout. As you can imagine, this caused problems and I had to go hunt for a solution. Finally, I found one and here it is:

You need to reconfigure keyboard configuration. Type the following:
sudo dpkg-reconfigure keyboard-configuration

If that doesn't work, you might need to install console-data package. Install it by typing:
sudo apt-get install console-data

and then try first step again.

If everything was fine, you should see this:

Using this you can select brand of your keyboard. I am running this virtual machine on my Acer laptop and as you can see it is listed there. You can use up/down/page up/page down keys to navigate through the list. When you have picked one, press enter to go to next screen.

You should now see this:

This is the language selection screen. Navigate through the list, select your language and as you did in previous one, press enter to move on to next screen.

This is the screen where you select your keyboard type:

Assuming you're using a qwerty keyboard for PC, you should just leave the first one selected and press enter. However, if you are running this on a mac or your keyboard is of different kind, you are free to choose whatever applies to you from the list and continue by pressing enter.

The next screen allows you to map a key on your keyboard to alternate grammar key (Alt Gr).

I have never used this key in my life for alternate grammar, however, hoping that I may use it at some point, I would map it to right-alt key (which actually is Alt Gr key). As usual, select one from the list and press enter.

Next one is compose key. Same drill. Pick one and press enter:

Compose key, if I remember correctly, allows you to type ASCII for certain characters when you press and hold it. This key on windows is left alt key however, here you can choose whatever you want for ubuntu server.

When you press enter, the wizard will quit. So that was the keyboard configuration screen which allows you to configure your keyboard pretty neatly. Have fun!

Assigning static IP address to your Raspberry Pi (WiFi)

When I first bought my Raspberry Pi, I had this problem. My router and TV are in different rooms and I don't have a ethernet cable. This restricts networking ability of my Pi which is quite annoying. I searched for articles on assigning static IP addresses to Raspberry Pi but most of them were talking about assigning it for eth0. I, on the other hand wanted wlan0 static IP.

Now, I could have just got a long ethernet cable and solve this whole problem but I was just annoyed and was constantly asking myself "Why isn't this working over wifi". So finally, after significant digging around, I found the solution. Here it goes:

First of all, take a backup of /etc/network/interfaces file:
pi@raspberrypi ~ $ sudo su -
root@raspberrypi:~# cp /etc/network/interfaces .


Now, in order to make this work, we need some information first. We need to find out the new static ip address that you want, gateway, mask, network and broadcast address. Assuming that you have a working wifi connection on your Raspberry Pi, type:
root@raspberrypi:~# ifconfig

From the output, the bit that we are interested in is:
wlan0     Link encap:Ethernet  HWaddr 00:00:00:00:00:00
          inet addr:192.168.1.99  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:19153 errors:0 dropped:24512 overruns:0 frame:0
          TX packets:25553 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3724822 (3.5 MiB)  TX bytes:31437802 (29.9 MiB)

The highlighted line will give you information about your current IP, broadcast and mask address. Take a note of the last two. So, in my case:
address: (this is the IP address you wish to reserve as static eg. 192.168.1.69)
mask: 255.255.255.0
broadcast: 192.168.1.255

To find out information about the last two bits, type:
root@raspberrypi:~# netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG        0 0          0 wlan0
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 wlan0

In my case:
gateway: 192.168.1.1
network (a.k.a destination): 192.168.1.0

Now that we have all the information that we need, edit the /etc/network/interfaces file using nano. Putting all the bits and pieces together, it should look like:

Press Ctrl + O to save.
Now, since I was doing this whole process over SSH, I had to write a script to restart wifi because if I didn't, as soon as I turn it off, it would also take away my SSH access rendering me powerless. You can do the following in either case. Create a new file called restartwifi.sh using nano in root home directory:
root@raspberrypi:~# nano restartwifi.sh

Now, I am aware that this script can be improved but this is what I had before and it has worked without any problems. Press Ctrl + O to save the script. Then press Ctrl + X to exit.

Make sure you edit permissions to make the script executable. Run:
root@raspberrypi:~# chmod +x restartwifi.sh

If you're not root, you'll have to prefix above command with "sudo". 

Now comes the moment of truth. Run the script to reset your wifi by typing following command:

root@raspberrypi:~# nohup ./restartwifi.sh &

If you're in putty session, you'll immediately go offline. If you've done everything correctly, you should be able to ssh into your Pi, after couple of minutes, on your chosen IP address.

Copying symbolic links (UNIX/Linux)

At some point, you might come across a situation where you have some files and a bunch of symlinks (symbolic links) in a folder and you wish to copy the data that is pointed at by the symlinks and not just blank links. If you are using Redhat Enterprise Linux, this should automatically happen when you call the cp command. However, on UNIX platforms or Linux distributions, this is not the case by default. In those cases, you will have to issue the following command:

cp -L {file(s)} {destination folder}

The -L option in cp command forces it to follow (dereference) symlinks and copy the data that is pointed at by the symlink.

In case you do not want to follow symlinks, you can issue the following command instead:

cp -P {file(s)} {destination folder} 

Mounting ISO images on linux

So I learnt this today and decided to share it with you guys. If you have an iso image and want to mount it on Linux, simply execute the following command in terminal:

sudo mount -o loop mydisk.iso myfoldername

So in my case, I had iso file called 2012photos.iso on my desktop and I wanted to mount it on folder called 2012pics which was on Desktop as well. I executed:

sudo mount -o loop /home/manthan/Desktop/2012photos.iso /home/manthan/Desktop/2012pics

Managing password expiry for a user account (Unix/Linux)

I find this very useful on several occasions so here we go. Login as root and execute the following:
passwd -x -1

So if you want to remove password expiry from bob's account, type:
passwd -x -1 bob

Notice that -1 parameter after -x represents number of days before the password expires. Since we do not want the password to expire, we have set it to -1. However, you can set it to any value you want. So, if you want your password to expire after 90 days (i.e. 3 months) for bob's account, type:
passwd -x 90 bob

Removing password expiry from your unix machine (Linux and Solaris)

Here's a quick one. If you have a server and you do not want the password for a user to expire (as it can screw some things up while its active), you need to execute the following commands as root:

passwd -x -1

where is the username whose password expiry you wish to remove. For instance, in my case, if username is dm014, I executed:

passwd -x -1 dm014

I have tested this and it works flawlessly on linux and solaris.

How to tell which Linux partition a directory is on?

This is really simple. First of all, you need to know the full path to that directory. "cd" into that directory and type:
pwd

and it will display full path of the directory you are in. Remember that or even better, copy it to the clipboard. Type the following command to know the name of the partition:
df

where replace with the full path of your directory. This in my case becomes "df /data/android/sample".

If you are already in the directory, you can type "df ." and it will give details about the partition you are currently on.

The output should be something like:

Filesystem    1024-blocks      Free %Used    Iused %Iused Mounted on
 /dev/hd4            32768     16016   52%     2271    14% /

The highlighted part is the name of your partition.

Linux - Windows comparison

Well, I wrote the following article a year ago. A YEAR AGO! I'm actually amazed that I wrote such a thing. Well, so... Here it goes...

Linux, as you all know, is a free-open source operating system. Inspire of having number of advantages over Windows, Linux is still free and of course open source. Now, you might be thinking why I'm using terms 'free' and 'open' source separately. 'Free' means that Linux is freely available and downloadable from internet. Some of the vendors like 'Ubuntu' offer CDs loaded with Linux absolutely free of cost. Open source means that if you know kernel programming and mechanics of operating system, then you can actually tweak Linux and also program Linux from a scratch. That's open source and Linux provides that flexibility. Unlike Linux, Windows is not open source. It does not allow user to modify its kernel or components. This is one of the reasons why Linux is hot favourite among young developers.

What separates them?
The main difference between Linux and windows is their build. Practically, Windows is created only by Microsoft Corporation while Linux has been created and tweaked by entire world! Why entire world? Because, it's open source. If I have Linux and I find a bug, then I can tweak and remove that thing myself. I also tell the organization about the bug which I fixed. So, they verify it and tell their rest of the consumers about the fix. So, that's how it all works. With windows, it is not like that. I must appreciate Microsoft’s efforts in improving windows because in new versions of windows – The Vista Series – they have put an improved version of error reporting like Problems Report and Solutions. With that you can just submit the error report. You cannot tell them how to fix that even if you know. Of course, you cannot fix grave errors in windows even if you think because it's not open source!

Who's better on performance?
Practically, on considering stable releases of Linux offered by several organizations like Ubuntu, Debian, Fedora, etc., no doubt Linux outperforms Windows. Windows has always been a bulky operating system taking several minutes to boot up, then a minute to get stable and so on. While with Linux, running on an average PC, booting up is a child's play for Linux, 10 – 15 seconds is a limit. Even if 'background processing' indicator on mouse cursor is on, you can still get your stuff by clicking. Exploring your computer is just with double-click of a button. You'll never find this speed on Windows!
Also, shutting down your PC is a matter of 5-6 seconds for Linux. And what's more? Linux works on a 1 GHz PC with 128 MB RAM and even 10 GB Hard Drive!

What's your colour today?
Unlike Windows, Linux provides a lot of customization features. It is so flexible that you can customize almost everything. From changing your wallpaper to behaviour and animation of minimizing/maximizing a window, you can change and tweak everything. Linux also provides various desktop effects – some of which even require a 3D card – which are super cool.
For Linux, wallpapers are billions, themes are millions, and yet there are thousands of programs that help even more to customize Linux. Also various types of desktop environment are available on Linux like GNOME (Most common with cool GUI), KDE (High end graphical interface), and Xfce (Fastest of all). These desktop environments not only handle the look and feel of the Linux environment but also provide you with lot of other flexibilities. Instead of trying out new desktop environments, I prefer to GNOME and its cool Compiz-Fusion Desktop effects which tweak almost all aspects of the operating system. In customization, Linux easily overcomes Windows.

Applications?
This is where Linux gets outperformed. Almost 91% of the applications available in the market run on windows. 5% work on Mac OS X, 3% of it in Linux and the rest are for other OS. Windows has its roots wide-spread all over the world because it was the first OS to offer the idea of multi-tasking and a lot more new features that were not even thought of at that time. While Linux – which is not that old but still is not as old as windows – has relatively very less number of applications available worldwide. Linux is rapidly getting wide spread in the world of computing because it is a lot more flexible and offers a lot more possibilities than Windows. Linux offers its developers great flexibility of working with applications. Previously, Linux was called non-user friendly because it was difficult to operate but now that it has been tweaked and is available in several forms, Linux has removed this badge of dishonour from itself.
Organizations like Ubuntu etc. have made Linux user friendly and easy to use operating system that provides its users with number of options which makes it easy to use.

Running Windows on Linux?
Also, now you can run Windows applications on Linux. With 'Wine' – an application for Linux – you can run windows applications and programs on Linux. Wine uses its DLL library to run Windows applications. If you dual boot Windows and Linux, then it'll be better. Wine uses Windows library and its own library in combination to run the application smoothly. Wine also has configuration screen where, you can select in which mode Wine should work. There are several modes available based on Windows OS Released. On my Linux, there are following modes:
• Windows Vista
• Windows XP
• Windows Me
• Windows 98
• Windows 3.0
And several others including server editions which I cannot recall right now.
Each of these modes create a separate environment for the application to run smoothly. Here, environment does not mean that your Linux look would change. You won't even notice that! So, are you thinking of running Windows Games on your Linux? Don't dream that! It won't run. Games are called as hefty softwares that require large amount of resources and also they have some of functions which are accessible to them in Windows but not in Linux. However, these people are trying a lot to make each and every application of Windows run on Linux. Good luck to them!

Where's the security?
Linux has security from viruses. Don't think that an antivirus system has been integrated to it. It isn't in this case. Because, Linux does not support viruses! Yes, Viruses don't run on Linux because they find it hard to harass you while you are logged in. Surprised? Ok, let’s be frank. There's one virus on Linux and that's – Bliss. But just think of it. Linux has just one virus to support while Windows is entirely surrounded by viruses. Viruses for Windows – maybe billions – Viruses for Linux – Just One!! And that too is not going to do any big thing to your computer! So, here we have arrived to the point where we must say that Linux is too stout to respond to viruses. Consider a case where a burglar has entered a bank, he shows a gun, fires in the air and expects people to panic. But what happens? People ignore him! He shoots at the cashier but finds that his guns do not work against him. Bullets just fall down touching the cashier! This is the case when virus enters into Linux. Virus would try hard to do harm but Linux won't execute any operations that virus has told. Virus will try to infiltrate your files but that also would not work because unlike windows, Linux is not having applications chained.

Little things that matter!
Yet, there are a lot of little things offered by Linux that provide great advantage over Windows. First of all, Linux has a 'Force Quit' feature. If you find any misbehaving application (Though you'll rarely find) that's taking too long to respond, you can use Force Quit. Force quit is a utility provided in Linux which immediately exits a non-responding application. So, you won't need to wait minutes for a non-responding application to end. Sometimes, you'll find that in windows, when an application does not respond, then your computer will also not respond. This happens because in windows, all applications are chained. When running applications are chained, they all get affected if one of them stops responding. This is not the case in Linux, In Linux, applications work independently, meaning, that they are not chained. So, if any damn application gets 'hanged' your computer and other applications will continue working in full swing. Second advantage in Linux, that it has an in-built archive manager. With archive manager, you can handle multiple types of archives without installing a separate program for it. Linux also has a highly advanced calculator with three extended modes which are 'Advanced', 'Financial' and 'Programming'. Linux also comes with Password and Encryption key manager which allows you to create passwords for applications and folders. It also allows you to create encryption keys whose strength ranges from 2048 bits to 4096 bits. With that, you can encrypt archives, files and folders to ensure that they are safe. Many Linux editions also allow you to change the way login screen looks and behaviour of the boot loader and its appearance. Linux also allows you to scale CPU usage. Meaning, you can set how much of the CPU power Linux should use. This helps you a lot in saving power. Suppose, you are reading an eBook on your Linux PC during a journey, if you were using windows, and even in power saver mode, you'll drain a lot of battery. But in Linux, it dims the display automatically. Now, reading an eBook does not require processing power. So, you can set the value to least mode. In this way, Linux will stop most of background processes which drain battery and will ensure that the battery is saved. Now, as were discussing about battery, it reminds me of an amazing feature that Linux provides. Linux has an in-built system that provides detailed information on the battery installed on your laptop. It includes details like your battery model, version, status, and vendor. These details are common in windows. Excluding these, it also tells you details about battery technology, serial number, current charge capacity of battery(in %), current charge(in watt-hour), last full charge(in watt-hour) and design charge(in watt-hour). It also provides you with information based on graphs too! I'm sure that these features are not available on windows.
If you refer to new versions of windows, in Vista line, you’ll find that they are having preview panes. The moment, you select an item, its preview (if available) is shown in the preview pane. Previews are available for pictures, music, videos and some other types. Suppose, I click a music file. If my computer or rather Windows media player has that plugin, then it will show a mini media player in the preview pane with a play button. If I click play button, it will search the plugin in Windows Media library and then it will play the file. On my VAIO BZ, windows music preview does not work correctly. For the first file I play on starting the computer, it takes a lot of time in loading the preview pane. After that it plays the file. Now, Ubuntu Linux 9.04 has something new to offer. Now, unlike windows, Ubuntu has a 'Movie Player' as default player. While exploring my computer, if I stumble upon a music file which I want to preview, I simply hover my mouse over it and immediately, it'll start playing the file. But unfortunately in this technique, pausing is not possible. If I move down my mouse from that music file, the file immediately stops playing. Ubuntu also does not offer a seek bar!
Also, there's one thing where Linux surrenders. It's Silverlight Experience! At Windows, you have Silverlight integrated in Windows Internet Explorer as well as many other web browsers like Firefox. But, in Ubuntu (or I assume in most of Linux operating systems) Silverlight isn't available. Yes, if you do visit a website featuring Silverlight, it'll tell you to download an alternative version of Silverlight as moonlight but even after downloading moon light, I can't get Silverlight experience – It still says that Silverlight is missing.
So, what do you learn from this? Most of all high-end Microsoft technologies aren't shared by Microsoft.
For all Photoshop editors, Linux is a disappointment because there's probably no Linux edition for adobe Photoshop.
Now, Linux also has an in built application called Disk Analyser. It is an application which tells you what folder has been eating, rather reserving your disk space. It is very useful application in that way because it lets you know what to delete to free your space. It not only tells you about your file system but tells you the 'free space sharing' on any other device that you connect. In this case, windows gets defeated. It's not having an in built application for this, so, you'll have to wander-on-the-web to get one!
I've heard that Linux produces outstanding graphics but I haven't seen it in action and therefore I tell you about that so sure. But I think that it would be able to produce out-standing graphics because it's swift. Windows and other operating system themselves allocate very high memory and therefore they have relatively less to dedicate for graphics. Whereas in Linux, this case is totally different. Linux itself uses very less of your RAM as well as your hard drive. So, it can dedicate more graphics to your games and all that stuff. This is just a guess and I am not so sure about it. Built-in games of Linux are very standard – sort of only two-dimensional and therefore, you cannot judge its actual capacity. However, some of real 3D games are available but aren't satisfactory because they aren't developed that passionately. I purely do not mean to discourage or hurt the Linux game developers but despite of such nice capabilities of Linux to handle the resources you won't get any game on Linux that can compete the one on windows. This might be because the real 'expert' game developers are looking for money and hence they are moving towards development of games for windows as most of the high-end games in windows aren't free. Now, you might be wondering about 3D modelling software like Maya? As of now, I haven't encountered any such software but I am still looking for it. There must be some sort of software available on Linux. Don't worry, you'll obviously find one.

Speed with appearance?
This is very easy question to answer – You compromise speed, you'll get speed. But after you use Linux, you'll probably disagree with this. Even after using themes and effects, you'll observe that your working speed has been little affected. Shocked? I was too but I soon discovered this myself on my computer. Of course, you might suffer from a sort of decrease in speed at first but this decrease in speed will be rather unnoticed by you unless you are using a computer that's got a configuration which is just on the minimum system requirements of Linux. Now, if you know this, then there are different window managers in Linux. Most popular and used is GNOME. Still, there are two other types called KDE and Xfce. GNOME is for a computer user who wants both speed and appearance. KDE is for one who wants very cool appearance, mostly for high-fi users. Xfce us mostly used by users who have very low system requirements and want speed. Now, this is not always true because some users who just don't care about appearance even after having a high-end system use Xfce. Another one, which is not a window manager (Though most of users think it is), is Compiz Fusion. It accelerates the appearance of your system and allows you to perform a lot more tweaks like changing the open/close/minimize/maximize animations, window borders, various effects and things like modifying the application switcher method. All these will be better understood by you if you use hands on approach and get at least a Live CD of Ubuntu Linux. Different window managers provide their own options of modifying the appearance on Linux while on windows? You know, you'll have to stick to the like or not Windows own window manager. Now this is mostly attracting to users but if you consider old versions, then they weren't attractive. But now, considering Windows Vista and Windows 7, they seem to be cool. Still, you don't get options of really 'modifying' them. Don't tell me about the 'themes' option. That's of no use. If you want to change desktop background and other sorts of things without affecting the real appearance, then it's better to do it yourself rather than depending on it.
Still thinking of Windows? Think again! Because Linux is used by huge servers and supercomputers. It's their choice because they rely on the usability, flexibility and resource management of Linux. Shouldn't you rely on it?

Conditions in Linux Shell Script

If statements are important in any programming language. I had a bit of hard time while doing if statements in linux since they are different than those in a conventional programming language. First of all, an if statement to compare whether a value in a variable val1 is equal to 5 is like:

val1=5
if [ $val1 -eq 5 ]
then
     Code here to do some stuff
else
     If above condition is false, then code here gets executed
fi
Note that spaces are important when defining condition. $val1 refers to the variable and -eq simply implies equal to operator to compare if the $val1 is equal to 5.


Seems pretty easy? It is. Other conditional operators are:

Conditional Operator	Linux Equivalent
Greater than	-gt
Less than	-lt
Equal to	-eq
Not equal to	-ne

Now, if I want to compare strings, for instance, if variable answer is equal to "yes" then the code will be as follows:


if [ "$val" = "yes" ]
then
     Code here to do some stuff
fi


Again, spaces are important here.
In here, '=' can easily be replaced by conventional conditional operators like '>' or '<' or '!=' etc.

Variables in Linux Shell Script

Well, this has been a bit tricky for me so I decided to put it on my blog so that it can be helpful to you guys.

First of all, How to define a variable?


count=0

There! I've just defined a variable count with 0 as its initial value.
Now, How do you print value of a variable?

echo "$count"

or

echo $count

Next. How do you perform calculations on a variable?
Okay. So assume that you have two variables called val1 and val2 with values 3 and 4 respectively. Now, you want val3 to have the addition of val1 and val2. So,


val1=3
val2=4
val3=`expr $val1 + $val2`

Similarly, to do subtraction:

val4=`expr $val1 - $val2`


Note that spaces are very important here.

Now, if you wish to assign value returned by a command to a variable, then it works by doing:

wcount=`wc -w myfile.txt`

anything between ` and ` will be executed as linux command and the value returned by that command will be assigned back to the variable to left.