Wednesday, August 10, 2005

tar in pieces

Ever needed to create a tar file that has multiple pieces ?
tar has some support for it, but it's pretty cumbersome.

Essentially, tar supports the "-L" switch which allows you to specify the maximum length before changing tapes. But you can use that to change files.
We will also need the "--new-volume-script" switch which will run a script every time we reached the max size. Finally, "--volno-file" is useful to track what volume you're on.
The basic trick is to rename/prepare the next file between volumes using the script tar will call.

The whole thing can be solved with 4 small scripts :
backup.sh - runs the backup command itself (and takes care of the last file).
pack.sh - gets called from backup.sh to handle each volume
restore.sh - used for restore
unpack.sh - gets called from restore.sh to handle volumes.

First backup :

backup.sh
========================
#!/bin/bash
if [ ! $# -eq "3" ]
then
echo "Usage $0 archive_name max_size_in_KB target_dir"
exit
fi
# remove volume counter
rm -rf volno
#create tar file
tar cvf $1 -L $2 --volno-file volno --new-volume-script "pack.sh $1" $3
# handle the last file
last_file_name=$1__`cat volno`.tar.gz
gzip -c $1 > $last_file_name
rm -rf $1
# END OF backup.sh


pack.sh
============
#!/bin/bash
#
if [ ! $# -eq "1" ]
then
echo "Usage: $0 file_name"
exit
fi
let "archive_no=`cat volno`-1"
echo Processing Archive $archive_no...
gzip -c $1 > $1__$archive_no.tar.gz
#END of pack.sh


and to restore it:

restore.sh
===========
#!/bin/bash
if [ ! $# -eq "1" ]
then
echo "Usage $0 first_archive_name_(without __X.tar.gz)"
exit
fi
# remove volume counter
rm -rf volno
#unzip the first file
gunzip -c $1__1.tar.gz > $1
#restore tar file
tar xvf $1 --volno-file volno --new-volume-script "/root/drm_backup/unpack.sh $1"
# remove temp file
rm -rf $1
# End of restore.sh

unpack.sh
==========
#!/bin/bash
#
if [ ! $# -eq "1" ]
then
echo "Usage: $0 file_name"
fi
let "archive_no=`cat volno`"
echo Processing Archive $archive_no...
gunzip -c $1__$archive_no.tar.gz > $1
# End of unpack.sh

SSH keys - automatic login failure (still getting password prompt)

One of the coolest features of SSH, is its support for secure keys. You can create public and private keys, and then no password is required for login from a certain host, while still maintaining a secure connection.
This is very useful for scripts.

There are a million guides on the web how to do it .. (just Google "how to setup ssh keys") but they many fail to mention one thing! What to do if you created/updated the "authorized_keys" file and the authentication still fails ??

Well, it turns out that SSHd is very picky when it comes to permissions on the file. It will FAIL (!!) if the authorized_keys permissions are not 600, or the directory is not 755. So if you updated the authorized_keys file, and you still get a password prompt. Ensure that your permissions are set as follows:

$ chmod 755 $HOME/.ssh (or better yet, 700)
$ chmod 600 $HOME/.ssh/authorized_keys

And try again ..
Good luck!

Tuesday, May 31, 2005

Finding out what a process is doing with strace

strace is a very powerfull and useful command, especially if you are trying to figure out what a certain process is doing ... (in other words, where the heck does it spend all its time). In essence what it does is tell you all the system calls and signals the process is making, which helps understand what it is doing.

You can run it in a couple of ways, either run the process within an "strace shell", or attach to an existing process.
The command prints A LOT of output, but it's usually pretty easy to understand what is going on

To run within an "strace shell":
strace -f -v COMMAND
strace -f -v -o output.txt COMMAND
(output.txt will have the strace output as oppsed to stdout)

To attach to a live process:
strace -f -v -p PID

strace is very powerful, and very detailed, but usually gives invaluable results!

(Thanks to my friend Eran for helping with syntax and such)

Recent update ..
Redhat just put out a nice little guide here : http://www.redhat.com/magazine/010aug05/features/strace/

Thursday, May 26, 2005

The power of mutt

Mutt is an e-mail client that for some reason many people don't know!

It's very powerful, especially when trying to send e-mail from command line with attachements, or similar requirements.
Here's a couple of useful commands I often use.

To send an e-mail with attachment (and no message body) use:
mutt -a attachment_filename -s "Subject goes here" user@domain.com < /dev/null

The above command will send the e-mail with the attachement and subject to user@domain.com. The redirect from /dev/null makes sure there is no message body.

If you want to send an HTML e-mail you can use:
mutt -e "my_hdr Content-Type: text/html" -s "Subject" user@domain.com < msg.html

The above command sends an HTML e-mail to user@domain.com, using msg.html as the input file. The -e parameter allows you to change the content type. You can also put this in your .muttrc if you use it often.

For more details see mutt.org

Saturday, May 21, 2005

mplayer is a better movie player !

Ok, so I tried Kaffeine (based on Xine) for a while .. I even upgraded to the latest xine engine and the latest Kaffeine version. And yet, some movies (mostly WMV) get distorted all the time. It kind'a looks like the refresh on the image does not work properly.
So, I gave up, and installed mplayer. I have to report, it's much better !
I think it loads faster, it plays inside the browser (unlike Kaffeine) and most important support movies better.

Installation is very quick using urpmi. As simple as: urpmi mplayer
This will install mplayer, and you still need the mplayer plugin for your browser. The latter can be installed using: urmpi mplayerplugin

Assuming you are like me .. you have Mozilla and FireFox installed, so the mplayerplugin gets installed in the wrong directory ...
To fix that, you just need to copy the file to the right places:
cp /usr/lib/mozilla/plugins/mplayerplug-in.so /usr/lib/firefox/plugins
cp /usr/lib/mozilla/components/mplayerplug-in.xpt /usr/lib/firefox/components

And that's that!

happy movie watching..

Wednesday, May 18, 2005

Changing the host name

I am sorry.. but changing the host name on a linux box is a pain in the ...
Come on .. why can't we come up with a single place (or at least a single command) that takes care of everything (and works on multiple distros).


The steps to change your host name are:
1. Change the current machine name : hostname 'New Name'
2. Change the boot settings at : /etc/sysconfig/network (change the HOSTNAME entry)
3. Change /etc/hosts. Make sure that in /etc/hosts you put both entries for your short and long host name. Usually you want to put the long and then the short.
e.g. : 192.168.6.14 benyolin.mydomain.com benyolin

To check it use :
hostname (should display the short one)
hostname -s (should display the short one)
hostname -f (should display the long one)

Enjoy!

Another way to look at PS

Sometimes, you want to know more about processes ..
PS can take all sorts of parameters.
I often use:
ps -ewwo pid,ppid,pmem,rss,vsize,tid,rtprio,ni,pri,psr,pcpu,etime,stat,args

and you can pipe grep " D " .. to see which processes in IO wait.

Monday, April 04, 2005

Adding Routes

Is it me, or does it feels like adding new static routes has been a moving target for too long ..
Every *nix flavor I worked on had a different scheme. In fact, even Linux changed the tune a couple of times..
It used to be that you added your routes in /etc/static-routes but (while may still work on some system) this is not the latest.

You add static routes per interface now.
Adding the route dynamically is easy, just type : route add <destination> gw <gateway>
But .. how do you make it persistent ? (i.e. linger after you boot)..

Well, you need to create a file in /etc/sysconfig/network-scripts.
The file name is: route-<interface> e.g. /etc/sysconfig/network-scripts/route-eth0

In this file add lines for your static routes. You can use "#" for comments.
Example:
# my local routes
172.20.3.0/24 via 192.168.0.1

that should do the trick ..
If you want to test your settings, you can run: ifup-routes eth0

Printing..

So, when I installed my machine I had my one netoworked laser printer (HP 4050TN), and Mandrake did a great job helping me to install it through the print manager. Worked without a hitch.
However, I recently got an inkjet, HP DeskJet 6540xi. After a quick look around I found out that I don't have the the right driver. However, HP is doing a damn good job to make sure the community is creating driver support for many printer, and http://linuxprinting.org is a great resource to learn how to configure your printers, as well as find the right drivers.

Mandrake comes with hpisj (which is the recommended driver for my deskjet). It is included in the printer-filters-10.1-0.2mdk rpm
(btw, you can find that out by typing : $ rpm -q --whatprovides /usr/bin/hpijs)
The included version is 1.6.1, which is almost the latest...
All that I had to do was to download the PPD file from linuxprinting.org, and using the Mandrake Control Center, go to Hardware->printing and add my new printer (first add the PPD and then pick it from the list).
After that, I just added a new queue, and started printing pictures!

One nice little tip, cups has a great little GUI to manage printers, jobs, etc. You can access it via: http://localhost:631

I have to admit, this was a lot easier than I thought!

print away.. :)

Sunday, March 20, 2005

RPM Hell

Ok, I am not even sure how it happened, but at some point I enabled a Mandrake Cooker site for urpmi (see the command urmpi.addmedia to add new sites).
well, be VERY careful when you do something like that. Becuase, from now on, many new updates you will install will be from the cooker site .. thus not necessarily the latest tested version but rather a version that is still under dev ...
Somehow I (during installation of some other package) I managed to upgrade my python version from 2.3.4 to 2.4.
The problem is that it broke a bunch of things ... among them bittorrent and rpmdrake.
So .. what do you do .. how do you go back ..

The first challenge was to discover which RPMs did I replace. Thanks to a tip I found on the RedHat site, I discovered the "--last" flag for the RPM command:
rpm -qa --last
will give you the list of RPMs and the day they were installed/updated. Great! Now I knew which ones I broke.
Next, I mounted the Mandrake original CD, pointed to the old Python RPMs and used the --old-package flag to update them.
RPM -Uvh --old-package python-2.3.4-6mdk libpython2.3-2.3.4-6mdk python-base-2.3.4-6mdk
(they are interdependent, so it's better to install them with one RPM command)
That fixed the Python RPMs, and now just running rpmdrake again, and installing updates upgraded them again to the lastest supported version (which is 2.3.4-6.1.10).

That got me out of RPM hell .. for now :)

Movie extensions for Firefox

Well, while Mandrake comes with some great tools (such as rpmdrake or urpmi) which make life really easy when you want to install new applications, it's pretty easy to screw things up ..

Ok .. so here's the deal. I installed firefox and then looked for extensions for it, so I can view movies and such. After doing some reasearch on the net, I found that there are kaffeine extensions for Mozilla/Firefox. There is even an RPM called: kaffeine-mozilla-0.2-1mdk.i586.rpm.
BUT, this RPM requires some newer libraries that I didn't have. Moreover, these libraries are not part of the standard Mandrake distrib yet, but rather are under Mandrake 'cooker', which is the development/beta phase of Mandrake packs.
So, the best way I found to install the kaffeine extensions is by downloading them from http://kaffeine.sourceforge.net .Then, you have to build the extensions and install them into the plugins directory of FireFox.
The main catch is that the RPM that is out there is for Kaffeine 0.5 which is not part of the standard Mandrake distro yet .. so you have to install the new Kaffeine if you want to use the extensions RPM. Or, just build them from the tar.gz.

Saturday, March 19, 2005

Digital Camera

If you haven't bought one yet, go to the gphoto site and make sure what you're about to buy is supported ! (http://gphoto.org/proj/libgphoto2/support.php).
If it's not .. you are looking for a world of pain :)

On my RHEL machine, I had to upgrade gphoto so it supported my Canon camera, and I never got it to work quite right. Also, the applications that come with RHEL to manage pictures are lame ! and I was never able to install gthumb that looked cool because the gnome version was too low.

On Mandrake, they have Digikam. Great application! very easy to use and does all the camera detection too.
However, one catch people ... It took me 3 frikin weeks to figure out that some computers (mine included) have different USB buses on the motherboard!!

My Dell Precision workstation has a USB 1.1 bus in front and USB 2 bus in the back .. go figure.
It wouldn't really matter except that when I connected the camera to the 1.1 bus, it got detected, but then gphoto was not able to communicate with it!

If you are looking for a way to figure out the speed of your bus, I don't know if that's the recommended way, but I just looked in the dmesg log. While booting the system will tell you the speed of the bus.
Mine says:
usb 1-2.3: new full speed USB device using address 3 the bus.
usb 2-1: new low speed USB device using address 2
(note the different terms "low speed and full speed")

It drove me crazy for days .. once I moved the camera to the fast bus ..that did the trick !

Which Window Manager

This is an easy one - after trying both Gnome and KDE -
KDE all the way !!

There are far more programs that work better on KDE, and far more useful utilities.
Most stuff today actually runs on both, but it just seems that KDE is more mature.

I have a PalmPilot, and the KPilot utility is prefect. btw, synching the Palm on 2.4 kernel is a nightmare! You have to be really creative with pilot-xfer, and I was never able to get it to sync automatically... I had to push the sync button and then run pilot-xfer.
On 2.6 kernel, with KDE - just push the button!

So, I am sure there will be a million people that will think otherwise, but my experience is KDE all the way.
Another testimony to the power of KDE is the new TUX magazine (http://www.tuxmagazine.com) a great magazine for novice Linux users. ALL their examples are KDE based. And these are experienced people :)

What Distro ??

Well, I have done so much research on this question, I still have no definite answer.
I tried the following: Redhat Enterprise Linux 3.0, Fedora Core 3, Ubuntu and Mandrake 10.1.
I finally settled on Mandrake!

Each one has benefits and drawbacks. Eventually, it boils down to the community, and the amount of support, drivers and people that are using it out there.
Ubuntu seemed to locked down .. I didn't like the whole sudo trick, that you don't really have root and all .. It felt to chewed ...
RHEL 3.0 - installation was very easy, but then the biggest drawback was that it supported a 2.4 kernel .. so any peripherals were a nightmare to install. Also, some newer utilities for digital pictures management and such did not work on the old gnome. And finally, it's a really old version of OpenOffice, which had all kinds of issues .. (RHEL 4 came out only a couple of weeks ago...)
The pros are that there is a huge following to RH in the US, and there are a ton of resources out there. Also, RHN is awesome.
If you do choose to go this route, and you don't want to pay the ridiculous amount of money RH is asking for, look closer on their site .. they have student rates ! All you need is a student friend, and you can get RHEL for $50 !! (including RHN).

A few weeks later I decided that I really need a newer kernel, and installed Mandrake 10.1. The installation was very smooth, and the user experience is awesome! It supports plug&play hardware and so far, anything I wanted to do was a breeze. Moreover, the Mandrake community (although it seems rather small in the US) is pretty big worldwide.
Mandrake provide great resources on their site. First of all, you have the Mandrake club, which gives you access to any RPM you'll ever need, and to user forums, and to much much more.
Then, they have MandrakeOnline, which is just like RHN !
Finally, MandrakeExpert - a kickass service ! After you spend two days trying to figure out something .. you can just post a question, and usually within 24 hours someone from the community will help you. Great idea, and great results.
I subscribed and paid the Mandrake fees. MandrakeClub includes MandrakeOnline membership. It's a little pricy ($130) but .. it makes life easier. You don't have to subscribe to run Mandrake. Almost any RPM can be found somewhere else on the net (pbone.net is a great source). But, it just makes life easier.

So my verdict thus far - Mandrake 10.1 rocks!
Even if you don't choose Mandrake, just make sure you use a 2.6 kernel, it makes a world of a difference.

At work I run FC-3 - it works great. It runs a 2.6 kernel, but it is clearly geared more towards a server than it is towards a Desktop. So while it does the job, I still vote Mandrake.

The Linux Desktop Experience

So .. I finally figured out what to do with this blog!
A few months ago I installed Linux as my desktop. While I LOVE it and I think it's one of the best decisions I have made in a long time, I realize that not many people have a Linux desktop yet, and it's really hard to find documentation and how to do certain things.
So, I decided to start sharing with the world my experiences ... hopefully it will help someone.