AdmiNirvana

I think I’ve mentioned many times on this blog, but one of the most satisfying things regarding Linux and Unix are that you’re never done learning about it. A never-ending lesson in operating systems! Well, chalk up another lightbulb moment for me this morning.

Imagine a script wherein a process needs to be checked for proper exit. Let’s say “mysqldump”. Typically I’d do something like this, for example:


#!/bin/bash
STATUS=1
while [ ${STATUS} -ne 0 ]
do
mysqldump -uroot -psomepass –all-databases > sql-backup.sql
STATUS=${?}
done


exit 0


That’ll work just fine–the special reserved variable ${?} contains the exit code of the last run command. Mysqldump is kind enough to use non-zero ones on any kind of error, so if it doesn’t work in our script we’ll retry.

But for instance, let’s say our script looks like this:


#!/bin/bash
STATUS=1
while [ ${STATUS} -ne 0 ]
do
mysqldump -uroot -psomepass –all-databases | gzip > sql-backup.sql
STATUS=${?}
done


exit 0


The problem here is that ${?} now contains the exit code for gzip, not mysqldump! Will gzip respond properly if mysqldump doesn’t provide an input stream from the pipe? Maybe, maybe not. Bottom line is that it isn’t reliable, and not what I’d consider good shell programming.

Instead, check out this solution:


#!/bin/bash


STATUS=1
while [ ${STATUS} -ne 0 ]
do
mysqldump -uroot -psomepass –all-databases | gzip > sql-backup.sql
STATUS=${PIPESTATUS[0]}
done


exit 0


The BASH reserved array ${PIPESTAUTUS[x]} contains the exit codes for all programs in the array. In this example, ${PIPESTATUS[0]} is mysqldump, and ${PIPESTATUS[1]} is gzip.

Found this interesting post over at SmugBlog.  Apparently these fellows experience the same issue with the Linux kernel swapping needlessly as we do at TLF.  At least I know we’re not alone now.

I think the best part is the solution they came up with… check it out.

As anyone whom reads this blog regularly knows, I’m a happy VMware Server user. In using it, I’ve come across some handy methods in administrating it and the virtual machines created with it. Without further ado, here they are!

Tip #1 - Start and stop your VMs from the command line

If your VMware server is headless and gui-less (you didn’t install a GUI did you?) it’s handy to be able to start and stop your VM processes with a command line tool over ssh. Use the vmware-cmd tool for this:

vmware-cmd /path/to/vmxfile.vmx stop <hard|soft>

or

vmware-cmd /path/to/vmxfile.vmx start

The third option is the powerop mode. ’soft’ uses the VMware tools within the guest OS, while ‘hard’ simply powers on and off the VM without the tools.

Tip #2 - Re-install your VM Tools quickly

After upgrading your kernel on Linux-based virtual machines, you’ll also have to re-compile vmware tools’ kernel modules. Upon initial installation, you probably executed the usual:

/usr/bin/vmware-config-tools.pl

But did you know you can speed up the process and make it automatic by using the default options? The next time you need to recompile your tools, use this instead

/usr/bin/vmware-config-tools.pl -default

Tip #3 - Fine-grain your VM’s priority

VMware Server does not provide the flexibility of ESX, but you can get it part-way there by using the Linux scheduler to prioritize your virtual machines. By default, VS gives all vmware-vmx processes a nice value of “-10″. In Linux, processes with “-20″ have the highest priority for system resources, and “20″ have the lowest. By adjusting your busy VMs to a higher negative number (e.g. -15) and your less-intensive VMs to a higher positive number (e.g 0) you can more finely tune your server’s performance and ensure timeslices on the host are more accurately granted.

To do this, use the `renice’ command. First, find the PIDs of your vmware-vmx processes, by using `ps’:

[root@tlfvm5 ~]# ps -ef | grep vmware-vmx

root      3374     1 13 Mar18 ?        2-20:03:36 /usr/lib/vmware/bin/vmware-vmx -C /vmware/tlfmonitor/tlfmonitor.vmx -@ ""

root      4833     1 15 Mar18 ?        3-04:09:11 /usr/lib/vmware/bin/vmware-vmx -C /vmware/DellMonitor/DellMonitor.vmx -@ ""

Then renice the appropriate PID. For example, to give the “tlfmonitor” a bit of a bump to “-12″:

renice -12 33

Like all good things, moderation is key. Start with smaller increments and note the change, then if needed bump it again. It should be noted that your reniced values will disappear as soon as the PID terminates. You can also give it a default higher priority via the .vmx file in the prority.grabbed and priority.ungrabbed directives (see http://sanbarrow.com/vmx/vmx-config-ini.html).

Tip #4 - Manage and extend your virtual disks

VMware Server comes with a tool to completely manage your .vmdk disks. The vmware-vdiskmanager tool can create, defrag, extend, and convert vmdks from one type to another. For example, to expand a vmdk from 10GB to 15GB, power off the VM and issue this command:

vmware-vdiskmanager -x 15Gb /path/to/vmdkfile.vmdk

Note that this extends the raw disk, but not the guest file system. For instance, after doing an extend in Linux on an ext3 file system, use “resize2fs” to adjust it accordingly. You may want to run the vmware-vdiskmanager command without arguments to see some help on the different options, as well as some examples.

Tip #5 - Install VMware tools from the command line

You don’t need to click “VM -> Install Vmware Tools…” on the Server Console to mount the virtual media. Do it from the command line!

vmrun installtools /path/to/vmxfile.vmx

This does precisely what clicking in the GUI does. Once this has been run from the host, go to your VM and mount up the /dev/cdrom device and find your tools RPM ready to go.

That’s it for now. Do you have any tips that are useful for other VMware Server administrators? If so, let me know!

I run across this quite a bit and sort of do a “tsk-tsk” to the perpetrator. Greg of yesthatsright.net suggested I make a post about it.

Let us assume you need to perform an operation on a bunch of files. I typically see it done as so in the BASH interpretor:


for I in `ls *.sh`
do
something
done


Now, performing the backtick (`) subshell generally causes some hard to handle data mangling issues. Most commonly a space in the file name or special character will result in odd permutations of ${I}. For example in a file listing such as:


file1.sh
file2.sh
file three.sh


Your resulting loop will end up with:


[bdowney@tlfmgt1 ~]$ for I in `ls *.sh`; do echo ${I}; done
file1.sh
file2.sh
file
three.sh


The subshell returns strings to `for’ which interprets whitespace as a new loop iteration. This obviously causes issues for your loop logic, and I have seen people build in some pretty elaborate methods to handle it. But it’s all for not–since `for’ is a reserved shell function, it inherently understands file globbing. Thus:


[bdowney@tlfmgt1 ~]$ for I in *.sh; do echo ${I}; done
file1.sh
file2.sh
file three.sh


Is all you need to do and solve your problem. But what if one has a list of filenames in a file, or needs to pass more arguments to provide a better list of said files (for example, ls -t?). Not to worry, the BASH built-in `read’ to the rescue!


[bdowney@tlfmgt1 ~]$ ls -tr *.sh | while read I; do echo ${I}; done
file1.sh
file2.sh
file three.sh


And as you’d expect, works the same when redirecting STDIN:

[bdowney@tlfmgt1 ~]$while read I; do echo ${I}; done < input.file


So stop wasting those extra shell processes!

I happened to run across this fella today, who also runs a very large VMware Server farm in a production environment. He makes a few mentions of his architecture which is slightly different than our approach, however it could prove handy to someone else building out such a thing.