I was asked about system utilization on Linux systems. I found thesystatset of tools which consist of: sar, sadf, mpstat, iostat, nfsiostat, cifsiostat, pidstat and sa tools.

I will talk about the iostat tool in this article and follow up about some of the other tools.

On Ubuntu or debian systems you can install this toolset with:

sudo apt-get install sysstat

Then you will have access to the iostat command. The command output from executing with no options is below:

# iostat
Linux 2.6.32-33-generic (sumo)  09/01/2011      _i686_  (2 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.11    0.00    0.47    0.01    0.00   99.41

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda               0.54         1.85         8.91    1611385    7740024

This displays the avg-cpu utilization and a device usage list with basic utilization.
Some options you can use are

One more way to use the iostat tool is to specify an interval and count to the command

iostat -d -x 10 5

This will display the output 5 more times with 10 seconds between each run.

Tags: Linux, system tools, utilization

I happened to be creating a script and found myself trying to copy and past some functions between them. Well my functions
had some comments and this caused the paste operation into vi to add comments to every line after the first comment.

So this:

sub somefunc {
  # input args
  my $var1 = shift;
  my $var2 = shift;

  if ($var1 < $var2)
  {
    return $var1;
  }
  else
  {
    return $var2;
  }
}

Became this:

sub somefunc {
  # input args
  #my $var1 = shift;
  #my $var2 = shift;
  #
  #if ($var1 < $var2)
  #{
  #  return $var1;
  #}
  #else
  #{
  #  return $var2;
  #}
  #}

This made all my formatting go awry as well, and there were about 4 different functions I was about to cut and past. So I looked into how to make vi accept the text as is without adding formatting. The answer is the paste and nopaste mode.

To enter the mode to allow pasting without formatting:

:set paste

To return to regular mode:

:set nopaste

There are also ways to bind a key to more quickly enter and exit this mode if you are pasting on a regular basis.
That will be in another article.

Tags: editor, vi

If you ever need to schedule a job regularly on a Unix system, they have a great utility called cron. This will run a job at a regularly scheduled time.

Here is the format:

* * * * *  command to be executed
_ _ _ _ _ 
| | | | |
| | | | +--- day of week (0 = Sunday - 6 = Saturday)
| | | +----- month (1 - 12)
| | +------- day of month (1-31)
| +--------- hour(0-23)
+----------- minute (0-59)

example of a script that runs every day at 7:

* 7 * * * sendmeupdate

When I was trying to terminate a python script after a failure, I tried to find the nicest way to exit. The sys.exit() command provided by importing the sys library was what I found.

The function can take an optional numeric argument, usually in the range of 0-127, and with no argument it returns 0. The return code is passed back to the Operating System.

import sys
sys.exit(1)

When exit returns a non-zero value, it is considered an error.

Tags: Python

If you have ever used vi to edit source code, there are occasions where you get on a system that has a very difficult to read color mode. At these times you want the colors to just go away. You can use the following in command mode in vi:

This will toggle off the syntax highlighting mode.

:syntax off

This will turn it back on.

:syntax on

Tags: editors, syntax, vi

When you need to do some processing in bash and display if a file exists, the following is a simple script that takes the filename in the $testfile variable and reports if it exists or not.

if [ -f $testfile ]
then
  echo $testfile exists!
else
  echo $testfile does not exist
fi

I was working on a system the other day and needed to see where the end of lines were and where there were tabs instead of spaces. And I found the list option to vi.

This will turn on the display of “$” to indicate the end of line and “^I” to indicate a tab.

:set list

This will turn off the option to display the end of line and tabs.

:set nolist

Tags: editors, vi

I regularly have to find where a file is located and usually use find:

find . | grep 

Where is replaced with the filename or partial name to match. So I finally decided to write a script to just run this and allow me to print the matching names and even potentially pass the -i flag to grep.

#!/bin/sh


if [ $# -lt 1 ]; then
  echo 1>&2 Usage: $0 ""
  exit 127
fi
icase=

while [ $# -ge 1 ]; do
   case $1 in
     -i)  icase=$1;;
      *)  search=$1 ;;
   esac
   shift
done

find .  | grep $icase $search

This is just a simple script which I have named ffind that searches starting from the current directory. This lets me search with the following commands:

ffind bak$

This will search for all files that end with “bak” .

ffind -i edr

This will search for all files with “edr” regardless of case

Tags: find, Linux, scripting, Shell, Utilities

I have been searching through source code and files for years. And I used to use find with a few arguments to get what I was looking for.

find . -exec grep  {} ; -print

I used the -print after the exec arguments to show the file only when the grep succeeded.  This worked but was a bit to type each time I used it. So I came up with a little script that would help me to search for the an argument and even allow a case insensitive search as well.
 

#!/bin/sh

if [ $# -lt 1 ]; then
  echo 1>&2 Usage: $0 ""
  exit 127
fi
icase=
search=

while [ $# -ge 1 ]; do
  case $1 in 
     -i) icase=$1;;
      *) search=$1 ;;
  esac
  shift
done

find . -type f -print0 | xargs -0 grep $icase $search

The script takes at least one argument and excepts a ‘-i’ argument to make grep use a case insensitive search. This will print the file name and the line that matches the search term. It can be easily adapted to show the count of matched patterns (‘-c’) or the line number (‘-n’).

Tags: find, scripting, Shell, Utilities

So I was trying to do some work the other morning in my hotel room and my Windows 7 computer gave me the dreaded blue screen of death and would not boot past loading a certain library file. I was able to boot into an Ubuntu disk that I happened to have and found that there were many S.M.A.R.T. errors on the disk and wished I had thought to monitor this. So it seemed that my drive was cooked and I could not get any of the data from it.

    I was disappointed that Spinrite was unable to save me from the problems that I was having and I paid 90 bucks for it. So I turned to tools in Linux that might do the job. I had used the dd utility to copy the contents of my drive to a new drive in the past, but dd will exit on the first error. So I found that there is an addition to the dd program called ddrescue which is part of the GNU Project.

This utility will let you copy the contents of a disk or partition and allow you to skip the errors or even retry the error spots a number of times.

The command line is something like:

    ddrescue -r 3 /dev/sda1 /media/newvolume/diskimage.iso /media/newvolume/logfilefordiskimage

The really nice thing is if you have to stop the command for some reason, it will restart from the end of the log file. It does take quite a bit of time. I am 6 1/2 hours into recovering a 500GB disk, but it beats losing all my data. I will follow up this post with the results of the rescue and the next steps I took to recover the data.

Part 2 of article

Tags: dd, ddrescue, Linux