If you ever need to schedule a job regularly on a Unix system, they have a great utility called cron. This will run a job at a regularly scheduled time.

Here is the format:

* * * * *  command to be executed
_ _ _ _ _ 
| | | | |
| | | | +--- day of week (0 = Sunday - 6 = Saturday)
| | | +----- month (1 - 12)
| | +------- day of month (1-31)
| +--------- hour(0-23)
+----------- minute (0-59)

example of a script that runs every day at 7:

* 7 * * * sendmeupdate

When I was trying to terminate a python script after a failure, I tried to find the nicest way to exit. The sys.exit() command provided by importing the sys library was what I found.

The function can take an optional numeric argument, usually in the range of 0-127, and with no argument it returns 0. The return code is passed back to the Operating System.

import sys
sys.exit(1)

When exit returns a non-zero value, it is considered an error.

Tags: Python

If you have ever used vi to edit source code, there are occasions where you get on a system that has a very difficult to read color mode. At these times you want the colors to just go away. You can use the following in command mode in vi:

This will toggle off the syntax highlighting mode.

:syntax off

This will turn it back on.

:syntax on

Tags: editors, syntax, vi

When you need to do some processing in bash and display if a file exists, the following is a simple script that takes the filename in the $testfile variable and reports if it exists or not.

if [ -f $testfile ]
then
  echo $testfile exists!
else
  echo $testfile does not exist
fi

I was working on a system the other day and needed to see where the end of lines were and where there were tabs instead of spaces. And I found the list option to vi.

This will turn on the display of “$” to indicate the end of line and “^I” to indicate a tab.

:set list

This will turn off the option to display the end of line and tabs.

:set nolist

Tags: editors, vi

I regularly have to find where a file is located and usually use find:

find . | grep 

Where is replaced with the filename or partial name to match. So I finally decided to write a script to just run this and allow me to print the matching names and even potentially pass the -i flag to grep.

#!/bin/sh


if [ $# -lt 1 ]; then
  echo 1>&2 Usage: $0 ""
  exit 127
fi
icase=

while [ $# -ge 1 ]; do
   case $1 in
     -i)  icase=$1;;
      *)  search=$1 ;;
   esac
   shift
done

find .  | grep $icase $search

This is just a simple script which I have named ffind that searches starting from the current directory. This lets me search with the following commands:

ffind bak$

This will search for all files that end with “bak” .

ffind -i edr

This will search for all files with “edr” regardless of case

Tags: find, Linux, scripting, Shell, Utilities

I have been searching through source code and files for years. And I used to use find with a few arguments to get what I was looking for.

find . -exec grep  {} ; -print

I used the -print after the exec arguments to show the file only when the grep succeeded.  This worked but was a bit to type each time I used it. So I came up with a little script that would help me to search for the an argument and even allow a case insensitive search as well.
 

#!/bin/sh

if [ $# -lt 1 ]; then
  echo 1>&2 Usage: $0 ""
  exit 127
fi
icase=
search=

while [ $# -ge 1 ]; do
  case $1 in 
     -i) icase=$1;;
      *) search=$1 ;;
  esac
  shift
done

find . -type f -print0 | xargs -0 grep $icase $search

The script takes at least one argument and excepts a ‘-i’ argument to make grep use a case insensitive search. This will print the file name and the line that matches the search term. It can be easily adapted to show the count of matched patterns (‘-c’) or the line number (‘-n’).

Tags: find, scripting, Shell, Utilities

I need to write custom Perl libraries sometimes and an easy method to make sure that they are found on the path is to use the PERL5LIB environment variable.

So either set it in your current environment or in a shell you can just specify the contents before you run your script.

PERL5LIB=/PATH/WHERE/LIB/IS perl scripttorun.pl 

Tags: perl, scripting

So, after a Linux crash that caused my previous ddrescue to fail(after 350GB), I started to look into how to speed it up, because 30+ hours did not sound like a good idea to me.
I look into ways to speed it up and using a larger block size did the trick. I experimented with a few different sizes and figured out that using 2M gave me the best throughput. So instead of a couple of megs a second i was getting over 24 MB/s. So about a 4 1/2 hour recovery after that.

ddrescue -b 2M /dev/sda2 /media/newdisk/olddata /media/newdisk/logfile

The ddrescue process will attempt to recover any lost sectors and retrim the lost data. Luckily, I only lost about 3100k of data so the most of my data is safe.

You can attempt to rerun ddrescue with the -r <# of times to retry> and it will continue to attempt to read those failed sectors that were unrecoverable. There is no guarantee that you will get back all the data.

Once this data is recovered, you can (hopefully) mount this file as a disk under linux.

 mount /media/newdisk/olddata /mnt/newdisk 

Now remember to back up your data and you will not go through the pain that I did to recover this data.

Part 1 of article

Tags: dd, ddrescue, Linux

So I was trying to do some work the other morning in my hotel room and my Windows 7 computer gave me the dreaded blue screen of death and would not boot past loading a certain library file. I was able to boot into an Ubuntu disk that I happened to have and found that there were many S.M.A.R.T. errors on the disk and wished I had thought to monitor this. So it seemed that my drive was cooked and I could not get any of the data from it.

    I was disappointed that Spinrite was unable to save me from the problems that I was having and I paid 90 bucks for it. So I turned to tools in Linux that might do the job. I had used the dd utility to copy the contents of my drive to a new drive in the past, but dd will exit on the first error. So I found that there is an addition to the dd program called ddrescue which is part of the GNU Project.

This utility will let you copy the contents of a disk or partition and allow you to skip the errors or even retry the error spots a number of times.

The command line is something like:

    ddrescue -r 3 /dev/sda1 /media/newvolume/diskimage.iso /media/newvolume/logfilefordiskimage

The really nice thing is if you have to stop the command for some reason, it will restart from the end of the log file. It does take quite a bit of time. I am 6 1/2 hours into recovering a 500GB disk, but it beats losing all my data. I will follow up this post with the results of the rescue and the next steps I took to recover the data.

Part 2 of article

Tags: dd, ddrescue, Linux