Tech Tips and Insights
When you need to do some processing in bash and display if a file exists, the following is a simple script that takes the filename in the $testfile variable and reports if it exists or not.
if [ -f $testfile ] then echo $testfile exists! else echo $testfile does not exist fi
I regularly have to find where a file is located and usually use find:
find . | grep
Where
#!/bin/sh if [ $# -lt 1 ]; then echo 1>&2 Usage: $0 "" exit 127 fi icase= while [ $# -ge 1 ]; do case $1 in -i) icase=$1;; *) search=$1 ;; esac shift done find . | grep $icase $search
This is just a simple script which I have named ffind that searches starting from the current directory. This lets me search with the following commands:
ffind bak$
This will search for all files that end with “bak” .
ffind -i edr
This will search for all files with “edr” regardless of case
I have been searching through source code and files for years. And I used to use find with a few arguments to get what I was looking for.
find . -exec grep {} ; -print
I used the -print after the exec arguments to show the file only when the grep succeeded. This worked but was a bit to type each time I used it. So I came up with a little script that would help me to search for the an argument and even allow a case insensitive search as well.
#!/bin/sh
if [ $# -lt 1 ]; then
echo 1>&2 Usage: $0 ""
exit 127
fi
icase=
search=
while [ $# -ge 1 ]; do
case $1 in
-i) icase=$1;;
*) search=$1 ;;
esac
shift
done
find . -type f -print0 | xargs -0 grep $icase $search
The script takes at least one argument and excepts a ‘-i’ argument to make grep use a case insensitive search. This will print the file name and the line that matches the search term. It can be easily adapted to show the count of matched patterns (‘-c’) or the line number (‘-n’).
So I was trying to do some work the other morning in my hotel room and my Windows 7 computer gave me the dreaded blue screen of death and would not boot past loading a certain library file. I was able to boot into an Ubuntu disk that I happened to have and found that there were many S.M.A.R.T. errors on the disk and wished I had thought to monitor this. So it seemed that my drive was cooked and I could not get any of the data from it.
I was disappointed that Spinrite was unable to save me from the problems that I was having and I paid 90 bucks for it. So I turned to tools in Linux that might do the job. I had used the dd utility to copy the contents of my drive to a new drive in the past, but dd will exit on the first error. So I found that there is an addition to the dd program called ddrescue which is part of the GNU Project.
This utility will let you copy the contents of a disk or partition and allow you to skip the errors or even retry the error spots a number of times.
The command line is something like:
ddrescue -r 3 /dev/sda1 /media/newvolume/diskimage.iso /media/newvolume/logfilefordiskimage
The really nice thing is if you have to stop the command for some reason, it will restart from the end of the log file. It does take quite a bit of time. I am 6 1/2 hours into recovering a 500GB disk, but it beats losing all my data. I will follow up this post with the results of the rescue and the next steps I took to recover the data.
Sometimes you have a drive that your want to mount under Linux. The mount command can make the drive available to you. But you need to also create a directory to hold the mount point.
mkdir /mnt/drivepoint
Then you can mount the drive to that mount point by using the mount command and giving the device identifier, in this case it is /dev/sda1, and the mount point which we created as /mnt/drivepoint. One thing to note, is that the mount will use whatever directory you specify, so if there are files located in the directory you use as a mount point, they will no longer be visible.
mount /dev/sda1 /mnt/drivepoint
This must be run as root or with the sudo command.
Every once in awhile, I need to backup a large directory from another computer. This can be done by creating a zip or tar of the files you are looking to save and then copying those zips to the other computer. But what it you have a very large directory and not enough room to store the extra file. Here is a way to use tar over ssh:
ssh tar zcvf – /wwwdata | ssh root@backupserver.mydomain.com “cat > /backup/wwwdata.tar.gz”
Did you know? You can Set file timestamp using touch command. One way to do this is:
touch -c -t YYMMDDhhmm filename.txt
I had to write a backup script the other day and used the find command to find the files that were older than a given amount of days.
DAYS_TO_KEEP=7
find /path/to/files* -type f -name *.bak -mtime $DAYS_TO_KEEP -exec rm {} ;
The -name argument uses the escaped asterisk so command completion does not replace it with all the file names.
The -mtime argument takes a number that is the number of days to look at.
The -exec argument takes a command and the {} is replaced with the current file name and the ; is the end of the command.
This is a small part of the power of the find command.
If you don’t have access to a Unix environment on your windows box, you can still search files with findstr. The findstr command will allow you to search files close to the same way as grep.
findstr /S "search string"
will recursively find your string in the current directory and subdirectories.
There are more options that you can find if you run the following command:
findstr /?
will show you the help message.
So every once in a while I want to see the progress of an upload or backup. So I turn to the watch command in Linux. The watch command gives the user the ability to re-run a command and see the updated output.
The syntax is ‘watch -n
So for watching a file downloads progress you could run the following:
watch -n 30 ls -l filedownloading
This would show the ls output and you could watch the size of the file grow as it is dowload progress.