This is an old revision of the document!


shell tips

Here's a tutorial: Advanced Bash-Scripting Guide.

Quick Tips

The History Expansion character is "!". To search the history for a previous "scp" command and only print it, try the first line below. But if you want to interactively find that command, type <Ctrl>+r,scp.

$ !?scp?:p
$ ^rscp

bash expansion

$ cp file{,.bk}

expands to

$ cp file file.bk

Replace all files that end with .JPG to .jpeg

for file in *.JPG; do mv $file ${file%.JPG}.jpeg; done
for file in *.JPG; do mv $file ${file/JPG/jpeg}; done

Then there are two different "rename" commands:

rename .JPG .jpg *.JPG 
rename "s/JPG/jpg/" *.JPG 

Command Template

Here's a template for shell commands that demonstrates a number of arguments, length of argument, etc. It could still stand a bit of clean-up according to the Google Shell Style Guide.

Another good resource is Better Bash Scripting in 15 minutes.

#!/bin/bash
set -euf -o pipefail # See: https://sipb.mit.edu/doc/safe-shell/
 
declare -r SCRIPT_NAME=$(basename "$BASH_SOURCE")
 
## exit the shell (default status code: 1) after printing the message to stderr
die() {
    echo >&2 "$1"
    exit ${2-1}
}
 
## the options used by this script
DISK=e
declare -i VERBOSE=0
 
## exit the shell (with status 2) after printing the message
usage() {
    echo "\
$SCRIPT_NAME -hv [Drive Letter] (default: $DISK)
    -h      Print this help text
    -v      Enable verbose output
"
    exit 2;
}
 
## Process the options
while getopts "hv" OPTION
do
  case $OPTION in
    h) usage;;
    v) VERBOSE=1;;
    \?) usage;;
  esac
done
 
## Process the arguments
shift $(($OPTIND - 1))
 
if [ $# -eq 0 ]; then
    : # Let the default be used
elif [ $# -eq 1 ]; then
    if [ ${#1} -eq 1 ]; then
        DISK=$1
    else
        # 64 is EX_USAGE from sysexits.h
        die "$SCRIPT_NAME: Drive Letter can only be one character long." 64
    fi
else
    usage;
fi
 
## Lock this if only one instance can run at a time
# UNIQUE_BASE=${TMPDIR:-/tmp}/"$SCRIPT_NAME".$$
LOCK_FILE=${TMPDIR:-/tmp}/"$SCRIPT_NAME"_"$DISK".lock
if [ -f "$LOCK_FILE" ]; then
  die "$SCRIPT_NAME is already running. ($LOCK_FILE was found.)"
fi
trap "rm -f $LOCK_FILE" EXIT
touch $LOCK_FILE
 
## The main work of this script
 
if [ ! -d /cygdrive/"$DISK"/backup/Users ]; then
    mkdir -p /cygdrive/"$DISK"/backup/Users
fi
 
((VERBOSE==1)) && echo "Starting at $(date)"
rsync /cygdrive/c/Users/me /cygdrive/"$DISK"/backup/Users

Miscellaneous Shell Tips

If you want a single column of just the file and path names, you can get it like so:

ls --format=single-column

But if you don't know what you're doing, you might construct something like so:

ls -Al | tr -s ' ' | cut -d ' ' -f10-
  1. List "almost all" items in "long" format (one line per item)
  2. Squeeze repeats of the space character
  3. Cut out everything from before the 10th column and show everything afterwards.

Of course, if you could assert the following:

  • none of the first columns were repeats (awk would only identify the first repeated column)
  • the desired column didn't have delimiters in it (filenames with spaces)

…you could use awk

... | awk '{print $10}'

Anyway, given a list of directories, they can be inserted into a cp command with xargs if you need.

cat list_of_directories_at_one_level.txt | xargs -I {} cp -r $SOURCEDIRPREFIX:{} $DEST

Useful bash command for finding strings within python files…

find . -name \*.py -type f -print0 | xargs -0 grep -nI "timeit"

Interesting way to use grep -v to remove paths from a list generated by find. Not sure about the escaped | character, though…

#!/bin/bash
find $PWD -regex ".*\.[hcHC]\(pp\|xx\)?" | \
    grep -v " \|unwantedpath/unwantedpath2\|unwantedpath3" > cscope.files
cscope -q -b

Here's how to find if a symbol is in a library, and how to search lots of object files and print the filename above the search…

nm obj-directory/libmyobject.a | c++filt | grep Initialize_my_obj
find bindirectory/ -name \*.a -exec nm /dev/null {} \; 2>/dev/null | \
    c++filt | grep -P "(^bindirectory.*\.a|T Initialize_my_obj)"

Also handy to merge two streams together…

( cat file1 && cat file2 ) | sort

When a little quick math is needed, use bc

$ bc <<< "obase=16;ibase=10;15"
F
$ bc -l <<< 1/3
.33333333333333333333
$ bc <<< "scale=2; 1/3"
.33
$ bc <<< "obase=10;ibase=16;B"
11

and, when coverting from hex to dec…

echo $((0x2dec))

But, then again, does that really seem easier than,

python -c "print int('B',16)"

There's a bash way to calculate how many days ago a date was:

$ echo $(( ($(date +%s) - $(date -d "2012-4-16" +%s)) / 86400 ))

And a Python way…

python -c "import datetime; print (datetime.date.today() - datetime.date( 2012, 4, 16 )).days"

And for displaying lines clipped at the right edge of the window instead wrapped:

cat_one_line_per_row() {
  cat "$@" | expand | cut -b1-$COLUMNS
}

or a "clip" command like so:

alias clip="expand | cut -b1-\$COLUMNS"

ctags's man page says that one of its bugs is that it has too many options. Ain't that the truth. Make note of the obscure flag here, –c++-kinds=+p, that tells ctags to process prototypes and method declarations.

ctags -n --if0=yes --c++-kinds=+p --langmap=c++:+.inl.lst \
    --langmap=asm:+.inc --file-tags=yes -R --extra=fq \
    --exclude=unwanted_file.lst \
    --exclude='*unwanted-directory*/*' \
    --regex-C++='/^.*CINIT.(.+),.*,.*,.*/CURLOPT_\1/'

When you want to repeat a command a few times…

seq 1 50 | xargs -I{} -n1 echo '{} Hello World!'

When you've set up Perforce to use an application for diff with export P4DIFF='vim -d' , you can still do a regular diff like so:

$ P4DIFF=; p4 diff hello-world.cpp

Keywords: bash shell sh zsh

.vimrc tips

Here's an alternative way to automatically save backups (with dates in the filename) everytime you save a file.

set backup
set backupdir=~/.vim/backup/
au BufWritePre * let &bex = '-' . strftime( "%Y%m%d-%H%M%S" )

That makes a lot of files, so you can clean out the backups with a cron job like this:

# at 3 in the morning on Mondays, delete files older than 30 days
0 3 * * 1 find $HOME/.vim/backup/ -type f -mtime +30 -delete

expect tips

What to do when it's not sure you're going to make a connection?

set times 0
set made_connection 0
set timeout 120
while { $times < 2 && $made_connection == 0 } {
    spawn nc $SERVER
    send "\r"
    expect {
        "login:" {
            send "john.doe\r"
            set made_connection 1
        } eof {
            sleep 1s
            set times [ expr $times + 1 ]
        } timeout {
            puts "Didn't expect to timeout."
            exit
        }
    }
}

I think the following is wrong-headed. It's not usually the case that spawn will fail.

set times 0;
while { $times < 2 && $made_connection == 0 } {
    if { [ catch { spawn nc $SERVER } pid ] } {
        set times [ expr $times + 1 ];
        sleep 1s;
    } else {
        set made_connection 1
    }
}

Perl tips

The module Search::Dict has a "look" function that can be used to do a binary search in an ordered dictionary file (a logfile (or log file) that starts with timestamps works). File::SortedSeek might also be recommended.

Additional Keywords

Linux, Unix, *nix

shell.1400020963.txt.gz · Last modified: 2014/05/13 15:42 by dblume
 
Recent changes RSS feed Driven by DokuWiki