This is an old revision of the document!

shell tips

Here's a tutorial: Advanced Bash-Scripting Guide.

Quick Tips

The History Expansion character is "!". To search the history for a previous "scp" command and only print it, try the first line below. But if you want to interactively find that command, type <Ctrl>+r,scp.

$ !?scp?:p
$ ^rscp

bash expansion

$ cp file{,.bk}

expands to

$ cp file file.bk

Replace all files that end with .JPG to .jpeg

for file in *.JPG; do mv $file ${file%.JPG}.jpeg; done
for file in *.JPG; do mv $file ${file/JPG/jpeg}; done

Then there are two different "rename" commands:

rename .JPG .jpg *.JPG 
rename "s/JPG/jpg/" *.JPG 

Command Template

Here's a template for shell commands that demonstrates a number of arguments, length of argument, etc. It could still stand a bit of clean-up according to the Google Shell Style Guide.

Another good resource is Better Bash Scripting in 15 minutes.

set -euf -o pipefail # See:
declare -r SCRIPT_NAME=$(basename "$BASH_SOURCE")
## exit the shell (default status code: 1) after printing the message to stderr
die() {
    echo >&2 "$1"
    exit ${2-1}
## the options used by this script
declare -i VERBOSE=0
## exit the shell (with status 2) after printing the message
usage() {
    echo "\
$SCRIPT_NAME -hv [Drive Letter] (default: $DISK)
    -h      Print this help text
    -v      Enable verbose output
    exit 2;
## Process the options
while getopts "hv" OPTION
  case $OPTION in
    h) usage;;
    v) VERBOSE=1;;
    \?) usage;;
## Process the arguments
shift $(($OPTIND - 1))
if [ $# -eq 0 ]; then
    : # Let the default be used
elif [ $# -eq 1 ]; then
    if [ ${#1} -eq 1 ]; then
        # 64 is EX_USAGE from sysexits.h
        die "$SCRIPT_NAME: Drive Letter can only be one character long." 64
## Lock this if only one instance can run at a time
if [ -f "$LOCK_FILE" ]; then
  die "$SCRIPT_NAME is already running. ($LOCK_FILE was found.)"
trap "rm -f $LOCK_FILE" EXIT
touch $LOCK_FILE
## The main work of this script
if [ ! -d /cygdrive/"$DISK"/backup/Users ]; then
    mkdir -p /cygdrive/"$DISK"/backup/Users
((VERBOSE==1)) && echo "Starting at $(date)"
rsync /cygdrive/c/Users/me /cygdrive/"$DISK"/backup/Users

Miscellaneous Shell Tips

If you want a single column of just the file and path names, you can get it like so:

ls --format=single-column

But if you don't know what you're doing, you might construct something like so:

ls -Al | tr -s ' ' | cut -d ' ' -f10-
  1. List "almost all" items in "long" format (one line per item)
  2. Squeeze repeats of the space character
  3. Cut out everything from before the 10th column and show everything afterwards.

Of course, if you could assert the following:

  • none of the first columns were repeats (awk would only identify the first repeated column)
  • the desired column didn't have delimiters in it (filenames with spaces)

…you could use awk

... | awk '{print $10}'

Anyway, given a list of directories, they can be inserted into a cp command with xargs if you need.

cat list_of_directories_at_one_level.txt | xargs -I {} cp -r $SOURCEDIRPREFIX:{} $DEST

Useful bash command for finding strings within python files…

find . -name \*.py -type f -print0 | xargs -0 grep -nI "timeit"
find . -type f \( -name \*.[ch]pp -or -name \*.[ch] \) -print0 | xargs -0 grep -nI printf

Interesting way to use grep -v to remove paths from a list generated by find. Not sure about the escaped | character, though…

find $PWD -regex ".*\.[hcHC]\(pp\|xx\)?" | \
    grep -v " \|unwantedpath/unwantedpath2\|unwantedpath3" > cscope.files
cscope -q -b

Here's how to find if a symbol is in a library, and how to search lots of object files and print the filename above the search…

nm obj-directory/libmyobject.a | c++filt | grep Initialize_my_obj
find bindirectory/ -name \*.a -exec nm /dev/null {} \; 2>/dev/null | \
    c++filt | grep -P "(^bindirectory.*\.a|T Initialize_my_obj)"

Also handy to merge two streams together…

( cat file1 && cat file2 ) | sort

When a little quick math is needed, use bc

$ bc <<< "obase=16;ibase=10;15"
$ bc -l <<< 1/3
$ bc <<< "scale=2; 1/3"
$ bc <<< "obase=10;ibase=16;B"

and, when coverting from hex to dec…

echo $((0x2dec))

But, then again, does that really seem easier than,

python -c "print int('B',16)"

There's a bash way to calculate how many days ago a date was:

$ echo $(( ($(date +%s) - $(date -d "2012-4-16" +%s)) / 86400 ))

And a Python way…

python -c "import datetime; print ( - 2012, 4, 16 )).days"

And for displaying lines clipped at the right edge of the window instead wrapped:

cat_one_line_per_row() {
  cat "$@" | expand | cut -b1-$COLUMNS

or a "clip" command like so:

alias clip="expand | cut -b1-\$COLUMNS"

ctags's man page says that one of its bugs is that it has too many options. Ain't that the truth. Make note of the obscure flag here, –c++-kinds=+p, that tells ctags to process prototypes and method declarations.

ctags -n --if0=yes --c++-kinds=+p --langmap=c++:+.inl.lst \ --file-tags=yes -R --extra=fq \
    --exclude=unwanted_file.lst \
    --exclude='*unwanted-directory*/*' \

When you want to repeat a command a few times…

seq 1 50 | xargs -I{} -n1 echo '{} Hello World!'

When you've set up Perforce to use an application for diff with export P4DIFF='vim -d' , you can still do a regular diff like so:

$ P4DIFF=; p4 diff hello-world.cpp

It's hard to be sure which Perforce changelist you sync'ed if you didn't explicitly sync to a changelist.

So, use p4_sync to sync to a specific changelist, and update a source file too.

p4_sync() {
    p4 changes -s submitted -m1 ... | tee p4_sync_to_change.txt 
    changelist=`cut -d " " -f 2 p4_sync_to_change.txt`
    p4 sync ...@$changelist
    if [ -w $changelist_filename ]
	sed -i 's/"[0-9]\+";/"'$changelist'";/' $changelist_filename

Note the use of $@ vs "$*" in the next function that automatically saves an archive of a telnet session.

telnet_log() {
    curtime=$(date -Iseconds)
    args=$(echo "$*" | tr ' ' '_')
    telnet $@ | tee $HOME/telnetlog/$args\_${curtime::-5}.log

Of course if you do that, you'll want to occasionally (via cronjob?) delete old archives.

find $HOME/telnetlog/ -type f -mtime +6 -delete

Keywords: bash shell sh zsh

.vimrc tips

Here's an alternative way to automatically save backups (with dates in the filename) everytime you save a file.

set backup
set backupdir=~/.vim/backup/
au BufWritePre * let &bex = '-' . strftime( "%Y%m%d-%H%M%S" )

That makes a lot of files, so you can clean out the backups with a cron job like this:

# at 3 in the morning on Mondays, delete files older than 30 days
0 3 * * 1 find $HOME/.vim/backup/ -type f -mtime +30 -delete

expect tips

What to do when it's not sure you're going to make a connection?

set times 0
set made_connection 0
set timeout 120
while { $times < 2 && $made_connection == 0 } {
    spawn nc $SERVER
    send "\r"
    expect {
        "login:" {
            send "john.doe\r"
            set made_connection 1
        } eof {
            sleep 1s
            set times [ expr $times + 1 ]
        } timeout {
            puts "Didn't expect to timeout."

I think the following is wrong-headed. It's not usually the case that spawn will fail.

set times 0;
while { $times < 2 && $made_connection == 0 } {
    if { [ catch { spawn nc $SERVER } pid ] } {
        set times [ expr $times + 1 ];
        sleep 1s;
    } else {
        set made_connection 1

Perl tips

The module Search::Dict has a "look" function that can be used to do a binary search in an ordered dictionary file (a logfile (or log file) that starts with timestamps works). File::SortedSeek might also be recommended.

Additional Keywords

Linux, Unix, *nix

shell.1439483751.txt.gz · Last modified: 2015/08/13 09:35 by dblume
Recent changes RSS feed Driven by DokuWiki