Sunday, December 2, 2012

Thread safe programming

There are several ways in which a function can be thread safe.
It can be reentrant. This means that a function has no state, and does not touch any global or static variables, so it can be called from multiple threads simultaneously. The term comes from allowing one thread to enter the function while another thread is already inside it.
It can have a critical section. This term gets thrown around a lot, but frankly I prefer critical data. A critical section occurs any time your code touches data that is shared across multiple threads. So I prefer to put the focus on that critical data.
If you use a mutex properly, you can synchronize access to the critical data, properly protecting from thread unsafe modifications. Mutexes and Locks are very useful, but with great power comes great responsibility. You must not lock the same mutex twice within the same thread (that is a self-deadlock). You must be careful if you acquire more than one mutex, as it increases your risk for deadlock. You must consistently protect your data with mutexes.
If all of your functions are thread safe, and all of your shared data properly protected, your application should be thread safe.
As Crazy Eddie said, this is a huge subject. I recommend reading up on boost threads, and using them accordingly.
low-level caveat: compilers can reorder statements, which can break thread safety. With multiple cores, each core has its own cache, and you need to properly sync the caches to have thread safety. Also, even if the compiler doesn't reorder statements, the hardware might. So, full, guaranteed thread safety isn't actually possible today. You can get 99.99% of the way there though, and work is being done with compiler vendors and cpu makers to fix this lingering caveat.
Anyway, if you're looking for a checklist to make a class thread-safe:
  • Identify any data that is shared across threads (if you miss it, you can't protect it)
  • create a member boost::mutex m_mutex and use it whenever you try to access that shared member data (ideally the shared data is private to the class, so you can be more certain that you're protecting it properly).
  • clean up globals. Globals are bad anyways, and good luck trying to do anything thread-safe with globals.
  • Beware the static keyword. It's actually not thread safe. So if you're trying to do a singleton, it won't work right.
  • Beware the Double-Checked Lock Paradigm. Most people who use it get it wrong in some subtle ways, and it's prone to breakage by the low-level caveat.
That's an incomplete checklist. I'll add more if I think of it, but hopefully it's enough to get you started.

Thread safety is a computer programming concept applicable in the context of multi-threaded programs. A piece of code is thread-safe if it only manipulates shared data structures in a manner that guarantees safe execution by multiple threads at the same time. There are various strategies for making thread-safe data structures.[1][2]
A key challenge in multi-threaded programming, thread safety was not a concern for most application developers until the 1990s when operating systems began to expose multiple threads for code execution. Today, a program may execute code on several threads simultaneously in a sharedaddress space where each of those threads has access to virtually all of the memory of every other thread. Thread safety is a property that allows code to run in multi-threaded environments by re-establishing some of the correspondences between the actual flow of control and the text of the program, by means of synchronization.

Levels of thread safety

Software libraries can provide certain thread-safety guarantees. For example, concurrent reads might be guaranteed to be thread-safe, but concurrent writes might not be. Whether or not a program using such a library is thread-safe depends on whether it uses the library in a manner consistent with those guarantees.
Different vendors use slightly different terminology for thread-safety:[3][4][5][6]
  • Thread safe: Implementation is guaranteed to be free of race conditions when accessed by multiple threads simultaneously.
  • Conditionally safe: Different threads can access different objects simultaneously, and access to shared data is protected from race conditions.
  • Not thread safe: Code should not be accessed simultaneously by different threads.
Thread safety guarantees usually also include design steps to prevent or limit the risk of different forms of deadlocks, as well as optimizations to maximize concurrent performance. However, deadlock-free guarantees can not always be given, since deadlocks can be caused by callbacks and violation of architectural layering independent of the library itself.

[edit]Implementation approaches

There are a several approaches for avoiding race conditions to achieve thread safety. The first class of approaches focuses on avoiding shared state, and includes:
Writing code in such a way that it can be partially executed by a thread, reexecuted by the same thread or simultaneously executed by another thread and still correctly complete the original execution. This requires the saving of state information in variables local to each execution, usually on a stack, instead of in static or global variables or other non-local state. All non-local state must be accessed through atomic operations and the data-structures must also be reentrant.
Thread-local storage 
Variables are localized so that each thread has its own private copy. These variables retain their values across subroutine and other code boundaries, and are thread-safe since they are local to each thread, even though the code which accesses them might be executed simultaneously by another thread.
The second class of approaches are synchronization-related, and are used in situations where shared state cannot be avoided:
Mutual exclusion
Access to shared data is serialized using mechanisms that ensure only one thread reads or writes to the shared data at any time. Incorporation of mutal exclusion needs to be well thought out, since improper usage can lead to side-effects like deadlockslivelocks and resource starvation.
Atomic operations 
Shared data are accessed by using atomic operations which cannot be interrupted by other threads. This usually requires using special machine language instructions, which might be available in a runtime library. Since the operations are atomic, the shared data are always kept in a valid state, no matter how other threads access it. Atomic operations form the basis of many thread locking mechanisms, and are used to implement mutual exclusion primitives.
Immutable objects 
The state of an object cannot be changed after construction. This implies that only read-only data is shared and inherent thread safety. Mutable (non-const) operations can then be implemented in such a way that they create new objects instead of modifying existing ones. This approach is used by the string implementations in Java, C# and python.[7]


In the following piece of C code, the function is thread-safe, but not reentrant:
int increment_counter ()
        static int counter = 0;
        static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
        // only allow one thread to increment at a time
        // store value before any other threads increment it further
        int result = counter;   
        return result;
In the above, increment_counter can be called by different threads without any problem since a mutex is used to synchronize all access to the shared counter variable. But if the function is used in a reentrant interrupt handler and a second interrupt arises inside the function, the second routine will hang forever. As interrupt servicing can disable other interrupts, the whole system could suffer.
The same function can be implemented to be both thread-safe and reentrant using the lock-free atomics in C++11:
int increment_counter ()
        static std::atomic<int> counter(0);
        // increment is guaranteed to be done atomically
        int result = ++counter;
        return result;

Wednesday, November 28, 2012


    723 ifneq ($(HW), 21255)
    724 ifeq ($(HW), 21240)
    726 CFLAGS_64 += -D_REENTRANT
    727 CFLAGS_64 += -wd47
    728 CFLAGS_64 += -wd69
    729 CFLAGS_64 += -wd174
    730 CFLAGS_64 += -wd271
    731 CFLAGS_64 += -wd1572
    732 endif
    733 ifeq ($(HW), 21250)
    734 CFLAGS_64 += -wd47
    735 CFLAGS_64 += -wd101
    736 CFLAGS_64 += -wd174
    737 CFLAGS_64 += -wd271
    738 CFLAGS_64 += -wd1572
    739 endif
    740 CFLAGS_64 += -wd68
    741 CFLAGS_64 += -wd167
    742 CFLAGS_64 += -wd181
    743 CFLAGS_64 += -wd556
    744 ifeq ($(HW), 21240)
    745 CXXFLAGS_64 += -wd82
    746 CXXFLAGS_64 += -wd69
    747 CXXFLAGS_64 += -wd271
    748 CXXFLAGS_64 += -wd181
    749 endif
    750 ifeq ($(HW), 21250)
    751 CXXFLAGS_64 += -wd82
    752 CXXFLAGS_64 += -wd69
    753 endif
    754 CXXFLAGS_64 += -wd68
    755 CXXFLAGS_64 += -wd181
    756 CXXFLAGS_64 += -wd411
    757 else
    758 CFLAGS_64 += -Wno-unused
    759 CFLAGS_64 += -Wno-implicit
    760 CFLAGS_64 += -Wno-redundant-decls
    761 CFLAGS_64 += -Wno-sign-compare
    762 CFLAGS_64 += -Wno-format-extra-args
    763 CFLAGS_64 += -Wno-format
    764 CFLAGS_64 += -Wno-uninitialized
    765 CFLAGS_64 += -Wno-missing-field-initializers
    766 CFLAGS_64 += -Wno-conversion
    767 endif

Saturday, November 24, 2012

Mac OS X TIGER 10.4 Users (Darwin) Apps v

   ClamXAV v.2.2.1

   Safari v.4.1.3

   MacPorts 2.1.2

   Xcode: v.2.5
   GNU cflow v.1.0
   Exuberant Ctags 5.8
   cscope: version 15.8a
   powerpc-apple-darwin8-gcc-4.0.1 (GCC) 4.0.1
   powerpc-apple-darwin8-g++-4.0.1 (GCC) 4.0.1
   Splint 3.1.2 --- 24 Nov 2012

Friday, November 23, 2012

Redirect all output in shell script to a file


echo "this is line 1 of my script"
cat /proc/cpuinfo
echo "this is the end of my script"
) 2>> /tmp/filename.log

typeset log_fl=`basename $0`_` date +'%H%M%S'`.log
  # sending this script output to the log file
exec 1>${log_fl}

bash internal variables

Built-in Shell Variables 
Built-in variables areautomatically set bythe shell and aretypically used inside shell scripts. 
Built-in variables can make use of the variable substitution patterns shown previously. Note 
that the$is not actually partofthe variable name, although the variable is always referenced 
this way.The following areavailable in any Bourne-compatible shell: 
$# Number of command-line arguments. 
$- Options currently in effect (arguments supplied on command line or to 
set). The shell sets some options automatically. 
$? Exit value of last executed command. 
$$ Pr ocess number of current process. 
$! Pr ocess number of last background command. 
$0 First word; that is, the command name. This will havethe full pathname if 
it was found via a PATHsearch. 
$n Individual arguments on command line (positional parameters). The 
Bourne shell allows only nine parameters to be referenced directly (n=1–9); 
Bash allowsnto be greater than 9 if specified as${n}. 
$*,$@ All arguments on command line ($1 $2...). 
"$*" All arguments on command line as one string ("$1 $2..."). The values are 
separated bythe first character in IFS. 
"$@" All arguments on command line, individually quoted ("$1" "$2"...). 

Variable Substitution 
No spaces should be used in the following expressions. The colon (:) is optional; if it’s 
included,varmust be nonnull as well as set. 
var=value... Seteach variablevarto avalue. 
${var} Usevalue ofvar;braces areoptional ifvaris separated from the 
following text. They arerequired for array variables. 
${var:-value} Usevarif set; otherwise, usevalue. 
${var:=value} Usevarif set; otherwise, usevalueand assignvaluetovar. 
${var:?value} Usevarif set; otherwise, printvalueand exit (if not interactive). If 
valueisn’t supplied, print the phrase “parameter null or not set.” 
${var:+value} Usevalueifvaris set; otherwise, use nothing. 
${#var} Usethe length ofvar. 
${#*} Usethe number of positional parameters. 
${#@} Same as previous. 
${var#pattern} Usevalue ofvarafter removingpatter nfrom the left. Remove the 
shor testmatching piece. 
${var##pattern} Same as#patter n,but remove the longest matching piece. 
${var%pattern} Usevalue ofvarafter removingpatter nfrom the right. Remove 
the shortest matching piece. 
${var%%pattern} Same as%patter n,but remove the longest matching piece. 
${!prefix*},${!prefix@} List of variables whose names begin withprefix. 
${var:pos},${var:pos:len} Star tingat positionpos(0-based) in variablevar,extractlenchar- 
acters, or extract rest of string if nolen.posandlenmay be arith- 
metic expressions. 
${var/pat/repl} Usevalue ofvar,with first match ofpatreplaced withrepl. 
${var/pat} Usevalue ofvar,with first match ofpatdeleted. 
${var//pat/repl} Usevalue ofvar,with ever ymatch ofpatreplaced withrepl. 
${var/#pat/repl} Usevalue ofvar,with match ofpatreplaced withrepl.Match 
must occur at beginning of the value. 
${var/%pat/repl} Usevalue ofvar,with match ofpatreplaced withrepl.Match 
must occur at end of the value. 
Bash provides a special syntax that lets one variable indirectly reference another: 
$greet="hello, world" Create initial variable 
$friendly_message=greet Aliasing variable 
$echo ${!friendly_message} Usethe alias 
hello, world 


9.1. Internal Variables

Builtin variables:
variables affecting bash script behavior
The path to the Bash binary itself

bash$ echo $BASH
An environmental variable pointing to a Bash startup file to be read when a script is invoked
A variable indicating the subshell level. This is a new addition to Bash, version 3.
See Example 21-1 for usage.
Process ID of the current instance of Bash. This is not the same as the $$ variable, but it often gives the same result.

bash4$ echo $$

bash4$ echo $BASHPID

bash4$ ps ax | grep bash4
11015 pts/2    R      0:00 bash4

But ...


echo "\$\$ outside of subshell = $$"                              # 9602
echo "\$BASH_SUBSHELL  outside of subshell = $BASH_SUBSHELL"      # 0
echo "\$BASHPID outside of subshell = $BASHPID"                   # 9602


( echo "\$\$ inside of subshell = $$"                             # 9602
  echo "\$BASH_SUBSHELL inside of subshell = $BASH_SUBSHELL"      # 1
  echo "\$BASHPID inside of subshell = $BASHPID" )                # 9603
  # Note that $$ returns PID of parent process.
A 6-element array containing version information about the installed release of Bash. This is similar to $BASH_VERSION, below, but a bit more detailed.

# Bash version info:

for n in 0 1 2 3 4 5
  echo "BASH_VERSINFO[$n] = ${BASH_VERSINFO[$n]}"

# BASH_VERSINFO[0] = 3                      # Major version no.
# BASH_VERSINFO[1] = 00                     # Minor version no.
# BASH_VERSINFO[2] = 14                     # Patch level.
# BASH_VERSINFO[3] = 1                      # Build version.
# BASH_VERSINFO[4] = release                # Release status.
# BASH_VERSINFO[5] = i386-redhat-linux-gnu  # Architecture
                                            # (same as $MACHTYPE).
The version of Bash installed on the system

bash$ echo $BASH_VERSION

tcsh% echo $BASH_VERSION
BASH_VERSION: Undefined variable.

Checking $BASH_VERSION is a good method of determining which shell is running. $SHELL does not necessarily give the correct answer.
A colon-separated list of search paths available to the cd command, similar in function to the $PATH variable for binaries. The $CDPATH variable may be set in the local ~/.bashrc file.

bash$ cd bash-doc
bash: cd: bash-doc: No such file or directory

bash$ CDPATH=/usr/share/doc
bash$ cd bash-doc

bash$ echo $PWD
The top value in the directory stack [1] (affected by pushd and popd)
This builtin variable corresponds to the dirs command, however dirs shows the entire contents of the directory stack.
The default editor invoked by a script, usually vi or emacs.
"effective" user ID number
Identification number of whatever identity the current user has assumed, perhaps by means of su.

CautionThe $EUID is not necessarily the same as the $UID.
Name of the current function

xyz23 ()
  echo "$FUNCNAME now executing."  # xyz23 now executing.


                                   # Null value outside a function.

See also Example A-50.
A list of filename patterns to be excluded from matching in globbing.
Groups current user belongs to
This is a listing (array) of the group id numbers for current user, as recorded in /etc/passwd and /etc/group.

root# echo $GROUPS

root# echo ${GROUPS[1]}

root# echo ${GROUPS[5]}
Home directory of the user, usually /home/username (see Example 10-7)
The hostname command assigns the system host name at bootup in an init script. However, the gethostname() function sets the Bash internal variable $HOSTNAME. See also Example 10-7.
host type
Like $MACHTYPE, identifies the system hardware.

bash$ echo $HOSTTYPE
internal field separator
This variable determines how Bash recognizes fields, or word boundaries, when it interprets character strings.

$IFS defaults to whitespace (space, tab, and newline), but may be changed, for example, to parse a comma-separated data file. Note that $* uses the first character held in $IFS. SeeExample 5-1.

bash$ echo "$IFS"

(With $IFS set to default, a blank line displays.)

bash$ echo "$IFS" | cat -vte
(Show whitespace: here a single space, ^I [horizontal tab],
  and newline, and display "$" at end-of-line.)

bash$ bash -c 'set w x y z; IFS=":-;"; echo "$*"'
(Read commands from string and assign any arguments to pos params.)

Caution$IFS does not handle whitespace the same as it does other characters.
Example 9-1. $IFS and whitespace



# The plus sign will be interpreted as a separator.
echo $var1     # a b c
echo $var2     # d-e-f
echo $var3     # g,h,i


# The plus sign reverts to default interpretation.
# The minus sign will be interpreted as a separator.
echo $var1     # a+b+c
echo $var2     # d e f
echo $var3     # g,h,i


# The comma will be interpreted as a separator.
# The minus sign reverts to default interpretation.
echo $var1     # a+b+c
echo $var2     # d-e-f
echo $var3     # g h i


IFS=" "
# The space character will be interpreted as a separator.
# The comma reverts to default interpretation.
echo $var1     # a+b+c
echo $var2     # d-e-f
echo $var3     # g,h,i

# ======================================================== #

# However ...
# $IFS treats whitespace differently than other characters.

  for arg
    echo "[$arg]"
  done #  ^    ^   Embed within brackets, for your viewing pleasure.

echo; echo "IFS=\" \""
echo "-------"

IFS=" "
var=" a  b c   "
#    ^ ^^   ^^^
output_args_one_per_line $var  # output_args_one_per_line `echo " a  b c   "`
# [a]
# [b]
# [c]

echo; echo "IFS=:"
echo "-----"

var=":a::b:c:::"               # Same pattern as above,
#    ^ ^^   ^^^                #+ but substituting ":" for " "  ...
output_args_one_per_line $var
# []
# [a]
# []
# [b]
# [c]
# []
# []

# Note "empty" brackets.
# The same thing happens with the "FS" field separator in awk.


(Many thanks, Stéphane Chazelas, for clarification and above examples.)
See also Example 16-41Example 11-7, and Example 19-14 for instructive examples of using $IFS.
Ignore EOF: how many end-of-files (control-D) the shell will ignore before logging out.
Often set in the .bashrc or /etc/profile files, this variable controls collation order in filename expansion and pattern matching. If mishandled, LC_COLLATE can cause unexpected results in filename globbing.

NoteAs of version 2.05 of Bash, filename globbing no longer distinguishes between lowercase and uppercase letters in a character range between brackets. For example, ls [A-M]* would match both File1.txt and file1.txt. To revert to the customary behavior of bracket matching, set LC_COLLATE to C by an export LC_COLLATE=C in /etc/profile and/or ~/.bashrc.
This internal variable controls character interpretation in globbing and pattern matching.
This variable is the line number of the shell script in which this variable appears. It has significance only within the script in which it appears, and is chiefly useful for debugging purposes.

last_cmd_arg=$_  # Save it.

echo "At line number $LINENO, variable \"v1\" = $v1"
echo "Last command argument processed = $last_cmd_arg"
machine type
Identifies the system hardware.

bash$ echo $MACHTYPE
Old working directory ("OLD-Print-Working-Directory", previous directory you were in).
operating system type

bash$ echo $OSTYPE
Path to binaries, usually /usr/bin//usr/X11R6/bin//usr/local/bin, etc.
When given a command, the shell automatically does a hash table search on the directories listed in the path for the executable. The path is stored in the environmental variable$PATH, a list of directories, separated by colons. Normally, the system stores the $PATH definition in /etc/profile and/or ~/.bashrc (see Appendix G).

bash$ echo $PATH

PATH=${PATH}:/opt/bin appends the /opt/bin directory to the current path. In a script, it may be expedient to temporarily add a directory to the path in this way. When the script exits, this restores the original $PATH (a child process, such as a script, may not change the environment of the parent process, the shell).

NoteThe current "working directory"./, is usually omitted from the $PATH as a security measure.
Array variable holding exit status(es) of last executed foreground pipe.

bash$ echo $PIPESTATUS

bash$ ls -al | bogus_command
bash: bogus_command: command not found
bash$ echo ${PIPESTATUS[1]}

bash$ ls -al | bogus_command
bash: bogus_command: command not found
bash$ echo $?

The members of the $PIPESTATUS array hold the exit status of each respective command executed in a pipe. $PIPESTATUS[0] holds the exit status of the first command in the pipe,$PIPESTATUS[1] the exit status of the second command, and so on.

CautionThe $PIPESTATUS variable may contain an erroneous 0 value in a login shell (in releases prior to 3.0 of Bash).

tcsh% bash

bash$ who | grep nobody | sort
bash$ echo ${PIPESTATUS[*]}

The above lines contained in a script would produce the expected 0 1 0 output.
Thank you, Wayne Pollock for pointing this out and supplying the above example.

NoteThe $PIPESTATUS variable gives unexpected results in some contexts.

bash$ echo $BASH_VERSION

bash$ $ ls | bogus_command | wc
bash: bogus_command: command not found
 0       0       0

bash$ echo ${PIPESTATUS[@]}
141 127 0

Chet Ramey attributes the above output to the behavior of ls. If ls writes to a pipe whose output is not read, then SIGPIPE kills it, and its exit status is 141. Otherwise its exit status is 0, as expected. This likewise is the case for tr.

Note$PIPESTATUS is a "volatile" variable. It needs to be captured immediately after the pipe in question, before any other command intervenes.

bash$ $ ls | bogus_command | wc
bash: bogus_command: command not found
 0       0       0

bash$ echo ${PIPESTATUS[@]}
0 127 0

bash$ echo ${PIPESTATUS[@]}

NoteThe pipefail option may be useful in cases where $PIPESTATUS does not give the desired information.

The $PPID of a process is the process ID (pid) of its parent process. [2]
Compare this with the pidof command.
A variable holding a command to be executed just before the primary prompt, $PS1 is to be displayed.
This is the main prompt, seen at the command-line.
The secondary prompt, seen when additional input is expected. It displays as ">".
The tertiary prompt, displayed in a select loop (see Example 11-29).
The quartenary prompt, shown at the beginning of each line of output when invoking a script with the -x option. It displays as "+".
Working directory (directory you are in at the time)
This is the analog to the pwd builtin command.



clear # Clear the screen.


cd $TargetDirectory
echo "Deleting stale files in $TargetDirectory."

if [ "$PWD" != "$TargetDirectory" ]
then    # Keep from wiping out wrong directory by accident.
  echo "Wrong directory!"
  echo "In $PWD, rather than $TargetDirectory!"
  echo "Bailing out!"

rm -rf *
rm .[A-Za-z0-9]*    # Delete dotfiles.
# rm -f .[^.]* ..?*   to remove filenames beginning with multiple dots.
# (shopt -s dotglob; rm -f *)   will also work.
# Thanks, S.C. for pointing this out.

#  A filename (`basename`) may contain all characters in the 0 - 255 range,
#+ except "/".
#  Deleting files beginning with weird characters, such as -
#+ is left as an exercise. (Hint: rm ./-weirdname or rm -- -weirdname)

ls -al              # Any files left?
echo "Done."
echo "Old files deleted in $TargetDirectory."

# Various other operations here, as necessary.

exit $?
The default value when a variable is not supplied to read. Also applicable to select menus, but only supplies the item number of the variable chosen, not the value of the variable itself.


# REPLY is the default value for a 'read' command.

echo -n "What is your favorite vegetable? "

echo "Your favorite vegetable is $REPLY."
#  REPLY holds the value of last "read" if and only if
#+ no variable supplied.

echo -n "What is your favorite fruit? "
read fruit
echo "Your favorite fruit is $fruit."
echo "but..."
echo "Value of \$REPLY is still $REPLY."
#  $REPLY is still set to its previous value because
#+ the variable $fruit absorbed the new "read" value.


exit 0
The number of seconds the script has been running.



echo "Hit Control-C to exit before $TIME_LIMIT seconds."

while [ "$SECONDS" -le "$TIME_LIMIT" ]
  if [ "$SECONDS" -eq 1 ]

  echo "This script has been running $SECONDS $units."
  #  On a slow or overburdened machine, the script may skip a count
  #+ every once in a while.
  sleep $INTERVAL

echo -e "\a"  # Beep!

exit 0
The list of enabled shell options, a readonly variable.

bash$ echo $SHELLOPTS
Shell level, how deeply Bash is nested. [3] If, at the command-line, $SHLVL is 1, then in a script it will increment to 2.

NoteThis variable is not affected by subshells. Use $BASH_SUBSHELL when you need an indication of subshell nesting.
If the $TMOUT environmental variable is set to a non-zero value time, then the shell prompt will time out after $time seconds. This will cause a logout.
As of version 2.05b of Bash, it is now possible to use $TMOUT in a script in combination with read.

# Works in scripts for Bash, versions 2.05b and later.

TMOUT=3    # Prompt times out at three seconds.

echo "What is your favorite song?"
echo "Quickly now, you only have $TMOUT seconds to answer!"
read song

if [ -z "$song" ]
  song="(no answer)"
  # Default response.

echo "Your favorite song is $song."

There are other, more complex, ways of implementing timed input in a script. One alternative is to set up a timing loop to signal the script when it times out. This also requires a signal handling routine to trap (see Example 32-5) the interrupt generated by the timing loop (whew!).
Example 9-2. Timed Input


# TMOUT=3    Also works, as of newer versions of Bash.

TIMELIMIT=3  # Three seconds in this instance.
             # May be set to different value.

  if [ "$answer" = TIMEOUT ]
    echo $answer
  else       # Don't want to mix up the two instances. 
    echo "Your favorite veggie is $answer"
    kill $!  #  Kills no-longer-needed TimerOn function
             #+ running in background.
             #  $! is PID of last job running in background.


  sleep $TIMELIMIT && kill -s 14 $$ &
  # Waits 3 seconds, then sends sigalarm to script.


trap Int14Vector $TIMER_INTERRUPT
# Timer interrupt (14) subverted for our purposes.

echo "What is your favorite vegetable "
read answer

#  Admittedly, this is a kludgy implementation of timed input.
#  However, the "-t" option to "read" simplifies this task.
#  See the "" script.
#  However, what about timing not just single user input,
#+ but an entire script?

#  If you need something really elegant ...
#+ consider writing the application in C or C++,
#+ using appropriate library functions, such as 'alarm' and 'setitimer.'

exit 0

An alternative is using stty.
Example 9-3. Once more, timed input


#  Written by Stephane Chazelas,
#+ and modified by the document author.

INTERVAL=5                # timeout interval

timedout_read() {
  old_tty_settings=`stty -g`
  stty -icanon min 0 time ${timeout}0
  eval read $varname      # or just  read $varname
  stty "$old_tty_settings"
  # See man page for "stty."

echo; echo -n "What's your name? Quick! "
timedout_read $INTERVAL your_name

#  This may not work on every terminal type.
#  The maximum timeout depends on the terminal.
#+ (it is often 25.5 seconds).


if [ ! -z "$your_name" ]  # If name input before timeout ...
  echo "Your name is $your_name."
  echo "Timed out."


# The behavior of this script differs somewhat from ""
# At each keystroke, the counter resets.

exit 0
Perhaps the simplest method is using the -t option to read.
Example 9-4. Timed read

# [time-out]
# Inspired by a suggestion from "syngin seven" (thanks).

TIMELIMIT=4         # 4 seconds

read -t $TIMELIMIT variable <&1
#                           ^^^
#  In this instance, "<&1" is needed for Bash 1.x and 2.x,
#  but unnecessary for Bash 3+.


if [ -z "$variable" ]  # Is null?
  echo "Timed out, variable still unset."
  echo "variable = $variable"

exit 0
User ID number
Current user's user identification number, as recorded in /etc/passwd
This is the current user's real id, even if she has temporarily assumed another identity through su$UID is a readonly variable, not subject to change from the command line or within a script, and is the counterpart to the id builtin.
Example 9-5. Am I root?

#   Am I root or not?

ROOT_UID=0   # Root has $UID 0.

if [ "$UID" -eq "$ROOT_UID" ]  # Will the real "root" please stand up?
  echo "You are root."
  echo "You are just an ordinary user (but mom loves you just the same)."

exit 0

# ============================================================= #
# Code below will not execute, because the script already exited.

# An alternate method of getting to the root of matters:


username=`id -nu`              # Or...   username=`whoami`
if [ "$username" = "$ROOTUSER_NAME" ]
  echo "Rooty, toot, toot. You are root."
  echo "You are just a regular fella."
See also Example 2-3.

NoteThe variables $ENV$LOGNAME$MAIL$TERM$USER, and $USERNAME are not Bash builtins. These are, however, often set as environmental variables in one of the Bash startup files$SHELL, the name of the user's login shell, may be set from /etc/passwd or in an "init" script, and it is likewise not a Bash builtin.

tcsh% echo $LOGNAME
tcsh% echo $SHELL
tcsh% echo $TERM

bash$ echo $LOGNAME
bash$ echo $SHELL
bash$ echo $TERM

Positional Parameters
$0$1$2, etc.
Positional parameters, passed from command line to script, passed to a function, or set to a variable (see Example 4-5 and Example 15-16)
Number of command-line arguments [4] or positional parameters (see Example 36-2)
All of the positional parameters, seen as a single word

Note"$*" must be quoted.
Same as $*, but each parameter is a quoted string, that is, the parameters are passed on intact, without interpretation or expansion. This means, among other things, that each parameter in the argument list is seen as a separate word.

NoteOf course, "$@" should be quoted.
Example 9-6. arglist: Listing arguments with $* and $@

# Invoke this script with several arguments, such as "one two three".


if [ ! -n "$1" ]
  echo "Usage: `basename $0` argument1 argument2 etc."
  exit $E_BADARGS


index=1          # Initialize count.

echo "Listing args with \"\$*\":"
for arg in "$*"  # Doesn't work properly if "$*" isn't quoted.
  echo "Arg #$index = $arg"
  let "index+=1"
done             # $* sees all arguments as single word. 
echo "Entire arg list seen as single word."


index=1          # Reset count.
                 # What happens if you forget to do this?

echo "Listing args with \"\$@\":"
for arg in "$@"
  echo "Arg #$index = $arg"
  let "index+=1"
done             # $@ sees arguments as separate words. 
echo "Arg list seen as separate words."


index=1          # Reset count.

echo "Listing args with \$* (unquoted):"
for arg in $*
  echo "Arg #$index = $arg"
  let "index+=1"
done             # Unquoted $* sees arguments as separate words. 
echo "Arg list seen as separate words."

exit 0
Following a shift, the $@ holds the remaining command-line parameters, lacking the previous $1, which was lost.

# Invoke with ./scriptname 1 2 3 4 5

echo "$@"    # 1 2 3 4 5
echo "$@"    # 2 3 4 5
echo "$@"    # 3 4 5

# Each "shift" loses parameter $1.
# "$@" then contains the remaining parameters.

The $@ special parameter finds use as a tool for filtering input into shell scripts. The cat "$@" construction accepts input to a script either from stdin or from files given as parameters to the script. See Example 16-24 and Example 16-25.

CautionThe $* and $@ parameters sometimes display inconsistent and puzzling behavior, depending on the setting of $IFS.
Example 9-7. Inconsistent $* and $@ behavior


#  Erratic behavior of the "$*" and "$@" internal Bash variables,
#+ depending on whether they are quoted or not.
#  Inconsistent handling of word splitting and linefeeds.

set -- "First one" "second" "third:one" "" "Fifth: :one"
# Setting the script arguments, $1, $2, etc.


echo 'IFS unchanged, using "$*"'
for i in "$*"               # quoted
do echo "$((c+=1)): [$i]"   # This line remains the same in every instance.
                            # Echo args.
echo ---

echo 'IFS unchanged, using $*'
for i in $*                 # unquoted
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS unchanged, using "$@"'
for i in "$@"
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS unchanged, using $@'
for i in $@
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using "$*"'
for i in "$*"
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using $*'
for i in $*
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using "$var" (var=$*)'
for i in "$var"
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using $var (var=$*)'
for i in $var
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using $var (var="$*")'
for i in $var
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using "$var" (var="$*")'
for i in "$var"
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using "$@"'
for i in "$@"
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using $@'
for i in $@
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using $var (var=$@)'
for i in $var
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using "$var" (var=$@)'
for i in "$var"
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using "$var" (var="$@")'
for i in "$var"
do echo "$((c+=1)): [$i]"
echo ---

echo 'IFS=":", using $var (var="$@")'
for i in $var
do echo "$((c+=1)): [$i]"


# Try this script with ksh or zsh -y.

exit 0

# This example script by Stephane Chazelas,
# and slightly modified by the document author.

NoteThe $@ and $* parameters differ only when between double quotes.
Example 9-8. $* and $@ when $IFS is empty


#  If $IFS set, but empty,
#+ then "$*" and "$@" do not echo positional params as expected.

mecho ()       # Echo positional parameters.
echo "$1,$2,$3";

IFS=""         # Set, but empty.
set a b c      # Positional parameters.

mecho "$*"     # abc,,
#                   ^^
mecho $*       # a,b,c

mecho $@       # a,b,c
mecho "$@"     # a,b,c

#  The behavior of $* and $@ when $IFS is empty depends
#+ on which Bash or sh version being run.
#  It is therefore inadvisable to depend on this "feature" in a script.

# Thanks, Stephane Chazelas.


Other Special Parameters
Flags passed to script (using set). See Example 15-16.

CautionThis was originally a ksh construct adopted into Bash, and unfortunately it does not seem to work reliably in Bash scripts. One possible use for it is to have a script self-test whether it is interactive.
PID (process ID) of last job run in background


COMMAND1="sleep 100"

echo "Logging PIDs background commands for script: $0" >> "$LOG"
# So they can be monitored, and killed as necessary.
echo >> "$LOG"

# Logging commands.

echo -n "PID of \"$COMMAND1\":  " >> "$LOG"
echo $! >> "$LOG"
# PID of "sleep 100":  1506

# Thank you, Jacques Lederer, for suggesting this.

Using $! for job control:

possibly_hanging_job & { sleep ${TIMEOUT}; eval 'kill -9 $!' &> /dev/null; }
# Forces completion of an ill-behaved program.
# Useful, for example, in init scripts.

# Thank you, Sylvain Fourmanoit, for this creative use of the "!" variable.

Or, alternately:

# This example by Matthew Sage.
# Used with permission.

TIMEOUT=30   # Timeout value in seconds

possibly_hanging_job & {
        while ((count < TIMEOUT )); do
                eval '[ ! -d "/proc/$!" ] && ((count = TIMEOUT))'
                # /proc is where information about running processes is found.
                # "-d" tests whether it exists (whether directory exists).
                # So, we're waiting for the job in question to show up.
                sleep 1
        eval '[ -d "/proc/$!" ] && kill -15 $!'
        # If the hanging job is running, kill it.
Special variable set to final argument of previous command executed.
Example 9-9. Underscore variable


echo $_              #  /bin/bash
                     #  Just called /bin/bash to run the script.
                     #  Note that this will vary according to
                     #+ how the script is invoked.

du >/dev/null        #  So no output from command.
echo $_              #  du

ls -al >/dev/null    #  So no output from command.
echo $_              #  -al  (last argument)

echo $_              #  :
Exit status of a command, function, or the script itself (see Example 24-7)
Process ID (PID) of the script itself. [5] The $$ variable often finds use in scripts to construct "unique" temp file names (see Example 32-6Example 16-31, and Example 15-27). This is usually simpler than invoking mktemp.


[1]stack register is a set of consecutive memory locations, such that the values stored (pushed) are retrieved (popped) in reverse order. The last value stored is the first retrieved. This is sometimes called a LIFO (last-in-first-out) or pushdown stack.
[2]The PID of the currently running script is $$, of course.
[3]Somewhat analogous to recursion, in this context nesting refers to a pattern embedded within a larger pattern. One of the definitions of nest, according to the 1913 edition of Webster's Dictionary, illustrates this beautifully: "A collection of boxes, cases, or the like, of graduated size, each put within the one next larger."
[4]The words "argument" and "parameter" are often used interchangeably. In the context of this document, they have the same precise meaning: a variable passed to a script or function.
[5]Within a script, inside a subshell, $$ returns the PID of the script, not the subshell.