Parameter expansion

FIXME incomplete:

array expansion support (though they are special forms of existing syntax, I'd render it in extra sections, just for overview). Started an article here: Arrays

Introduction

One core functionality of Bash is to manage parameters. A parameter is an entity that stores values and is referenced by a name, a number or a special symbol.

parameters referenced by a name are called variables
parameters referenced by a number are called positional parameters and reflect the arguments given to a shell
parameters referenced by a special symbol are auto-set parameters that have different special meanings and uses

Parameter expansion is the procedure to get the value from the referenced entity, like expanding a variable to print its value. On expansion time you can do very nasty things with the parameter or its value. These things are described here.

If you saw some parameter expansion syntax somewhere, and need to check what it can be, try the overview section below!

For a more technical view what a parameter is and which types exist, see the dictionary entry for "parameter".

Overview

Looking for a specific syntax you saw, without knowing the name?

Simple usage
- $PARAMETER
- ${PARAMETER}
Indirection
- ${!PARAMETER}
Case modification
- ${PARAMETER^}
- ${PARAMETER^^}
- ${PARAMETER,}
- ${PARAMETER,,}
Variable name expansion
- ${!PREFIX*}
- ${!PREFIX@}
Substring removal (also for filename manipulation!)
- ${PARAMETER#PATTERN}
- ${PARAMETER##PATTERN}
- ${PARAMETER%PATTERN}
- ${PARAMETER%%PATTERN}
Search and replace
- ${PARAMETER/PATTERN/STRING}
- ${PARAMETER//PATTERN/STRING}
String length
- ${#PARAMETER}
Substring expansion
- ${PARAMETER:OFFSET}
- ${PARAMETER:OFFSET:LENGTH}
Use a default value
- ${PARAMETER:-WORD}
- ${PARAMETER-WORD}
Assign a default value
- ${PARAMETER:=WORD}
- ${PARAMETER=WORD}
Use an alternate value
- ${PARAMETER:+WORD}
- ${PARAMETER+WORD}

Simple usage

$PARAMETER

${PARAMETER}

The easiest form is to just use a parameter's name within braces. This is identical to using $FOO like you see it everywhere, but has the advantage that it can be immediately followed by characters that would be interpreted as part of the parameter name otherwise. Compare these two expressions (WORD=“car” for example), where we want to print a word with a trailing “s”:

echo "The plural of $WORD is most likely $WORDs"
echo "The plural of $WORD is most likely ${WORD}s"

Why does the first one fail? It prints nothing, because a parameter (variable) named ”WORDs” is undefined and thus printed as ”” (nothing). Without using braces for parameter expansion, Bash will interpret the sequence of all valid characters from the introducing ”$” up to the last valid character as name of the parameter. When using braces you just force Bash to only interpret the name inside your braces.

Also, please remember, that parameter names are (like nearly everything in UNIX®) case sentitive!

The second form with the curly braces is also needed to access positional parameters (arguments to a script) behind $9:

echo "Argument  1 is: $1"
echo "Argument 10 is: ${10}"

Indirection

${!PARAMETER}

Everywhere you can name a parameter to expand, like for example

${PARAMETER}

${PARAMETER:0:3}

you can use the form

${!PARAMETER}

which will enter a level of indirection: The referenced parameter is not PARAMETER itself, but the parameter named by the value of it. If your parameter PARAMETER has the value ”TEMP”, then ${!PARAMETER} really references TEMP:

read -p "Which variable do you want to inspect? " look_var

echo "The value of \"$look_var\" is: \"${!look_var}\""

Of course the indirection also works with special variables:

# set some fake positional parameters
set one two three four

# get the LAST argument ("#" stores the number of arguments, so "!#" will reference the LAST argument)
echo ${!#}

This is also known as “variable variables” or “indirect reference”.

Case modification

:V4:

${PARAMETER^}

${PARAMETER^^}

${PARAMETER,}

${PARAMETER,,}

These expansion operators modify the case of the letters in the expanded text.

The ^ operator modifies the first character to uppercase, the , operator to lowercase. When using the double-form (^^ and ,,), all characters are converted.

Example: Rename all *.txt filenames to lowercase

for file in *.txt; do
  mv "$file" "${file,,}"
done

Note: The feature worked word-wise in Bash 4 RC1 (a modification of a parameter containing hello world ended up in Hello World, not Hello world). In the final Bash 4 version it works on the whole parameter, regardless of something like “words”. IMHO a technically cleaner implementation. Thanks to Chet.

Variable name expansion

${!PREFIX*}

${!PREFIX@}

This expands to a list of all set variable names beginning with the string PREFIX. The elements of the list are separated by the first character in the IFS-variable (<space> by default).

This will show all defined variable names (not values!) beginning with “BASH”:

$ echo ${!BASH*}
BASH BASH_ARGC BASH_ARGV BASH_COMMAND BASH_LINENO BASH_SOURCE BASH_SUBSHELL BASH_VERSINFO BASH_VERSION

Substring removal

${PARAMETER#PATTERN}

${PARAMETER##PATTERN}

${PARAMETER%PATTERN}

${PARAMETER%%PATTERN}

This one can expand only a part of a parameter's value, given a pattern to describe what to remove from the string. The pattern is interpreted just like a pattern to describe a filename to match (globbing). See Pattern matching for more.

Example string (just a quote from a big man):

MYSTRING="Be liberal in what you accept, and conservative in what you send"

From the beginning

${PARAMETER#PATTERN} and ${PARAMETER##PATTERN}

This form is to remove the described pattern trying to match it from the beginning of the string. The operator ”#” will try to remove the shortest text matching the pattern, while ”##” tries to do it with the longest text matching. Look at the following examples to get the idea (matched text ~~marked striked~~, remember it will be removed!):

Syntax	Result
`${MYSTRING#* }`	Be liberal in what you accept, and conservative in what you send
`${MYSTRING##* }`	~~Be liberal in what you accept, and conservative in what you~~ send

From the end

${PARAMETER%PATTERN} and ${PARAMETER%%PATTERN}

In the second form everything will be the same, except that Bash now tries to match the pattern from the end of the string:

Syntax	Result
`${MYSTRING% *}`	Be liberal in what you accept, and conservative in what you ~~send~~
`${MYSTRING%% *}`	Be ~~liberal in what you accept, and conservative in what you send~~

Common use

How the heck does that help to make my life easier?

Well, maybe the most common use for it is to extract parts of a filename. Just look at the following list with examples:

Get basename
- ${FILENAME%.*}
- ⇒ bash_hackers.txt
Get extension
- ${FILENAME##*.}
- ⇒ bash_hackers.txt
Get directory name
- ${PATHNAME%/*}
- ⇒ /home/bash/bash_hackers.txt
Get filename
- ${PATHNAME##*/}
- ⇒ /home/bash/bash_hackers.txt

Search and replace

${PARAMETER/PATTERN/STRING}

${PARAMETER//PATTERN/STRING}

This one can substitute (replace) a substring matched by a pattern, on expansion time. The matched substring will be entirely removed and the given string will be inserted. Again some example string for the tests:

MYSTRING="Be liberal in what you accept, and conservative in what you send"

The two forms only differ in the number of slashes after the parameter name: ${PARAMETER/PATTERN/STRING} and ${PARAMETER//PATTERN/STRING}

The first one (one slash) is to only substitute the first occurrence of the given pattern, the second one (two slashes) is to substitute all occurrences of the pattern.

First, let's try to say “happy” instead of “conservative” in our example string:

${MYSTRING//conservative/happy}

⇒ Be liberal in what you accept, and conservativehappy in what you send

Since there is only one “conservative” in that example, it really doesn't matter which of the two forms we use.

Let's play with the word “in”, I don't know if it makes any sense, but let's substitute it with “by”.

First form: Substitute first occurrence

${MYSTRING/in/by}

⇒ Be liberal inby what you accept, and conservative in what you send

Second form: Substitute all occurrences

${MYSTRING//in/by}

⇒ Be liberal inby what you accept, and conservative inby what you send

Anchoring Additionally you can “anchor” an expression: A # (hashmark) will indicate that your expression is matched against the beginning portion of the string, a % (percent-sign) will do it for the end portion.

MYSTRING=xxxxxxxxxx
echo ${MYSTRING//#x/y}  # RESULT: yxxxxxxxxx
echo ${MYSTRING//%x/y}  # RESULT: xxxxxxxxxy

String length

${#PARAMETER}

When you use this form, the length of the parameter's value is expanded. Again, a quote from a big man, to have a test text:

MYSTRING="Be liberal in what you accept, and conservative in what you send"

Using echo ${#MYSTRING}...

⇒ 64

There's not much to say about it, mh?

Substring expansion

${PARAMETER:OFFSET}

${PARAMETER:OFFSET:LENGTH}

This one can expand only a part of a parameter's value, given a position to start and maybe a length. If LENGTH is omitted, the parameter will be expanded up to the end of the string.

OFFSET and LENGTH can be any arithmetic expression. Take care: The OFFSET starts at 0, not at 1!

Example string (a quote from a big man):

MYSTRING=“Be liberal in what you accept, and conservative in what you send”

In the first form, the expansion is used without a length value, note that the offset 0 is the first character:

echo ${MYSTRING:34}

⇒ Be liberal in what you accept, and conservative in what you send

In the second form we also give a length value:

echo ${MYSTRING:34:13}

⇒ Be liberal in what you accept, and conservative in what you send

If the given offset is negative, it's counted from the end of the string, i.e. an offset of -1 is the last character. In that case, the length still counts forward, of course. One special thing is to do when using a negative offset: You need to separate the (negative) number from the colon:

${MYSTRING: -10:5}
${MYSTRING:(-10):5}

Why? Because it's interpreted as the parameter expansion syntax to use a default value.

Note: This syntax can also be used to expand individual elements of an array.

Use a default value

${PARAMETER:-WORD}

${PARAMETER-WORD}

If the parameter PARAMETER is unset (never was defined) or null (empty), this one expands to WORD, otherwise it expands to the value of PARAMETER, as if it just was ${PARAMETER}. If you omit the : (colon), like shown in the second form, the default value is only used when the parameter was unset, not when it was empty.

echo "Your home directory is: ${HOME:-/home/$USER}."
echo "${HOME:-/home/$USER} will be used to store your personal data."

If HOME is unset or empty, everytime you want to print something useful, you need to put that parameter syntax in.

#!/bin/bash

read -p "Enter your gender (just press ENTER to not tell us): " GENDER
echo "Your gender is ${GENDER:-a secret}."

It will print “Your gender is a secret.” when you don't enter the gender. Note that the default value is used on expansion time, it is not assigned to the parameter.

Assign a default value

${PARAMETER:=WORD}

${PARAMETER=WORD}

This one works like the using default values, but the default text you give is not only expanded, but also assigned to the parameter, if it was unset or null. Equivalent to using a default value, when you omit the : (colon), as shown in the second form, the default value will only be assigned when the parameter was unset.

echo "Your home directory is: ${HOME:=/home/$USER}."
echo "$HOME will be used to store your personal data."

After the first expansion here (${HOME:=/home/$USER}), HOME is set and usable.

Let's change our code example from above:

#!/bin/bash

read -p "Enter your gender (just press ENTER to not tell us): " GENDER
echo "Your gender is ${GENDER:=a secret}."
echo "Ah, in case you forgot, your gender is really: $GENDER"

Use an alternate value

${PARAMETER:+WORD}

${PARAMETER+WORD}

This form expands to nothing if the parameter is unset or empty. If it is set, it does not expand to the parameter's value, but to some text you can specify:

echo "The Java application was installed and can be started.${JAVAPATH:+ NOTE: JAVAPATH seems to be set}"

The above code will simply add a warning if JAVAPATH is set (because it could influence the startup behaviour of that imaginary application).

Some more unrealistic example... Ask for some flags (for whatever reason), and then, if they were set, print a warning and also print the flags:

#!/bin/bash

read -p "If you want to use special flags, enter them now: " SPECIAL_FLAGS
echo "The installation of the application is finished${SPECIAL_FLAGS:+ (NOTE: there are special flags set: $SPECIAL_FLAGS)}."

If you omit the colon, as shown in the second form (${PARAMETER+WORD}), the alternate value will not be used when the parameter is empty, only if it is unset! You can use it, for example, to complain if variables you need (and that can be empty) are undefined:

# test that with the three stages:

# unset foo
# foo=""
# foo="something"

if [[ ${foo+isset} = isset ]]; then
  echo "foo is set..."
else
  echo "foo is not set..."
fi

Code examples

Substring removal

Removing the first 6 characters from a text string:

STRING="Hello world"

# only print it
echo "${STRING#?????}"

# store it
STRING=${STRING#?????}