In the last article, you were introduced to the UNIX / Linux Bash Shell Command. You learnt why the shell is important and why a system administrator should have a mastering knowledge of Linux shell. In this article, we are going to build upon the basics we learnt in the previous one. A lot of Bash Shell Command features are waiting for us. Are you ready? Let’s go!!

If you haven’t read the previous article, I suggest that you refer back to it before starting to read this one:

Using Linux Bash Shell Command

Bash Shell Command Auto-Completion

The most interesting and useful feature in the Bash Shell Command is the Command auto-completion. Don’t wonder if you become a system administrator for a UNIX system that doesn’t come with Bash Shell Command pre-installed, and other users / administrators start asking you: is it possible to have Bash Shell Command on this server?

I bet they will be missing this feature particularly!!

The command auto-completion feature in Bash Shell Command helps the administrator to quickly type commands without having to remember the full spelling of the command (that is a great help if you know!!)

Let’s see how this feature work: if you remember the commands we used in the previous topic. One of them gets the machine name (hostname) of the Linux box. Was it host, host-name, host_name, or what?

I suggest that we go to the Linux Bash Shell Command, and try to guess and remember. In the shell, I typed ho and pressed the Tab key. Surprisingly, the shell made something strange: it completed the word host

Now, let’s press Tab key twice.

Now, the shell presents the possible commands that start with host part. They are: hostid , hostname , and hostnamectl.

Obviously, the one we need is hostname. Now, type an additional n letter, then press Tab:

The shell completed the command for you to be hostname. Now, press Enter to execute it:

It worked. You may find it silly and say: all of this just for typing such an easy to remember command hostname ?

The answer is simple: Consider a command like systemd-tty-ask-password-agent , that consists of 31 characters: could you remember it easily? Got the idea?! Great!!

File and Directory names Auto-Completion

The same feature is also available when typing directories (paths) and files’ names as arguments to Linux commands. Consider we need to print the operating system release of our Linux machine. In the Linux shell, type cat /e and press Tab. The shell completes the path to /etc/ . Now, type red

Press the Tab key.

The shell completed the path to /etc/redhat-release . Note that the cursor has been moved one space away from the word (path) just auto-completed. The shell wants to tell you that the pathname (directory or file name) is now complete, and you can type an option, another argument, or just press Enter. Pressing Enter will execute the command:

The operating system type and its release have been printed on the screen.

The cat command prints the content of one or more text files on the standard output (screen).

Bash Shell Command Expansion

The Shell Expansion is a feature of the Linux Shell in which a series of conversion and substitution operations take place when the user types a command.

The shell interpreter takes the command line through a journey of seven steps:

Step #1: Curly Braces Expansion.

Step #2: Tilde Expansion: if the Tilde symbol ~ appears in the command line, it is replaced by the full path of the home directory of the user. For example, ~student will be substituted by /home/student.

Step #3: Variables Expansion. In this step, shell variables that are preceded by dollar sign $ are substituted by their values. For example, if the command was: echo $LANG , the value of the variable LANG (which is en_US.UTF-8) is expanded to replace the $LANG. So, the command in its final form will be:

echo  en_US.UTF-8

And this is what will be printed to the screen.

Step #4: Command Substitution.

Step #5: Arithmetic Expansion.

Step #6: Word Splitting.

Step #7: Wildcard Expansion.

I mentioned two of the seven steps (Tilde and Variable Expansions) with some detail and examples, because we have already introduced them in the previous article. The remaining five steps are going to be discussed now.

Curly Braces Expansion

The curly braces are used in Linux shell to generate sequences of characters or lists of words. This can be very useful in shell scripting to define a loop that iterates over a range of numbers or characters. Also, it is useful when there is a need to create large number of directories or files.

Assume you need to create folders for the months of the year whose names are 01, 02, 03, … , 12

You can do it simply in one step using the mkdir command (that creates new directories) with the braces expansion:

We used the ls command to list the contents of the current directory, where we created the new directories (01 to 12). The output shows that we have successfully created the twelve directories with the names we want and in one step.

So, back to the syntax we used:

{01..12}

When encountered by the shell interpreter, the shell expands this structure into its equivalent result:

01 02 03 04 05 06 07 08 09 10 11 12

So, if you need to generate the numeric sequence from numX to numY with an increment of one, use the syntax {numX..numY}

Now, let’s try something interesting!! How to print even numbers between 0 and 200?

Have you noticed the difference? The same previous syntax but added an increment (step) at the end. So, the syntax becomes:

{numX..numY..INCREMENT}

Again, you may find it silly and ask: Why do I have to care for such strange curly braces expansion? To create twelve directories for the months of year? It is not difficult to type the same command twelve times like that:

mkdir 01

mkdir 02

mkdir 03

..

..

mkdir 12

Fine, again consider you need to create an individual directory for each day of the year, named: 000, 001, 002, …, 364, and 365.

Will you run the mkdir command 365 times?!!

Of course, you could use a loop that will give you the solution in four lines of code (will discuss Loops in much detail in the Shell Scripting Series), but why don’t you give the following solution a try?!

So fast and simple like magic!! Isn’t it?!

Now, consider we need to print multiples of 10 from 100 to 0, in descending order. Will the same syntax apply?!

Look at this:

It worked in the opposite direction!!

Another possible use of the curly braces is to generate sequences of letters. Look at the following:

Here we have printed all the set of uppercase alphabet letters. We could also print just a subset of the set of letters:

And also lowercase letters:

And in descending order as well:

A very interesting feature of the curly braces expansion is the ability to multiply two sets together. Check the following simple structure that will generate the sequence: Aa Ab Ac …. Az Ba Bb Bc … Bz … … Za Zb Zc … Zz.

Command Substitution

In Command Substitution, if the shell encounters one or more commands “enclosed” as argument to another command, it executes the enclosed command(s) and substitutes its(their) output to be part of the argument to outer (main) command.

Consider we need to run a backup whose log will be written to certain specified log file, whose name should be named after the machine name. We know already how to get the machine name using the command hostname. Of course it is possible to just write the machine name directly in the backup command. But what if you need to make this command generic so that it could work on any of your Linux servers in the future?

Another case to consider: Assume you are writing a menu script with options that the user should choose from. When the user enters an option, the user’s choice and action should be audited (logged) with date and timestamp of doing the action to a log file.

To create a file for the first case (containing the machine’s name), run the command:

touch `hostname`_backup.log

As you see, a new file was created with name rhel9-srv_backup.log.

The touch command is one of the possible ways to create files under UNIX and Linux. When the command touch `hostname`_backup.log was seen by the shell, the shell does its checklist:

1. Curly expansion: Here, none.
2. Tilde expansion: No Tilde’s here.
3. Variable substitution: No variables used in the above command.
4. Command substitution: If there is something enclosed by backquotes ` `  or $( ) , the shell executes the enclosed command (hostname in this example), and substitutes its output (rhel9-srv) to be an argument to the main command (in our case the command touch).

Now, let’s see how to implement the other scenario (writing an event with date and timestamp to a log file). To append something to a file, the >> operator is used. (The use of this operator will be discussed in detail in a separate article when talking about input and output redirection).

To display the system date and time, use the command date.

So, to write a message to the audit trail log file, that user ahmed has entered option b and pressed Enter:

Let’s write another message to the same audit trail:

The new message has been appended to the audit trail file. Note that the backquotes ` ` and $( ) have the same meaning and effect. So, they are different methods to enclose a command inside another command.

Arithmetic Expansion

If the shell encounters an arithmetic operation inside the command line, then the arithmetic operation is calculated first, and its result is expanded (substituted) in its place.

The simplest form of the arithmetic calculations in Linux shell is using the following syntax:

$((arithmetic_expression))

Where the arithmetic operation could be + (addition), – (subtraction), * multiplication, / (division), % (remainder), ** (power), and many others.

For example, in the following command line:

echo 3 raised to the power 4 equals = $((3**4))

The mathematical operation 3**4 will be calculated first. Its result (81) will be inserted in the line to replace the part $((3**4)). As a result, the command now becomes:

echo 3 raised to the power 4 equals = 81

That will be printed to the standard output (screen) as follows:

Word Splitting

When analyzing a command entered in the Linux shell, the shell scans each character in the command. If it encounters a character that is considered a separator (space, tab, or new line), it splits the line into words. If there are adjacent spaces, tabs, or new lines, they are treated as one, and the duplicated separator characters are removed. This remains true if that part (containing multiple spaces, tabs or new lines) is not surrounded by quotes ” “. In this case, the duplicates are not removed.

Consider the following two commands:

<p>echo $USER                            $SHELL</p>

<p>echo "$USER                           $SHELL"</p>

What is the difference between both? Let’s see them when executed and see if there would be differences.

Have you noticed the difference?!

In the first command, the multiple spaces were treated as if they was one. The duplication was suppressed (removed), and the shell printed the values of $USER and $SHELL separated by only one space.

In the second command, as the argument of the echo command was surrounded by double quotes ” ” , the shell expanded (substituted) the values of the variables but treated the spaces literally. So, the multiple spaces were printed as they are.

16 Must-Know Bash Commands for Data Scientists

In the next article, we will continue with the Linux Shell. We will discuss the Pathname (Wildcard) Expansion and the shell history.

I hope you enjoyed reading and learning with us. See you in the next article.