Shell scripts

2.10. Shell scripts#

Although the shell looks like a place where you can just type in commands, itā€™s actually much more than that. Besides being a command line interface, itā€™s also a command language interpreter. Rather than the user typing commands one by one in the terminal, a shell process can also be used to read a sequence of commands from a simple text file and execute these in order (as if the user typed them) without intervention. Such files are called shell scripts, and they are very useful as they allow you to quickly automate things.

Linux systems often use shell scripts to manage its subsystems and ā€œglueā€ its different parts together, for example to configure and startup networking subsystems, update software packages, etc. But shell scripts are also a common tool for regular users, who want to simplify executing sequences of commands that they often rely on, for example to build some code and run some tests.

2.10.1. Creating a script#

A shell script is simply a plain text file with instructions that should be executed in a shell. Just as you would need to press Enter to execute a command in the command line interface, subsequent commands can be separated in the text file by placing them on new lines.

Using nano (see Section 2.5.3.1), create a new text file list containing just the following lines of text:

ls -l
echo LISTING DONE

You can also add comments to the file, since the shell ignores anything after # until the end of the line. So the following file content would functionally be the same:

# ----------------------
# This is a test script
# ----------------------

ls -l               # list the current directory
echo LISTING DONE   # inform user

Now there are several ways how we may want to execute this script:

  1. We might want to run this in an interactive shell, just as if we literally wrote the commands ourselves. This allows the script to affect the configuration of our current interactive shell. We call this sourcing the script.

  2. We might want to run it as a standalone process, which prevents the script from affecting the configuration of our current interactive shell. For this we have to execute the script.

We will explore both options below, which will also clarify what we mean with ā€œthe configurationā€ of the shell.

2.10.2. Source a script with source#

First, letā€™s take a look at executing the shell script in the current shell. This can be done with the source [FILENAME] instruction in bash. So, assuming your current working directory contains the list script, then you can simply run

$ source list

This should show you the output of ls -l and then the line LISTING DONE, just as if you would have typed these commands yourself in the terminal one by one.

Note

Sourcing can also be done with a special syntax in bash, namely by writing . instead of source as the command. So the source example above can also be written as:

$ . list

This notation can be confusing if you are not used to it, because it looks similar to other uses of . in the shell, such as representing the current directory (as in ls .) or as the beginning of a file name of a hidden file. In this manual we will prefer to use the more explicit source command, but it is good to be aware of this notation in case you encounter it in other documentation.

Since sourcing a file means all lines are executed in your current shell, without starting any new process, the script can access and set both regular and environment variables just as if you would have types the commands yourself. This makes sourcing suitable for scripts that are intended to setup one or more environment variables for later use.

Exercise 2.118

Make a new text file testvars, and use to print and set some regular and environment variables,

echo "Starting testvars"
echo "ENV_VAR     : $ENV_VAR"        # environment variable
echo "REGULAR_VAR : $REGULAR_VAR"    # non-environment variable

# update the variables
REGULAR_VAR="Updated $REGULAR_VAR"
export ENV_VAR="Updated $ENV_VAR"

Now run source testvars multiple times, what do you see in the output?

Afterwards, use echo and env to verify that sourcing the script has updated the variables in your current shell.

2.10.3. Running a script as a separate process#

Now leave the editor and enter in the terminal:

$ sh list

This tells the command line interpreter you are currently running (a bash process) to start another interpreter process (sh). By giving the sh interpreter the file name of the script list, this new process will open the file and execute its instructions.

If everything is correct, you should see in the terminal first the result of ls -l, displaying the content of the local working directory, and then the text LISTING DONE. Your script worked!

Once the script has no more instructions, the created sh process automatically terminates, and control is handed back to your interactive bash shell.

Exercise 2.119

Make a new script (give it a different filename, weā€™ll keep using the list script below), which when executed does the following:

  1. prints Hello world! to the terminal (Hint: use echo)

  2. prints You are here:

  3. prints the current working directory

Run the script to confirm that it works.

Exercise 2.120

What happens when we try to access regular and environment variables in a child process? Test this by executing the testvars script from Exercise 2.118 several times,

sh testvars

Also inspect the values of the variables with echo and env after executing the script again. Draw conclusions on the following:

  1. Can the exeuted script access all variables? Why (not)?

  2. Is the script able to update the variables in your shell? Why (not)?

Exercise 2.121

The sh shell used here is shell program like bash, but it is more basic. Although bash might not be installed on every Linux distribution, there should always be a basic shell called sh.

Letā€™s confirm that sh is a shell, by starting a new sh process in your interactive prompt

$ sh

You should see a new prompt, probably more basic and less colorful than the default bash prompt. Test that you can run simple commands in this shell just as in bash, e.g. ls, pwd, etc.

Next, run pstree to look at the process tree, and confirm that the sh process is a separate child process of the bash shell you were working in.

Once you are done, and want to stop the sh shell to return back to your bash shell, simply type

$ exit

which tells the current shell (sh) that it should quit. You should now see your normal bash prompt again.

Finally, confirm once more with pstree that the sh child process has terminated.

2.10.4. Making the script executable#

Executing a script can be done even easier, without the need to start a new shell process manually. Re-open the file list in nano and add #!/bin/sh as the very first line of the file. The file content should now look like this:

#!/bin/sh
ls -l
echo LISTING DONE

Save the changes to the list file and leave nano.

Next, we have to change the permissions on the list file such that the fileā€™s owner (you) can execute it. Remember you can use chmod to change file permissions:

$ chmod +x list

Once the file is marked as executable, you should be able to run the script directly, using ./list:

$ ./list

It should work just as before.

Exercise 2.122

Why do we need to mark the script as executable with chmod first?

Use chmod to remove the executable permission from the list script. Then, try to run the script again with ./list. What do you see in the terminal output?

Donā€™t forget to make the file executable again before continuing after this exercise.

So why did this work? When you try to execute a file, and bash notices that the file is a simple text file, it will look if the first line starts with #! (this character combination is called a hashbang or shebang). If so, if look take the remainder of that line and treat it as an interpreter to process the remainder of the file. In this case, bash will start the sh interpreter since /bin/sh is simply the full path to that interpreter program.

Note

The same mechanism can also be used turn scripts of other interpreted languages into executable files. For example, if our text file would contain Python code, you could add a first line #!/usr/bin/python3 (assuming Python 3 is installed in /usr/bin/python3) and mark the file as executable. Then, executing the python file in your shell would result in it automatically starting a Python process to run the code.

2.10.5. Handling optional arguments in scripts#

Ok, till now our list shell script provides a simple way to list files in the current working directory. But when we run the ls -l command in the terminal with an additional argument, like ls -l /usr/bin, then it would list the content of that directory (/usr/bin) instead of the current working directory. Letā€™s see if we can similarly give our script an optional argument to list a specific directory:

Exercise 2.123

Try listing the /usr/bin directory with your script:

$ ./list /usr/bin

Does it show the content of that directory? Why (not)?

To change this behaviour, we can use the arguments the user specifies within the shell script. These arguments are in the shell script available as variables called $1, $2, etc., where $1 is the first argument and so on. You can also get a list of all arguments using $*. Finally, the command name itself (here, list) is stored in $0.

Exercise 2.124

Re-write your shell script list to contain the following:

#!/bin/sh

ls -l $*
echo LISTING DONE

Now try again to use ./list to list the files in /usr/bin.

2.10.6. The PATH environment variable#

You probably noticed that weā€™ve been calling your shell script using ./list, not just list. This is necessary because the shell doesnā€™t automatically search the current working directory for commands: it is not in the path variable. The shell uses many variables to control its behaviour (which will be discussed in more detail in Section 2.9.1). The variable PATH contains a list of directories which the shell will search for programs (each directory is separated by a :). When you try to execute a file, your shell will search these directories in order to find the file name.

For example, if you would create your own ls script file and put it in a directory that occurs before /bin in the PATH variable, it will be used instead of the original system version of ls. Alternatively, you could update your PATH to include new directories, or to change their order.

Exercise 2.125

Look at your PATH shell variable using the env command (it can be helpful to search for PATH using a pipe to grep, see Section 2.7.2). How many directories are included in your PATH?

Exercise 2.126

Have a look at the .profile file in your home directory (this file is executed every time you start a shell). See how it automatically adds a $HOME/bin directory to your path if it exists (HOME is the variable that stores your home directory). Create a ~/bin directory and move your script there. Now make the shell reread the .profile file using source .profile, and try running list again.

Exercise 2.127

Now create a script cdl in ~/bin which:

  • changes the directory to the directory specified as an argument;

  • shows a list of files (including permissions) and pipes the output through less.

Exercise 2.128

A difficult question: why should .profile not contain a line #!/bin/sh ?

Exercise 2.129

Create an executable shell script ~/bin/aboutme that prints your username and home directory.

Contents