The Shell and Shell Commands

What is the Shell?

Whenever you login to a Unix system you are placed in a program called the shell. All of your work is done within the shell.
The shell is your interface to the operating system. It acts as a command interpreter; it takes each command and passes it to the operating system. It then displays the results of this operation on your screen.
There are several shells in widespread use. The most common ones are described below.

Bourne shell (sh): Original Unix shell written by Steve Bourne of Bell Labs. Available on all UNIX systems. Does not have the interactive facilites provided by modern shells such as the C shell and Korn shell. The Bourne shell does provide an easy to use language with which you can write shell scripts.
C shell (csh): Written at the University of California, Berkley. As it name indicates, it provides a C like language with which to write shell scripts.
Korn shell (ksh): Written by David Korn of bell labs. It is now provided as the standard shell on Unix systems. Provides all the features of the C and TC shells together with a shell programming language similar to that of the original Bourne shell.
TC Shell (tcsh): Available in the public domain. It provides all the features of the C shell together with EMACS style editing of the command line.
Bourne Again Shell (bash): Public domain shell written by the Free Software Foundation under their GNU initiative. Ultimately it is intended to be a full implementation of the IEEE Posix Shell and Tools specification. Widely used within the academic commnity. Provides all the interactive features of the C shell (csh) and the Korn shell (ksh). Its programming language is compatible with the Bourne shell (sh).

Your login shell is usually established by the local System Administrator when your userid is created. You can determine your login shell with the command:

     echo $SHELL

Each shell has a default prompt. For the 5 most common shells:

     

     $ (dollar sign)      - sh, ksh, bash 

     % (percent sign)     - csh, tcsh

Depending upon the shell, certain features will be available. The table below summarizes the features available with the 5 most common shells. Note: these features will be described in detail later.


                     Bourne  C    TC   Korn   BASH

                       sh   csh  tcsh   ksh   bash

______________________________________________________

Programming language   Yes  Yes   Yes   Yes   Yes

Shell variables        Yes  Yes   Yes   Yes   Yes

Command alias          No   Yes   Yes   Yes   Yes

Command history        No   Yes   Yes   Yes   Yes

Filename completion    No   Yes*  Yes   Yes*  Yes

Command line editing   No   No    Yes   Yes*  Yes

Job control            No   Yes   Yes   Yes   Yes

______________________________________________________



* not the default setting for this shell

Processes

Whenever you enter a command at the shell prompt, it invokes a program. While this program is running it is called a process. Your login shell is also a process, created for you upon logging in and existing until you logout.
UNIX is a multi-tasking operating system. Any user can have multiple processes running simultaneously, including multiple login sessions. As you do your work within the login shell, each command creates at least one new process while it executes.
Process id: every process in a UNIX system has a unique PID - process identifier.
ps - displays information about processes. Note that the ps command differs between different UNIX systems - see the local ps man page for details.

To see your current shell's processes:


% ps 



      PID TTY      S         TIME COMMAND

25830 ttyp1        S      0:29.55 nedit -server Shell.html

29618 ttyp1       IW +    0:00.12 more -svf

32077 ttyp1       IW      0:01.56 -csh (csh)

kill - use the kill command to send a signal to a process. In most cases, this will be a kill signal, hence the command name. However, other types of signals are usually supported. Note that you can only kill processes which you own. The command syntax is:

     kill [-signal] process_identifier(PID)


     kill  63878     - kills process 63878

     kill -9 1225    - kills (kills!) process 1225. Use if 

                       simple kill doesn't work.

     kill -STOP 2339 - stops process 2339

     kill -CONT 2339 - continues stopped process 2339

     kill -l         - list the supported kill signals

Suspend a process: Use CTRL-Z.
Background a process: Normally, commands operate in the foreground - you can not do additional work until the command completes. Backgrounding a command allows you to continue working at the shell prompt.

To start a job in the background, use an ampersand (&) when you invoke the command:

     myprog &

     

     myprog       - execute a process 

     CTRL-Z       - suspend the process 

     bg           - put suspended process in background

Foreground a process: To move a background job to the foreground, find its "job" number and then use the "fg" command. In this example, the jobs command shows that two processes are running in the background. The fg command is used to bring the second job (%2) to the foreground.

     

     jobs

     [1]  + Running      xcalc

     [2]    Running      find / -name core -print

     fg %2

Stop a job running in the background: Use the jobs command to find its job number, and then use the stop command. You can then bring it to the foreground or restart execution later.

     

     jobs

     [1]  + Running      xcalc

     [2]    Running      find / -name core -print

     stop %2

Kill a job running in the background, use the jobs command to find its job number, and then use the kill command. Note that you can also use the ps and kill commands to accomplish the same task.

     

     jobs

     [1]  + Running      xcalc

     [2]    Running      find / -name core -print

     kill %2

Some notes about background processes:

If a background job tries to read from the terminal, it will automatically be stopped by the shell. If this happens, you must put it in the foreground to supply the input.
The shell will warn you if you attempt to logout and jobs are still running in the background. You can then use the jobs command to review the list of jobs and act accordingly. Alternately, you can simply issue the logout command again and you will be permitted to exit.

Redirection

Redirection refers to changing the shell's normal method of handling standard output (stdout), standard input (stdin) and standard error (stderr) for processes. By default, all of these are from/to your screen.
The following symbols are used on the shell command line to redirect a process's stdin, stdout and/or stderr to another location, such as a file or device.

     

     >       - redirect stdout (overwrite)

     >>      - redirect stdout (append) 

     <       - redirect stdin 

     2>      - redirect stderr (sh,ksh,bash) 

     >&      - redirect stdout and stderr (csh,tcsh)

Examples:

     

     mail tony < memo           - uses the file memo as input 

                                  to the mail program

     ls -l > my.directory       - redirects output of ls -l 

                                  command to a file called 

                                  my.directory.  If the file

                                  already exists, it is 

                                  overwritten

     cat Mail/jsmith >> Oldmail - appends the contents of 

                                  Mail/jsmith to the file 

                                  Oldmail (does not overwrite)

     myprog >& output           - redirects stdout and stderr 

                                  from myprog's execution to

                                  a file called output

                                  (csh,tcsh)

     (myprog > out) >& err      - redirects stdout from myprog's

                                  execution to a file called out

                                  and stderr to the file err

                                  (csh,tcsh)

     myprog 2> runtime.errors   - redirects stderr from

                                  myprog's execution to a

                                  file called runtime.errors

                                  (sh,ksh,bash)

     myprog > output 2>&        - redirects stderr and stdout

                                  from myprog's execution 

                                  to a file called output

                                  (sh,ksh,bash)

     myprog > out 2> err        - redirects stdout from myprog's

                                  execution to a file called out

                                  and stderr to the file err

                                  (sh,ksh,bash)

Pipes

A pipe is used by the shell to connect the stdout of one command directly to the stdin of another command.
The symbol for a pipe is the vertical bar ( | ). The command syntax is:

 

     command1 [arguments]  |  command2 [arguments]

Pipes accomplish with one command what otherwise would take intermediate files and multiple commands. For example, operation 1 and operation 2 are equivalent:

 

Operation 1

     who > temp

     sort temp



Operation 2

     who | sort

Pipes do not affect the contents of the input files.
Two very common uses of a pipe are with the "more" and "grep" utilities. Some examples:

 

     ls -al | more

     who | more

     ps ug | grep myuserid

     who | grep  kelly

Filters

A filter is a command that processes an input stream of data to produce an output stream of data.
Command lines which use a filter will include a pipes to connect it to the stdout of one process and the stdin of another process.
For example, the command line below takes the output of "who" and sorts it. The sorted output is then passed to the lp command for printing. In this example, sort is a filter.

 

     who  |  sort  | lp

Both filters and pipes demonstrate a basic UNIX principle: Expect the output of every program to become the input of another, yet unknown, program to combine simple tools to perform complex tasks. Exercises

Features (csh)

Each shell has its own set of features. Those of the C Shell are discussed below.

Command history: The history mechanism maintains a list of recently used command lines, called events. Provides a shorthand for reexecuting previous commands. To use history:

Set the history variable:


     set history = 100

Issue the history command and view the output:


     history



     35  12:34   sort < unsorted > sorted.list

     36  12:34   cat sorted.list

     37  12:35   ls * > unsorted

     38  12:35   cat unsorted

     39  12:35   ls

     40  13:06   history

     41  13:08   alias

     42  13:11   ls

     43  13:11   vi .cshrc

To save history events across all login sessions, set the savehistory variable:


     set savehistory = 50

Event reexecution: Allows you to specify a shorthand for reexecuting a previous event. Works with the history list.


     !!          - repeats last command 

     !number     - repeats numbered command from history list

     !string     - repeats last command starting with string

Modifying previous events: Allows you to correct typos in previous command, or modify it to create a new command.


     ^old^new             - changes the string "old" to the 

                            string "new" in the last command  

                            issued 

     !number:s/old/new/   - changes numbered command from 

                            history list; substitutes the 

                            string "old" with the string "new".

                            Note that there is no space 

                            between number and :

Aliases: The alias command allows you to define new commands. Useful for creating shorthands for longer commands. The syntax is.


     alias  entered_command  executed_command


     alias  m     more

     alias  rm    "rm -i"

     alias  h     "history -r | more"

     alias  xpvm  /source/pd/xpvm/src/RS6K/xpvm

     alias

     unalias alias_name

Filename Generation: When you give the shell abbreviated filenames which contain special characters (metacharacters), it can generate filenames which match existing files. Some examples appear below:

     

     ls  *.txt    - list files with .txt suffix 

     ls [abc]*    - list files with names that start with a, 

                    b or c

     lpr prog?.c  - print files named prog?.c where ? is 

                    any character

     cd ~jsmith   - change to user jsmith's home directory

     set noglob

Filename Completion: The shell will complete a filename for you if you type in only enough characters to make the name unique. This feature is a great time saver and helps to prevent typos when trying to type long filenames.

To use filename completion, you need to set filec, either on the command line or in one of your initialization files.

     set filec

     

     cd Intro<ESC>

Variables (csh)

Each shell has its own set of variables and rules for using variables. Those associated with the C Shell are discussed below.

The shell has variables which are predefined as well as variables which you define (user defined).
Shell variables control many aspects of how your shell environment behaves. Modifying these variables (and creating new ones) allows you to customize your shell environment.
Shell variables are used extensively when creating shell scripts (covered later).
Variables can be

local - current shell only
global - current shell and child processes/shells
string - treated as character
numeric - treated as numbers
arrays - contain more than one value

Commands used to declare and manipulate shell variables:


     set            - assigns non-numeric string variables 

                      locally

     unset          - removes a previously "set" variable 

     set            - shows all "set" variables

     setenv         - assigns non-numeric string variables 

                      globally

     unsetenv       - removes a previously setenv variable

     setenv         - shows all setenv variables

     @              - assigns numeric variables locally

     echo $variable - displays value of variable

Examples


     set name=fred               - Sets variable name to 

                                   value of fred

     unset name                  - Unsets the variable name

     set path=($path . ~/bin)    - Adds to current setting 

                                   of string array variable 

                                   path

     setenv DISPLAY farragut:0     - Sets environment variable 

                                   DISPLAY

     echo $HOME                  - Displays value of variable 

                                   HOME

     set colors=(red green blue) - Assigns three values to 

                                   the string array variable 

                                   colors

     @ count = 1                 - Sets numeric variable 

                                   count to one

     @ count = ($count + 1)      - Adds one to numeric 

                                   variable count

     set counts = (1 22 4 9)     - Assigns 4 values to 

                                   numeric array variable 

                                   counts

     @ counts[2] = 5             - Assigns values to second

                                   element of numeric array

                                   variable counts

     @ echo $counts[3]           - Displays value of third

                                   element of numeric array

                                   variable counts

Predefined Shell Variables. A number of shell variables are predefined by the shell itself or inherited from the system environment. Some of the more common ones are described below.

$: Contains the process id of the current shell
argv: Contains the command line arguments for an invoked command. argv is an array, with $argv[0] set to the name of the invoked command, $argv[1] set to the first argument, $argv[2] set to the second argument...and so on. $argv[*] can be used to specify all arguments. You may also use the shorthand $n where n is the number of the argument.
#argv: Set to the actual number of arguments in argv, excluding argv[0]
cdpath: Expands the search path for the cd command. By default, the cd command issued with a simple filename will search only the working directory. Use cdpath to increase the number of directories searched.
CWD or cwd: Holds that name of the working/current directory.
echo: Causes the shell to echo the command before executing it. Use set/unset to turn this on/off.
filec: Enables file completion. Use set/unset to turn this on/off.
history: Controls the size of the history list. 100 is recommended as safe size. If the number is too large, the shell may run out of memory.
HOME or home: The pathname of your home directory
ignoreeof: Prevents exiting the shell by typing CTRL-D, and thus, prevents accidentally logging off. Use set/unset to turn this on/off.
noclobber: Prevents you from accidentally overwriting a file when you redirect output. Use set/unset to turn this on/off.
noglob: Prevents the shell from generating/expanding ambiguous filenames. Use set/unset to turn this on/off.
notify: Tells the shell to notify you immediately when a background job completes. Ordinarily, notification will wait until the next shell prompt to prevent interruption of work. Use set/unset to turn this on/off.
PATH or path: Specifies the path that the shell searches when asked to execute a command. If an executable is not found in the path, you must specify its full pathname.
prompt: Allows you to customize the shell prompt. By default, the C shell prompt is simply a percent sign (%). For example, to display the machine name you are logged into as part of your prompt:
savehist: Specifies how many the number of command events to save as history after you logout. These events are saved in a file called .history in your home directory. The shell uses these as your initial history after you login again.
shell: Contains the pathname of the shell
status: Contains the exit status of the last executed command
USER or user: Contains you login userid
verbose: Causes the shell to display each command after a history substitution. Use set/unset to turn this on/off.

Initialization Files

System-wide shell initialization files are common on UNIX systems. These files can be "sourced" automatically by the shell and are typically setup by the System Administrator for the local environment.
Some examples of system-wide initialization files might be:


     /etc/environment

     /etc/profile

     /etc/cshrc

     /etc/login

Every shell provides a means for the user to customize certain aspects of the shell's behavior. These customizations usually permit you to augment and/or override the system-wide defaults.
User customizations are specified in initialization files located in the top level of your home directory.
Naming: Depending upon the shell, you must name your initialization file(s) accordingly.


Executed during interactive login 

      .login        - csh, tcsh

      .profile      - sh, ksh, bash

      .bash_profile - bash (alternative 1)

      .bash_login   - bash (alternative 2)



Executed for every new shell 

      .cshrc        - csh, tcsh

      .tcshrc       - tcsh

      .kshrc        - ksh

      .bashrc       - bash

The C shell uses two files to set user preferences:

.cshrc

runs at each login before the .login file
runs whenever another shell is created
runs when a new window is opened
runs when many utilities are invoked
typically sets alias information, path variable, filec, prompt, etc.

.login

runs at invocation of login shell, after the .cshrc file
typically sets terminal characteristics and one time shell options and environment variables

After changing your .login or .cshrc files, you must "source" them for the changes to take effect:


     source .login

     source .cshrc

The system administrator may/may not provide you with default .cshrc and .login files when you first obtain your userid. If they are provided, be careful about modifying them - especially removing specifications which are required for your local system.
Example .cshrc and .login files are provided below:

Logout Files

You are able to specify commands which the shell will execute upon logout. These commands are kept in a file located in the top level of your home directory.


     .logout>        - csh, tcsh 

     .bash_logout    - bash