This page has been accessed times since May 14, 1996.

• Introductory Material
• Review of UNIX Concepts
• Advanced UNIX Commands

• Your local system administrator should be your first point of contact
• The EITS Helpdesk provides e-mail and phone support to UGA faculty, staff and students for general computing-related questions
• e-mail: helpdesk@listserv.uga.edu
• phone: 542-3106

• e-mail: wsg@listserv.uga.edu
• phone: 542-3106

• A shell is a program that reads commands and executes them.
• The common shells are the C shell (csh), the Bourne shell (sh) and the Korn shell (ksh).
• The shells differ in scripting syntax and a few other minor ways.
• You may be able to change your login shell using the chsh command.

• Shell variables are simply strings of characters that hold other values, for example $HOME may stand for /home/jones.
• Shell variables are indicated to the shell by a $ as the first character.
• To see all of your built-in shell variables, use the set or env commands.
• In general, shell variables that are special to the shell are written in uppercase, and user-assigned shell variables are written in lowercase:
• Example:

  $HOME

  $myvar

• To see the value of a specific shell variable, use the echo command:

  % echo $HOME

• Several environment variables are special to the shell. These include:

  $HOME

  your login directory

  $PATH

  directories that will be searched for commands

  $TERM

  your terminal type

  $DISPLAY

  local hostname for X-windows applications

  $MAIL

  location of your mailbox

• To set shell environment variables:

  csh:

  % setenv PATH /usr/bin:/bin

  % setenv PATH $PATH\:$HOME/bin

  sh or ksh:

  % PATH=/usr/bin:/bin

  % export PATH

  ksh

  % export PATH=$PATH\:/usr/bin:/bin

• To access the value stored in a shell variable, specify the variable including the "$".
• Example:

  % setenv prog /usr/local/apps/fft

  % $prog < $HOME/fft.input > /tmp/fft.out

• The special chracters for filename substitution are:

  *

  Matches zero or more characters

  ?

  Matches exactly one character

  [..]

  Matches a list or range of characters at one position. Some valid ranges are [a-z], [A-Z], and [0-9].

  {..}

  Substitution list -- left to right order is preserved. Matches are sorted at a low level to preserve this order.

  ~

  Home directory. Also specified by $HOME.

• Commands can be substituted in a similar manner to shell variables.
• Enclose a command in back-tics, (` `) to substitute it into another expression.
• Example:

  % setenv users `who | wc | awk '{print $1}'`

  % echo On `date` there were $users users on.

  On Mon May 6 11:37:16 EDT 1996 there were 27 users on.

• It is possible to redirect command input and output away from the keyboard and screen and from/to files:

  <

  Read standard input from file

  >

  Write standard output to new file

  >>

  Append standard output to file

  |

  Send standard output from the first command to the standard input of the next

• Examples:

  % ls > dir.list

  % ls $dir1 | more

• The general form is a tree.
• The top of the tree, commonly called the root, is denoted by a single forward slash (/). All files are located "below" the root directory.
• Filenames are specified in one of three forms:

  Absolute paths begin with a slash:

  % more /home/jones/adir/kermit.txt

  Relative paths (no slash at the beginning) start their search in the current directory:

  % more adir/kermit.txt

  Filenames with no paths are presumed to be in the current directory:

  % more kermit.txt

• UNIX is a multiuser, multitasking operating system (task = process).
• On a UNIX system, it is possible to run many programs and commands (processes) at the same time.
• Each user can have multiple processes running at the same time.

• The options to the ps command depend on the flavor of UNIX:
• System V systems (AIX, IRIX, Solaris, Linux, DEC Unix):

  % ps -ef

  all processes, full listing

  % ps -fuuser

  all processes associated with user, full listing

• BSD systems (SunOS, AIX, BSD/OS, DEC Unix):

  % ps aux

  all processes, full listing

  % ps u

  all processes associated with running user, full listing

• User can specify date and time to run a command or several commands in a file:

  % at 15:00 command_file

  % at now + 2 days command_file

  % at 3pm June 8 command_file

  % echo 'halt -q' | at now + 1 minute

• To list your jobs in the at queue:

  % at -l

• To remove the job jobid from the queue:

  % at -r jobid

• The cron susbystem is used to run recurring commands (daily backups, accounting, etc.)
• The list of commands to run is found in the user's crontab file, usually located in the file: /var/spool/cron/crontabs/userid
• Use the command crontab -e to create or modify the crontab file.
• crontab -l lists the contents of the crontab file.

• Comments begin with a # in the first column
• There are six fields in a crontab entry:

     Minute:		0-59
     Hour:		0-23
     Day of Month:	1-31
     Month:		1-12
     Day of Week:	0-6     (0 = Sunday)
     Command to run
  

• Possible wildcards are

  ,

  Comma: To separate multiple values

  -

  Dash: To specify a range

  *

  Asterisk: To specify all values in field

• Example:

  # run backups at 9:00pm Monday through Friday
  0 21 * * 1-5  /backup_files 2> /dev/null
  # run a program every 10 minutes
  0,10,20,30,40,50 * * * * /home/jsmith/bin/send_data
  # run a program at 7:21am on the 1st day of every month
  21 7 1 * *  /home/jsmith/bin/monthly_program
  

• CPU time on a UNIX computer is given to processes based on their priority.
• The nice command is used to run a command at a lower priority:

  % nice mycommand &

  % nice -15 mycommand &

• The root user can raise the priority of a process

  % nice --10 wall <<end
  System shutdown in 2 minutes\!
  end
  

• In some shells, all background processes are terminated when you logout.
• To protect a process from hangups, run it with the nohup command:
• Example:

  % nohup cc -c *.c &

• By default, both standard output and standard error are redirected to the file nohup.out.

jobs

list your background jobs

kill pid

kill the specified process

kill %jobid

kill a background job

^z

suspend the current interactive job

bg

put a suspended job in the background

fg %jobid

bring a job to the foreground

batch

run a command when the system load permits. The syntax is similar to that of the at command.

• There are other queuing systems which may be installed on your system.
• Common batch systems are: LoadLeveler, NQS and DQS.
• Contact your local system adminstrator for information on queues as well as CPU limits for non-batch processes.

• UNIX command philosophy
• Most commands perform only simple tasks. This leads to small and efficient commands.
• Several commands are used together to perform more complex tasks.
• The following commands, used alone or with other commands, are very useful.

• awk matches patterns in files and performs operations on them.
• awk is also a programming language with a syntax similar to C.
• It is commonly used as a filter for many problems.
• The awk utility is covered in detail in the UNIX Utilities II: awk training course.
• Examples

  Delete the second field of every line in the file a.dat

  % awk '{$2= ""; print}' a.dat > a.dat.new

  Print the second field of every line in the file a.dat

  % awk '{print $2}' a.dat | more

  Print out all lines in the file a.dat that are longer than 72 characters:

  % awk 'length > 72' a.dat

  Kill all processes belonging to user jdoe

  % ps aux | awk '$1=="jdoe" {print "kill " $2}' | sh

• Programs written in awk can be quite complex.
• Many numerical and string functions are available, as are variables and multi-dimensional arrays.
• To execute the awk program, program.awk:

  % awk -f program.awk input_file

• A sample awk program follows

{
  x1 += $1
  x2 += $1*$1
}
END {
  x1 = x1/NR
  x2 = x2/NR
  sigma = sqrt(x2 - x1*x1)
  if (NR > 1) std_err = sigma/sqrt(NR - 1)
  print "Number of points = " NR
  print "Mean = " x1
  print "Standard Deviation = " sigma
  print "Standard Error = " std_err
}

• The cut command is used to write out selected characters or fields from each line of a file.
• Examples:

  Print the second field of every line in the file a.dat

  % cut -f2 -d\ a.dat | more

  Print the first 5 charcters of every line in the file a.dat

  % cut -c1-5 a.dat | more

• diff reports differences between two files.
• Output lines are flagged with < for the first file and > for the second file.
• The output of diff is the list of ed commands needed to change the second file to the first.
• Example:

  % diff file1 file2

  1c1

  < this is file1

  ---

  > this is file2

• The find command locates specified files and lets you perform a number of actions on (or to) them.
• find will search recursively down the directory tree starting at the directory specified on the command line.
• find uses words for options, not single letters.
• Some of the more common options are:

  -name filename

  look for the specified filename

  -print

  print the filenames found

  -exec cmd

  run the command cmd

  -newer filename

  find files newer than the file specified by filename

• Examples:

  % find . -name slides.txt -print

  % find / -name \*.c -print

  % find / -name \*.old -exec rm -i {} \;

• grep searches a file (or a group of files) for a specified regular expression
• case matters in the matching unless the -i option is specified
• The -v option prints all lines not containing the pattern
• Examples:

  % grep Smith *.letter

  % grep -i smith *.letter

  % ps aux | grep -v root

• It is often advantageous to assign multiple names to a single file.
• The ln command links two files or directories. Either name can be used to access the file.
• Examples:

  create fort.1 which points to input.file

  % ln input.file fort.1

  link the directory /usr/local/lib/X11 to /usr/lib/X11

  % ln -s /usr/local/lib/X11 /usr/lib/X11

• The | character is used to pipe the standard output of a program to the standard input of another.
• mkfifo is used to create a named pipe -- the output of a program can be directed to a file, which another program can open and read.
• On some systems the command is: mknod p
• Example:

  % mkfifo fort.11

  % zcat big_file.Z >> fort.11 &

  % mknod fort.11 p

  % zcat big_file.Z >> fort.11 &

• The sort utility sorts files based on user-defined keys.
• If multiple files are specified, they are first merged, then sorted as a single file.
• Common options:

  -r

  sort in reverse order

  -b

  ignore leading spaces

  +f.c

  specify start of sort key (field.column)

  -f.c

  specify end of sort key (field.column)

  -n

  sort numerically

• The stty command is used to set or change the terminal/line parameters for a UNIX session.
• There are over 40 parameters you may change.
• stty -a will show the current value of all terminal settings.
• The sequence ^q^jstty sane^j can often be used to return control of your terminal to you after a problem.
• Consult the stty man page for system-dependent options.

• Nearly all UNIX systems have a compiler for the C programming language (cc) and many also have a FORTRAN compiler (f77).
• Common options:

  -O

  optimize the program

  -c

  compile but do not link

  -g

  produce debugging information

• There are usually many system-dependent options for compilers. Consult the cc and f77 man pages for more information.

• The make utility maintains up-to-date versions of programs.
• The user defines dependencies for a target program and rules to update the program based on those dependencies.
• If none of the dependencies have changed since the last program update, the target is not rebuilt.

# Fortran compiler options
FFLAGS = -O3
# Object files needed for test.x
OBJS = main.o sub.o

# Link to get the executable file test.x

test.x: $(OBJS)
        f77 $(FFLAGS) $(OBJS) -o test.x

clean:
        rm -f $(OBJS) test.x

• A shell script is a collection of UNIX commands which are executed together.
• Similar to .BAT files in DOS and .COM files in VMS.
• Shell scripts are the primary tool for combining the specialized UNIX commands to perform more complex and useful tasks.
• Shell scripts also allow users to make use of the simple programming languages built into the different shells.

#!/bin/csh
#
# Check for all processes owned by the specified
# user ($1) on a set of machines
#
foreach comp ( sa sb sc sd se sf sg sh si sj sk )
  echo $comp
  rsh $comp ps aux | grep $1
  echo "  "
end

#!/bin/ksh
#
# Check for all processes owned by the specified
# user ($1) on a set of machines
#
for comp in sa sb sc sd se sf sg sh si sj sk
  do
    echo $comp
    rsh $comp ps aux | grep $1
    echo "  "
done

#!/bin/ksh
llq | tail +3 | awk '{
if (($2 != "") && ($2 != "jobs")) users[$2]++}
END {
  for (i in users) {
    printf("%8s  %d\n",i,users[i]);
    x += users[i];
  }
  print
  print "Total number of jobs = "  x
}' | sort +1nr

#!/bin/ksh
# Automatically submit 50 jobs when 3 or less in queue
i=1
while test $i -le 50
 do
  nj=`llq -u joe | tail +3 | wc | awk '{print $1-2}'`
  if test $nj -le 3
   then
     echo Submitted job number $i at `date`
     ugsub medium myprog.ksh $i
     i=`expr $i + 1`
   fi
  sleep 300
 done