Overview of Class One

by Torleif Mohling
CU Boulder, dept of Computer Science
(01/19/2001)

0 Contents

1 Intro and Administrivia

Please see: ./intro for the administrative details. Note well the mailing-list signup instructions. Please read the following manual pages (preferably on an OpenBSD machine, but any BSD or Linux box should be fine):

We also talked about the vi text editor. It is imperative that you learn this editor if you don't already know it. It is very useful for system administration and you need to know it for situations where no other reasonable editor is available (like when a broken machine is in single-user mode.) There is a great VI cribsheet I found somewhere on the 'net. You can get it at my FTP site, see: ftp://ftp.cs.colorado.edu/users/tor/saclass/vi-reference.ps

There are a number of tutorials for VI available on the web. Do a search on vi and tutorial and you'll be amazed!

2 A brief history of Unix

I'm not going to reiterate here what was said in class. If you are really interested in more details about Unix' lurid history, there are some online versions:

The most important thing to understand about this history is the major split between Unix systems derived from System-V (i.e. AT&T) and those derived from BSD. We will be exploring some of these differences throughout the course.

3 Logging into a Un*x box

In the old days, one encountered a logon prompt like:

    OpenBSD/i386 (quetzel) (ttyp3)

login:

Notice the ttyp3 - this is the name of the device you are logging in on. These days, devices like this are usually pseudo devices (I was using telnet to localhost on my laptop in an xterm window so I could cut+paste the prompt into this document); however, it easy to set up an old VT100 as an interface to your PC/Un*x box using one of the serial ports... if you can find a working VT100!

It is much more likely however, that you will encounter one of several graphical logon windows. A common one used in the student labs is XDM . XDM presents you with a similar logon, giving (usually) the name of the machine and two entry fields for a login and a password .

In either case, logging into the machine is generally straight forward: simply type in your user name and password (hitting <ENTER> after each of course :) and there you are. However, what happens if you can't log in and, "I know my password is right"? If you are using XDM, there is a good chance that there is an error in one or more of your dot files . Other problems besides the obvious username or password being wrong, are mostly related to network issues: the user's homedir is not being exported or mounted correctly being the most common symptom. We will discuss such network problems later in the semester.

3.1 Dot Files

The term dot files refers to files whose names start with a '.'. You might have noticed that such files don't appear by default when you use the ls command; to get them to appear you need to use the -a flag to ls. For example:

    xibalba % ls 
    Desktop/   foo.1    hw1.ps   nsmail/    pix-letter.wp    working/
    bookmarks  history  ks.tmpl  olmec-bib  pixGCletter1.wp
    xibalba % ls -a
    ./             .enlightenment/       .netscape/        history
    ../            .gnome/               .screenrc         hw1.ps
    .ICEauthority  .gnome-desktop/       .ssh/             ks.tmpl
    .Xauthority    .gnome-help-browser/  .wprc/            nsmail/
    .Xdefaults     .gnome_private/       .xsession-errors  olmec-bib
    .alias         .kde/                 Desktop/          pix-letter.wp
    .cshrc         .kderc                bookmarks         pixGCletter1.wp
    .e-conf/       .mc/                  foo.1             working/
    xibalba % 
Let's look at some particular dot files (not necessarily in the above ls output.)

There are lots of different dot files associated with various commands. Sometimes information about the syntax of those files is given in the manpage for the command itself. Other times, the file will have its very own manpage.

Take a look at the default files you got with your CSEL lab account. Some of the basic files, like .login and .cshrc and .xinitrc have lots of nice comments. Can you think of a reason why you might not want to define lots of user environment variables and shell aliases in your .cshrc file and instead move them to the .login file? One reason is speed of startup, not that this matters so much anymore; but if you are firing up a tcsh command of some kind, you don't need all the user environment setup stuff, so why bother processing it.

3.2 Using the Failsafe Logon Mode

Sometimes, after changing something in your .xinitrc (suitably linked to also be your .xsession file :) you try and log back in and, doh! you simply get kicked back out to the XDM logon prompt. All is not lost, though, because you can still log in using XDM's failsafe mode. To start a failsafe session, you type in your username and passwd almost like usual, except instead of hitting <ENTER> after you type your password, strike the <F1> function key. This should grant you a single xterm window (and no window manager running) from which you can look at the, hopefully useful, .xsession-errors file.

If you happen to be logging in to a machine with the CDE based dtgreet logon prompt, you must select a failsafe session with the options button. To do so select Options->session->failsafe and then type your username and password as normal (i.e. don't use the F1 function key like with XDM). With CDE, I believe you also have to look in a .dt subdirectory for several different files where error messages might reside..

Sometimes even the failsafe session doesn't work, or provide useful information. We'll discuss troubleshooting in the X environment when we talk about X windows later in the semester.

3.3 The Shell

There are a large number of shells to choose from: sh, csh, tcsh, bash, zsh, ksh to mention some of the most widely used. However, it is worth first noting a little history. The very first shell was called the bourne shell, aka /bin/sh. This is the default shell in most system-V style Unix versions. In response to a desire for additional features and an different programming syntax, the C shell was written, aka /bin/csh. This new shell provided exciting stuff like job control and a C-like syntax. The C shell is the default shell in BSD style Unix versions. These two shells basically define the syntax of subsequent ones. The shell I use, tcsh, is a variant of the csh shell, while bash, another popular shell, is considered to be a variant of the bourne shell (although it incorporates lots of useful tcsh features).

The shell is your interface with the operating system. Most of them are quite powerful, providing a programming facility with variables and control structure like any programming language, as well as other features like job control and command line editing. I will provide some examples using the tcsh shell.

Variables can be either scalar or list . To set a scalar:

    huizil % set foo = "bar"
    huizil % echo "foo = $foo"
    foo = bar
And to set a list:

    huizil % set path = ( /{,s}bin /usr/{,s,local/,local/s}bin )
    huizil % echo $path
    /bin /sbin /usr/bin /usr/sbin /usr/local/bin /usr/local/sbin
Notice how the shell expanded the items in curly braces.

It is possible to iterate over a list and perform some command. For example:

    huizil % set cmd = 'ls'
    huizil % foreach p ($path)
    foreach? if ( -e ${p}/$cmd ) then
    foreach?    echo "cmd $cmd found in $p"
    foreach? endif
    foreach? end
    cmd ls found in /bin
    cmd ls found in /usr/bin
Let's revisit some of the dotfiles. Specifically, the .login and .cshrc files. To illustrate when each of these files is parsed, we'll perform a simple experiment. First we need to edit each of the dotfiles and add an echo line. Here is the line I added to my .cshrc file:

    echo "ECHO FROM CSHRC"
I added a similar line to my .login file and then logged out and back in again. Here is the result:

    xbalanque% ssh coatl
    ECHO FROM CSHRC
    ECHO FROM LOGIN
    Thu Jan 18 19:33:42 MST 2001
    coatl % 
Notice the order: the .cshrc file is parsed first.

Now, I mis-stated this in class, so let's see in exactly which other situations the .cshrc is parsed. First, I will set a variable using command substitution:

    coatl % set foo = `hostname`
    coatl % 
So, the file is definately not parsed when you perform a command substition (a sub-shell is still spawned, though; how might you prove this? hint: what is the value of the special shell variable $$ ?) In fact, the .cshrc file only gets parsed when a new sub-shell is invoked that uses the csh or tcsh. The following example is simply invoking a new instance of tcsh:

    coatl % echo $$
    9295
    coatl % tcsh
    ECHO FROM CSHRC
    coatl % echo $$
    3042
    coatl % exit
    coatl % 
The second example involves writing a very short tcsh script. Here is the script, in a file called hello.

    coatl % cat hello
    #!/bin/csh
    echo hello world
    coatl % 
You can see it is quite short, only two lines of text in the file. The first line is important: it indicates what shell interpreter should be used to process the script. In this case I want /bin/csh (the C shell) to process it. (What happens if the path given on the first line doesn't exist or isn't executable?) Here is the result of invoking the 'hello' script:

    coatl % chmod +x hello
    coatl % ./hello
    ECHO FROM CSHRC
    hello world
    coatl % 
First I had to turn on execeute permissions on the file. Notice the expected result of running the script: my own .cshrc file was processed before the script!

The tcsh(1) manpage is extensive, if not even frightening! The document is divided into two major parts, the DESCRIPTION and the REFERENCE parts. Most of the interesting information about the shell is found in the DESCRIPTION part, but the all-important definitions for builtin commands and special variables are found in the REFERENCE part. (Just what are 'builtin' commands, by the way?) Here are what I consider to be the more important sections of the DESCRIPTION part of the manpage:

4 Documentation

I believe one of the fundamental traits of a good system administrator is the ability to know where to look for answers. Simply memorizing everything is of course a fruitless waste of time and life for most of us; so it is far better to remember the general concepts and where to look for the specifics when you need to. Ideally, every command you can use from a shell ought to have a concise and standardized manpage, so that you can quickly get the information you need from it. Life is not so grand though, and many version of Unix have frustrating or even non-existant manual pages. The default end user installation of Solaris, for example doesn't even include the manpages!

There are many other sources of documentation, both online and in hardcopy (like books). I find it to be the case that printed material is almost invariably out of date, no matter how recently it was published. That is a good reason to always stick with online manpages as there is a good chance they match the actual functionality present in a given command.

4.1 Online Manual Pages

The online manual is divided up into a number of different sections, numbered from section one, the User Commands. Most sections have an intro page which may be viewed; e.g. on an OpenBSD box one can view the section two intro page as follows:

    saclass-1 % man 2 intro
    INTRO(2)                  OpenBSD Programmer's Manual                 INTRO(2)

NAME intro - introduction to system calls and error numbers ...

Here is an overview of the sections of the manual in OpenBSD:

You should take a look at the manpage for the man command itself. Partitcularly, read it for different Operating Systems - e.g. read the OpenBSD man manpage, and then the Solaris man manpage - they are somewhat different. You have probably noted the syntax for reading a manpage of a section other than section one. Roughly, it is:

    man [ $section ] $manpage
That is, the $section may be specified, but it is optional. If you do not specify a section, but the manpage you have asked for only exists on some other section than one, it will of course be printed. The usefulness of specifying the section number is only obvious when there are manpages in several sections that have the same name (e.g. passwd(1) and passwd(5).)

4.2 Linux HowTo Doc

The manpages on Linux systems tend to be fairly frustrating. In some cases the manpage simply isn't present (hasn't ever been written). In other cases the manpage is simply difficult to read or doesn't tell the whole story. I like it when the manpage gives a concise synopsis of the command's syntax and somewhere near the top of the page, delineates the commands options in a tabular format so that you can look up the option and quickly see what it does or how it is used. A frustrating manpage will have the list of options and their usage buried within one or more looong paragraphs forcing one to skim through a lot more text to find what is being looked for.

The linux community offers no apologies for its sad state of manpage-ness. Instead, they point the inquiring user at the extensive collection of HowTo documents, as well as the Linux Documentation Project itself. please see: ./intro for web links.

4.3 Other WWW Resources

The list of online resources is basically endless. I find the best way to navigate is by finding good jumpgates that have lots of links that are relevant to you. You also need to be able to efficiently make use of search engines like google. If you are just looking for software, then see: http://freshmeat.org

4.4 A Note About Printing

I strongly suggest that you try and get used to reading things online as this will significantly reduce the impact on our forests, as well as help to ensure that you are getting up-to-date information. However, I understand that there is a need to sometimes print things out. If you decide that you need to print out stuff like manpages please use a print command like a2ps or mpage as well as a duplexing printer like akira in the CSEL lab. The print commands will let you print 4-up so that 4 pages get printed (smaller) on one page of paper, giving you 8 pages per sheet of paper! For example:

    coatl % man tcsh | /tools/cs/a2ps/bin/a2ps -4 -Phurl
will print the tcsh manpage at 4 virtual pages per real page of paper on a printer named 'hurl'.

5 The ps Command

The ps command comes in two main flavors, the ATT or SYS-V flavor and the BSD flavor. Read the manpages for ps on an OpenBsd system and a Solaris system to find out the differences in detail.

Here is an BSD example:

    baelzebub % ps aux
    USER       PID %CPU %MEM   VSZ   RSS TT   STAT STARTED       TIME COMMAND
    tor       7369  2.0  3.2   648   996 C0  S+     6:15PM    4:39.67 mpg123 -z Reg
    root         1  0.0  0.6   312   184 ??  Is     9:20AM    0:00.02 /sbin/init 
    root         2  0.0 19.6     0  6136 ??  DL     9:20AM    0:00.01 (pagedaemon)
    root         3  0.0 19.6     0  6136 ??  DL     9:20AM    0:00.70 (update)
    root         4  0.0 19.6     0  6136 ??  DL     9:20AM    0:00.42 (apm0)
    root     18555  0.0  1.1   396   332 ??  Is     9:20AM    0:00.04 dhclient dc0 
    root     31645  0.0  1.3    96   408 ??  Ss     9:20AM    0:00.12 syslogd 
    root     19329  0.0  1.2    64   356 ??  Is     9:20AM    0:00.00 portmap 
    root     29069  0.0  1.7    80   512 ??  Is     9:20AM    0:00.01 inetd 
    root      3166  0.0  2.8   252   864 ??  Ss     9:20AM    0:00.20 /usr/sbin/ssh
    root     28344  0.0  4.5   768  1416 C0- I      9:20AM    0:00.10 /usr/local/sb
    root     15634  0.0  1.6   224   504 ??  Is     9:20AM    0:00.11 cron 
    tor      24793  0.0  2.2   908   680 C0  Is     9:20AM    0:00.15 -tcsh (tcsh)
    root     32433  0.0  1.7    44   516 C1  Is+    9:20AM    0:00.01 /usr/libexec/
    root      6163  0.0  1.7    44   512 C2  Is+    9:20AM    0:00.00 /usr/libexec/
    root     14561  0.0  1.7    44   512 C3  Is+    9:20AM    0:00.00 /usr/libexec/
    root     11963  0.0  1.7    44   512 C5  Is+    9:20AM    0:00.01 /usr/libexec/
    root     17515  0.9  3.8   276  1176 ??  S      8:44PM    0:00.21 sshd: tor@tty
    tor      10090  0.0  2.4   916   732 p0  Ss     8:44PM    0:00.04 -tcsh (tcsh)
    root         0  0.0 19.6     0  6136 ??  DLs    9:20AM    0:00.02 (swapper)
    tor      16777  0.0  0.7   276   212 p0  R+     8:44PM    0:00.01 ps -aux 
    baelzebub % 
The a flag say to list process that belong to other users as well as your own. The x flag says to list processes that don't have "controlling terminals" (what does that mean?). Finally, the u flag says to give a specific set of information for each process listed. You can look in the manpage to see exactly what each of the above fields mean. Notice that the command is often truncated in the above example. Use the ww flag to get the entire command line for each process:

    xbalanque% ps auxww | more
    USER       PID %CPU %MEM   VSZ   RSS TT   STAT STARTED       TIME COMMAND
    root      3026  2.7  4.6  8776 11912 ??  Ss     7Jan01   26:33.47 /usr/X11R6/bin
    /X vt05 -auth /usr/X11R6/lib/X11/xdm/authdir/authfiles/A:0-WN7442 (XF86_SVGA)
    root     23332  0.0  0.2    96   416 ??  Ss     7Jan01    0:02.34 syslogd 
    root      1089  0.0  0.1    68   352 ??  Is     7Jan01    0:00.01 portmap 
    root      3017  0.0  0.1   364   296 ??  Is     7Jan01    0:00.03 mountd 
    root     24963  0.0  0.0   124   112 ??  Is     7Jan01    0:00.01 nfsd: master (
    nfsd)
    root     17756  0.0  0.4   300   928 ??  Is     7Jan01    0:05.94 /usr/sbin/sshd
    ...

How does the sys-V ps command differ from the BSD version? The answer to this question is left as an excersize for the reader ;) Log into a Solaris machine in the student lab (like bfs.cs.colorado.edu) and run the /usr/bin/ps command as well as read the manpage. Note that there is also a /usr/ucb/ps command on Solaris machines. This is part of the "BSD compatibility" that Solaris maintains with its BSD roots (SunOS 4.x was Sun's first OS which was a BSD flavor, SunOS 5.x is what most people recognize as Solaris). I recommend that you have /usr/bin in your path before /usr/ucb because the reverse introduces subtle behavioral differences...

6 The grep Command

The command grep and its next-of-kin egrep and fgrep make use of regular expressions to search files or text-streams for specific patterns. Grep is easily on of the commands I use more than any other; being able to effectively utilize regular expressions makes grep invaluable for lots of purposes.

One of the things you will do frequently is to grep through the output of the ps command. By way of an example, I refer back to the ps section where you will notice that I switched machines between my two examples. The reason is that the first process listed for the machine named baelzebub is called mpg123 and it has an extremely long command line. Check it out:

    baelzebub % ps auxww | grep mpg123 | wc
           2    1970  131380 
The wc command gives a 'word-count' of the input: two lines, 1970 words and 131380 characters! In the example, I piped the output of the ps command into the grep command and then further piped the output of grep into the wc command. By the way, baelzebub is my home stereo machine, playing lots music in random order; the mpg123 command takes a list of mp3 files as arguments. I invoke it with regular expressions like this:

    baelzebub % mpg123 -z /mp3/[Rwl]*/[CBS]*/*.mp3
That would play albums from /mp3/Reggae, /mp3/world, and /mp3/latin. Further, only albums that started with the letters B C or S would be played. The -z flag says to play in random order.

The egrep (or grep -E ) command lets you use an extended set of regular expressions. For example:

    coatl % grep -E '(tor|root)' /etc/passwd
    root:*:0:0:Charlie &:/root:/bin/csh
    daemon:*:1:1:The devil himself:/root:/sbin/nologin
    operator:*:2:5:System &:/operator:/sbin/nologin
    csop:*:11528:7:CS Operators,EE00-69,,:/usr/local/adm/dumps:/bin/tcsh
    tor:*:11518:147:Torleif Mohling,ECCS1B05,x20677,,:/home/fcsk/tor:/bin/tcsh
    coatl % 
The regular expression needs to be placed in quotes so that the shell doesn't interprete the | or ( ) characters.

7 Sudo

Sudo is program that lets you run a program with another user's permissions. Very often (and by default) that user is root. With sudo, as a system administrator, you can can grant very specific permissions to specific people or groups, to the granularity of specific machine, command and argument. Much care must be taken if you don't trust the person to whom you are granting sudo privs. Any command that can invoke a sub-shell is vulnerable to an attack by a malicious sudo user (these types of programs include any text editors), another less suspicious command that is dangerous is chmod (the user can turn on the setuid bit for /bin/csh and just invoke a root shell!) Despite its vulnerabilities, sudo is a very useful administrative tool. It provides command logging so if mistakes are made, one only needs to consult the sudo log to verify.

When you use sudo, it asks you for your password, sometimes it will output a warning first, too:

    coatl % sudo visudo

We trust you have received the usual lecture from the local System Administrator. It usually boils down to these two things:

#1) Respect the privacy of others. #2) Think before you type.

    Password:
    ...
READ the sudo(1) manpage! What flag lets you view your own sudo priveledges? What flag lets you specify another user than root to be owner of the process? How do you change your own, or someone else's privs?

You will all get sudo privs on the example machines like saclass.cs.colorado.edu. Before you actually get them, though, I'd like to reiterate a few things about it. When you have sudo permissions, particularly if they are unrestricted root permissions, you wield a dangerous power: you can mistakenly reboot the wrong machine by typing in the "wrong" window of your desktop; you can remove any file, you can change any file. Please drive with care!

Sudo's config file is /etc/sudoers. It has a very rich specification language that allows you to specify permissions to the level of specific user, host, command and arguments. You can read more about it in the sudoers(5) manpage.

You each need to sign a sudoers agreement before you can be enabled for sudo on the saclass machines. When you are enabled you will be enabled to use one command. I will leave it as an exercise for you to figure out what that command is and then use it to grant yourself more useful permissions following a template in the /etc/sudoers file.

8 Using RCS - the Revision Control System

Using some kind of version control system is absolutely essential when you are editing system files; particularly if there are any other folks who might possibly be editing that file at the same time. RCS is a common and easy to use system.

For the following examples I will be editing the file /etc/rc.conf. The very first thing that needs to be done is to create an RCS directory in the same directory as the files which will be placed under RCS's control:

    quetzel % cd /etc
    quetzel % sudo mkdir RCS
The next thing that needs to be done for initially for each file that will be in the RCS system is to perform an initial check-in as follows:

    quetzel % sudo ci -u rc.conf
    RCS/rc.conf,v  <--  rc.conf
    enter description, terminated with single '.' or end of file:
    NOTE: This is NOT the log message!
    >> boot time startup configuration parameters
    >> .
    initial revision: 1.1
    done
The command to check-in a file is ci and the -u argument says to unlock the file which means that it becomes read-only. The first time you check in a file, RCS asks you for a description of the file. You end the description by typing a . (dot) on a line by itself.

Now, to edit the file you need to check it back out with a lock so that you can edit it:

    quetzel % sudo co -l rc.conf
    RCS/rc.conf,v  -->  rc.conf
    revision 1.1 (locked)
    done
You use the co command check a file out and the -l argument to lock it. Now you can edit the file with your favorite editor. Once you are done, you check the file back in and unlock it. This time, RCS will ask you for a log message which should describe the changes you made to the file. Again, you terminate the log entry with a . on a line by itself:

    quetzel % sudo vi rc.conf
    quetzel % sudo ci -u !$
    sudo ci -u rc.conf
    RCS/rc.conf,v  <--  rc.conf
    new revision: 1.2; previous revision: 1.1
    enter log message, terminated with single '.' or end of file:
    >> turned off sendmail
    >> .
    done
That is pretty much all there is to it! There are some other useful commands. First, rlog can be used to print out the version information including the description and log entries for the file:

    quetzel % rlog rc.conf

RCS file: RCS/rc.conf,v Working file: rc.conf head: 1.2 branch: locks: strict access list: symbolic names: keyword substitution: kv total revisions: 2; selected revisions: 2 description: boot time startup configuration parameters ---------------------------- revision 1.2 date: 2000/01/22 21:52:26; author: tor; state: Exp; lines: +1 -1 turned off sendmail ---------------------------- revision 1.1 date: 2000/01/22 21:48:20; author: tor; state: Exp; Initial revision =============================================================================

Another useful command is called rcsdiff and it is used to show the differences between two versions. By default it compares the current file with the most recently checked-in version:

    quetzel % rcsdiff rc.conf
    ===================================================================
    RCS file: RCS/rc.conf,v
    retrieving revision 1.2
    diff -r1.2 rc.conf
In this case there is no difference. We can compare to an earlier version using the -r argument:

    quetzel % rcsdiff -r1.1 rc.conf
    ===================================================================
    RCS file: RCS/rc.conf,v
    retrieving revision 1.1
    diff -r1.1 rc.conf
    12c12
    < sendmail_flags="-q30m"        # for normal use: "-bd -q30m"
    ---
    > sendmail_flags=NO     # for normal use: "-bd -q30m"
This time we can see that the sendmail_flags flags line was changed between the two versions.

Finally, the rcs command itself is sometimes necessary to, for example, break a lock held by someone else. Read the manpage for details.

9 The kill command

Kill is another useful command, and it is definately not used only to 'kill' processes! More generally, kill is used to send a signal to a process. How that signal is received is basically up to the folks of wrote the program; different signals can be caught and used to trigger different functionality. Some signals can not be caught in this manner. For example the KILL signal (number nine) can not be caught and it terminates the process without mercy.

Here is a list of some signals, excerpted from the kill(1) manpage on OpenBSD:

       1       HUP (hang up)
       2       INT (interrupt)
       3       QUIT (quit)
       6       ABRT (abort)
       9       KILL (non-catchable, non-ignorable kill)
       14      ALRM (alarm clock)
       15      TERM (software termination signal)
Signals may be referenced by name or by number; in the interests of safety, it is recommended that you reference signals by name when using the kill command. This is to avoid the chance that the number may be a different signal (if, for example, you were logged into a different architecture than you thought).

Here is an example of using kill to cause a process to re-read its configuration file. Syslogd is a server that listens for log messages sent from other server processes, syslogd then writes these log messages to specific logfiles depending on the originating server and the severity of the message (from informational to critical errors). When a change is made to syslogd's configuration file, /etc/syslog.conf then syslogd needs to be notified to re-read the configuration. To do this you need to send syslogd the SIGHUP signal (traditionally signal number one).

    coatl % ps aux | grep syslogd
    tor      29438  0.0  0.4   148   472 q2  S+    11:03PM    0:00.01 grep syslogd 
    root      8525  0.0  0.3   108   412 ??  Ss    10:56AM    0:00.22 syslogd 
    coatl % sudo kill -HUP 8525
    Password:
    coatl % 
Another example is a pesky netscape process festering on a linux box in the CSEL lab. Here's a look at it with ps:

    em % ps aux | grep netscape
    joeuser   1718 99.6  0.1 33348   72 ?        R     2000 62289:36 [netscape.exe]
You can see that this a runaway process: it has chewed up 62289 minutes of CPU time! It also doesn't respond to any signal except the merciless KILL signal:

    em % ps aux | grep netscape
    joeuser   1718 99.6  0.1 33348   72 ?        R     2000 62291:35 [netscape.exe]
    em % sudo kill 1718
    Password:
    em % ps aux | grep netscape
    joeuser   1718 99.6  0.1 33348   72 ?        R     2000 62291:35 [netscape.exe]
    em % sudo kill -KILL 1718
    em % ps aux | grep netscape
    em % 

10 The awk command

Awk is not a command I use every day. However it is still very useful every now again and it is invaluable when it comes to writing shell scripts. Awk assumes that its input (either STDIN from a pipe or a file) is in a tabular format, that is, each line of input is a series of fields. Given a field-seperator (a <space> by default) awk can parse out specific columns from the table. Awk can do much, much more than that though. The manpage styles it as a "pattern-directed scanning and processing language."

To illustrate, here is a simple example. Take a look at the /etc/passwd file (I'm using head(1), which is a simple command that just prints the first few lines of a file):

    coatl % head /etc/passwd
    root:*:0:0:Charlie &:/root:/bin/csh
    daemon:*:1:1:The devil himself:/root:/sbin/nologin
    operator:*:2:5:System &:/operator:/sbin/nologin
    bin:*:3:7:Binaries Commands and Source,,,:/:/sbin/nologin
    uucp:*:66:1:UNIX-to-UNIX Copy:/var/spool/uucppublic:/usr/libexec/uucp/uucico
    www:*:67:67:HTTP server:/var/www:/sbin/nologin
    named:*:70:70:BIND Name Service Daemon:/var/named:/sbin/nologin
    nobody:*:32767:32767:Unprivileged user:/nonexistent:/sbin/nologin
    coatl % 
This is perfect input for awk; each line is a colon-seperated set of fields. If we wanted just the usernames from the file we could use awk:

    coatl % head /etc/passwd | awk -F: '{ print $1 }'
    root
    daemon
    operator
    bin
    uucp
    www
    named
    nobody
    coatl % 
Be aware that awk's syntax can vary depending on the version of Unix that you are using. It wont change between BSD and linux much, though. Again, the single-quotes are necessary to keep the shell from interpreting the { } characters.

Go to: top / index

Source document: class01.txt
Last modified: 0
Category: standard/specification/policy/procedure/guide/manual
Obsoletes: