Most of you are familiar with the Network File System at least from the point of view of the user: every time you login to a Unix box in the CSEL lab, your homedirectory is accessed via NFS. In the lab case, your homedirectory physically lives on a specialized computer called a file server whose sole purpose in life is to provide filesystems to other machines via NFS. This particular file server's name is cia. Cia exports a homedirectory filesystem called /nfs/home which other machines mount on a local mount-point directory called /home/cia. (The whole story is actually more complicated because AMD, the automount daemon, is involved too, but we'll defer that until later.)
NFS was originally developed at Sun Microsystems in the middle 1980's. It consists of several client and server-side components which we will explore below. The NFS applications themselves rely on another facility developed at Sun call Remote Procedure Call or RPC.
RPC is basically an additional networking layer that sits between NFS (the application layer) and the transport layer. NFS can use either TCP or UDP for its transport layer protocol. However, the details of networking are not present in the NFS application code because they are hidden by the abstraction offered by RPC.
Another concept that needs to be mentioned in association with RPC is that of eXternal Data Representation (XDR) which is simply a method of sharing data between different computers that may physically represent binary numbers in their CPUs differently (i.e. big versus little endien representation). Unlike the related concept of Network Byte Order that is used for encoding information in TCP/IP packet headers, XDR deals with more than simple binary reperesentation of numbers, by including definitions for floating-point numbers, strings and so on.
Applications that make use of RPC have associated program and version numbers. When a given RPC-based application server starts up it registers itself with another program called the portmapper. The portmapper simply keeps track of RPC program and version numbers in association with relevant TCP or UDP portnumbers. There is a file /etc/rpc that contains common mappings:
suod % cat /etc/rpc
# RPC program number data base
# $Revision: 1.3 $ (from 88/08/01 4.0 RPCSRC)
#
portmapper 100000 portmap sunrpc
rstat_svc 100001 rstatd rstat rup perfmeter
rusersd 100002 rusers
nfs 100003 nfsprog
ypserv 100004 ypprog
mountd 100005 mount showmount
ypbind 100007
....
To query the portmapper
you can use the rpcinfo command. For example, to list all of the
registered RPC services on a local machine do:
saclass-2 % rpcinfo -p
program vers proto port
100000 2 tcp 111 portmapper
100000 2 udp 111 portmapper
100001 1 udp 911 rstatd
100001 2 udp 911 rstatd
100001 3 udp 911 rstatd
100002 1 udp 804 rusersd
100002 2 udp 804 rusersd
100002 3 udp 804 rusersd
100003 2 tcp 2049 nfs
100003 3 tcp 2049 nfs
The same rpcinfo command can take an optional hostname as well:
saclass-2 % rpcinfo -p saclass-1
program vers proto port
100000 2 tcp 111 portmapper
100000 2 udp 111 portmapper
100001 1 udp 792 rstatd
100001 2 udp 792 rstatd
100001 3 udp 792 rstatd
100002 1 udp 745 rusersd
100002 2 udp 745 rusersd
100002 3 udp 745 rusersd
100005 1 udp 617 mountd
100005 3 udp 617 mountd
100005 1 tcp 720 mountd
100005 3 tcp 720 mountd
100021 1 udp 940 nlockmgr
100021 3 udp 940 nlockmgr
100021 1 tcp 949 nlockmgr
100021 3 tcp 949 nlockmgr
100003 2 udp 2049 nfs
100003 3 udp 2049 nfs
100003 2 tcp 2049 nfs
100003 3 tcp 2049 nfs
Generally, NFS client capability is already present in your Operating System's Kernel. If it is not, with Linux, for example, you will have to re-configure and compile the kernel. For NFS to function at all, you will of course have to have networking in general working as well as have an available filesystem exported from an NFS server.
You can use the showmount(8) command to list out exported NFS filesystems for a machine by giving it the -e argument:
saclass-2 % showmount -e saclass
Export list for saclass:
/home 10.0.0.0
You are returned a list of the filesystems exported by the host in question,
along with the machines or networks to which the filesystem is exported.
The basic process of making an NFS filesystem usable on an NFS client is pretty much just like making an ordinary UFS filesystem available: you use the mount command. Only to mount an NFS filesystem the 'device' is specified as:
host:directory
For example, to mount the filesystem exported from saclass-1 I
could issue the following mount command:
saclass-2 % sudo mount saclass:/home /home
That would mount the NFS filesystem from saclass onto the mount-point /home
on saclass-2. The fact that this filesystem is mounted on saclass-2 is
now also apparent when you use the df command or mount with no arguments:
saclass-2 % mount
..
/dev/hda1 on /boot type ext2 (rw)
/dev/hda8 on /usr type ext2 (rw)
/dev/hda7 on /var type ext2 (rw)
saclass:/home on /home type nfs (rw,addr=128.138.192.13)
The mount command, when invoked without any arguments simply
lists out the currently mounted filesystems, giving you
the device-name, the mount-point, the filesystem-type
and some other information. In this case, the additional
information associated with the NFS mount tell you that
NFS version three is being used; that UDP is being
used as the transport protocol and that the timeout is 100 seconds.
You can make NFS mounts like in the above example happen at boot startup time by adding entries to the /etc/fstab file as follows:
saclass:/home /home nfs rw,bg,soft,intr 0 0
This is a typically formatted entry in the /etc/fstab file. Here are
explanations for each field:
saclass:/home the filesystem 'device' to be mounted
/home the local mount-point for the filesystem
nfs the filesystem type
rw,bg,soft,intr a list of mount 'options' for the filesystem
0 0 the first zero indicates that the dump program
should not backup this filesystem;
the second zero indicates that the fsck program
should not attempt to check this filesystem
Here is a partial list of mount options:
rw mount filesystem read-write
ro mount filesystem read-only
bg if mount fails, keep trying in the background
hard if server goes down, make operations accessing it BLOCK
soft if server goes down, allow operations accessing it to
FAIL with an error
retrans=N number of times to retry request before FAILING
(must also specify 'soft')
timeo=N timeout period for a given request (in TENTHS of seconds)
intr allow users to INTERRUPT blocked requests (making
then return an error
rsize=N set the read buffer size (in bytes)
wsize=N set the write buffer size (in bytes)
Also, you can unmount partitions in the standard fashion:
saclass-2 % sudo umount /home
All of the above functionality is present by default under OpenBSD and
RedHat Linux. However, to be official you should actually turn on
NFS client services for OpenBSD in the /etc/rc.conf file. Make sure
there is a line in the file that looks like this:
nfs_client=YES
You don't actually need to do this in order to utilize basic
NFS client functionality, however, there is another client NFS daemon
called nfsiod(8) that will be started if you do enable NFS client
services. The nfsiod daemon is used to improve NFS performance
by doing filesystem block caching. It is not necessary to the
basic functionality of the NFS client, it only improves performance.
Sometimes you will see it called biod instead. Not all versions
of Unix include this daemon (for example, it is not present on
the RedHat systems.)
Setting up an NFS server is only slightly more complicated than setting up the NFS client. As before, the NFS functionality must be present in the Kernel (and for OpenBSD and RedHat it is by default) as well as the fact that networking must be operational.
To enable NFS server service on an OpenBSD box, again, you must edit the /etc/rc.conf file and make sure a line appears like this:
nfs_server=YES
And to enable such service on a RedHat box you need to make sure that
the knfsd RPM has been installed. For example, to see if the RPM
is already installed:
dwarf % rpm --query knfsd
knfsd-1.4.7-7
If it is not, you will have to obtain and install the RPM.
This will install the relevant applications as well as
the /etc/rc.d/init.d/nfs startup script. You need to make sure that
symbolic links are present in the appropriate runlevel directories
for your system.
There are two main daemons involved in the NFS server. These are the nfsd and mountd daemons. The mountd daemon handles the 'mount' requests from NFS client machines in terms of validating the request based on the contents of the /etc/exports file. The nfsd daemon handles the actual filesystem access requests (like reads and writes ) from the client.
There are a few other daemons that may be running also:
rquotad this daemon deals with user filesystem quotas
rlockd this daemon (theoretically) provides
file locking for NFS-mounted files.
The rlockd (sometimes seen as rpc.lockd or just lockd) is known to
be somewhat buggy, so don't rely on it.
Normally, four copies of nfsd are started (i.e. nfsd starts up and creates three copies of itself. The typical startup syntax for OpenBSD is:
saclass % nfsd -tun 4
The exact number of nfsd process that you want running on your NFS server
is dependent on a number of factors including how busy your
server is and how many CPUs or hardware contexts each CPU
has. The important thing to remember is that too many or too
few nfsd processes will both hurt performance.
The /etc/exports file is the heart of the NFS server setup. Unfortunately the format of the file differs between Linux and OpenBSD and between either of them and other operating systems. In fact, the solaris operating system changes things even more. To configure NFS service under Solaris you need to edit the /etc/dfs/dfstab file and read the shareall(1M) manpage. You should always consult the particular OS's manuals for details.
The basic format of entries in the exports file for OpenBSD divides each entry up into three components as follows:
<dir(s)-to-export> <export-options> <list-of-hosts>
The <dir(s)-to-export> component consists of one or more directories
within a single filesystem to be exported from the server.
It is important to understand that NFS filesystems do not
necessarily correspond directly to Unix Filesystems.
For example, you might have a Unix Filesystem /usr but only
export the sub-directories /usr/local and /usr/share.
You may specify multiple white-space seperated directories,
but they must all be given as absolute paths
(i.e. with a leading / ) and they must all
reside in the same underlying Unix Filesystem.
NFS Filesystems also may not span Unix Filesystem boundaries. For example, if you exported the / (root) filesystem the mountpoints for other filesystems would appear, but the directories, e.g. */usr*, would be empty.
The second component of an /etc/exports entry is the <export-options> list. The options list is, as you might guess, entirely optional and may be left out if you don't mind the default behavior. Options are used to cause the filesystem to be exported read-write or read-only and to specify how various user UID and GID values are dealt with (particularly the root user!). There are some other options too; see the manpage for details.
The default behavior is to map the root user credentials to -2:-2, meaning a UID of -2 and a GID of -2. All other users are authenticated based on matching UID numbers. To remap the root UID use the -maproot option. For example, to over-ride the default behavior and force the root credentials to be unchanged, you could give a -maproot option as follows:
-maproot=0:0
A related option is -mapall that is used in conjunction with the -ro
(read-only) option. The -mapall is used to map every UID to a
specific one, usually a user like nobody.
Finally, the third component of an exports entry specifies the machines to which the filesystem(s) will be exported. Individual hosts may be specified by hostname or IP address. Entire subnets may be specified using the -network=<net> and -mask=<mask> flags. The <net> may be specified as an IP address or as a symbolic network name (found in the /etc/networks file). You may also specify groups of machine by using netgroup names (found in the /etc/netgroup file).
The exports file for a Linux system has a different syntax.
The best reference for Linux NFS is probably the NFS Howto Doc.
See: http://www.linuxdoc.org/HOWTO/NFS-HOWTO.html
While the syntax does differ, the meaning is still the same: the
options mostly control how UIDs will be mapped over NFS.
The automounter concept is kind of a difficult one to introduce. So, instead I will first illustrate some problems that automounters address.
We have already seen how NFS clients can mount filesystems from NFS servers. For a very small site you can simply maintain all the NFS mounts between machines by hand; i.e. simply setting up fixed entries in each machines' /etc/fstab files to cause NFS mounts to happen at boot time. Maintaining these relationships for larger sites becomes an administrative headache: imagine 100 different machines, each exporting its own home-directory filesystem - some of these homedir filesystems are like /home/cia with homedirs for hundreds of users, while others are just homedirs for perhaps only the owner of the given machine. Now imagine maintaining the NFS mounts and the information in each of those hundred machines' /etc/fstab files! Doing this by hand would be a nightmare!
AMD was originally written by Jan-Simon Pendry and Nick Williams as a (free) replacement for the bug-ridden automountd program that was produced by Sun. Sun's version has been improved significantly since that time; the Linux version of the automountd (called autofs ) is similar to the Sun version. The current version of AMD is called Am-utils (version am-utils-6.0.5) and is maintained by Erez Zadok.
ecks:tor % cd /tools/cs/perl5/bin
ecks:bin % pwd
/nfs/fiche-202/tools/cs/x86+Linux2/perl-5.004_04/bin
The above example, clearly illustrates the difference.
I reference the perl binary with the same path that I cd'd
into, i.e. /tools/cs/perl5/bin.
But if you use the pwd command to show your real current
directory, you see the underlying NFS mountpoint path,
i.e. /nfs/fiche-202/tools/cs/x86+Linux2/perl-5.004_04/bin.
So how does it work? The underlying automount-point is /tools. AMD takes control of this directory and applies rules to it that come from an amdmap. There is a different amdmap for each automount-point. The map describes locations of filesystems and how those filesystems or volumes within those filesystems are accessed. (A volume is basically a subtree of a filesystem.) We'll see an example that utilizes volumes when look at the tools map in more detail below.
Before diving into the /tools example, let's look at a simpler one: homedirs. Home directories are simpler because there aren't any architecture/OS distinctions. In the CSEL lab, the /home map is pretty simple because the only game in town is /home/cia! We still use amdmaps for this scenario because we use one map that gets distributed to all the machines in the lab and we don't have to edit any of the local /etc/fstab files.
Here is what the map looks like (you can see it for yourself on any of the lab machines in the directory /usr/local/etc/amdmaps, btw):
/defaults type:=nfs;rfs:=${autodir}${path};fs:=${autodir}${path};\
rhost:=${key}.cs.colorado.edu;opts:=rw,${NFS_BOILERPLATE},\
${OS_OPTIONS}timeo=${NFS_TIMEO},retrans=${NFS_RETRANS},\
rsize=${NFS_RSIZE},wsize=${NFS_WSIZE}
* host==${key};type:=link \
host!=${key}
cia rfs:=/nfs${path};\
wire==cu-cs-hpsc;rhost:=cia;rfs:=/nfs${path}; \
rhost:=cia-fs;rfs:=/nfs${path};
This might look like gibberish, but let's look at each section in more
detail.
The first section is defining default behavior for the rest of the map.
The type defines the filesystem; by setting it to nfs in the /defaults section we only need to specify the type if it is not an nfs filesystem. I'll explain other types as they are encountered.
The next attribute, rfs, stands for remote file system and in this case we are giving a regular expression for what the remote filesystem mount-path will be. The variable autodir is the name of the directory where the actual NFS mounts happen. The choice of directory name is arbitrary, but it is good to be consistent. In the CSEL lab we use the directory /nfs as the autodir. The variable path is filled in at runtime.
The fs attribute defines the default local mountpoint for the NFS filesystem. The rhost is the remote host from which the filesystem is exported. Finally, the opts are NFS related mount options.
Except for the /defaults section, every other entry basically has the following syntax:
KEY LOCATION1 LOCATION2 ...
First, keep in mind that this map is defining the behavior of the
the /home automount-point. The KEY really just ends up being
a subdirectory of the automount-point.
For example, if you cd into the directory /home/cia you get the /home part which is the automount-point, and you get the /cia part which is the key. The two together make up the path variable when it gets expanded at runtime.
Now let's look at the next entry in the map:
* host==${key};type:=link \
host!=${key}
For the CSEL lab, this rule isn't necessary, but let's
look at it anyway. We'll see it in action with the /home
map for the research side of the CS networks. Anyway, the
KEY is simply a '*' which has its standard semantics: it is
a default rule that will be tried if there is not a better
match for the given KEY. For example, if I cd /home/cia then
the KEY cia has a better match (the next rule is specifically
for the KEY cia ) so this rule won't match, on the other hand,
if I cd /home/fooberry, then this rule would be tried.
There are two LOCATIONS given in the rule. We also have a new concept introduced called the selector which acts like an if statement, and a new type called link.
The first LOCATION is:
host==${key};type:=link
The first part is a selector and it is effectively an if
statment which says, "if the key is equal to the (local) hostname,
then.." If the selector evaluates to true then the LOCATION
is considered valid. In this case, we are assuming the filesystem
is in fact not an NFS filesystem, but a local one, so only
a symbolic link is made to it (that's why the type is
called link !).
The second LOCATION:
host!=${key}
Is also a selector and guaranteed to evaluate to true
if the previous selector evaluated to false.
Remember that in both cases, the attributes defined in the /defaults section
apply. We'll look at a similar rule and see it in action when we
get to the next amdmap example.
Now we have the rule for the cia KEY:
cia rfs:=/nfs${path};\
wire==cu-cs-hpsc;rhost:=cia;rfs:=/nfs${path}; \
rhost:=cia-fs;rfs:=/nfs${path};
Actually, given the attributes defined in the /defaults section,
most of this rule is really redundant. It could be re-written
to have exactly the same behavior as follows:
cia wire==cu-cs-hpsc;rhost:=cia \
rhost:=cia-fs
Again we have two LOCATIONS specified. The first LOCATION has
another selector only this time it is an if statement that
says, "if the local subnet (i.e. wire ) is cu-cs-hpsc, then.."
The machine cia has two different network interfaces.
One of them is associated with the hostname cia and is
on the CSEL subnet (aka cu-cs-hpsc) the other is associated
with the hostname cia-fs and is on a subnet in the CS research
networks.
Now let's look at the /home map for the research nets,
it's a lot more interesting:
/defaults type:=nfs;rfs:=${autodir}${path};fs:=${autodir}${path};\
rhost:=${key}.cs.colorado.edu;opts:=rw,${NFS_BOILERPLATE},\
${OS_OPTIONS}timeo=${NFS_TIMEO},retrans=${NFS_RETRANS},\
rsize=${NFS_RSIZE},wsize=${NFS_WSIZE}
* host==${key};type:=link \
host!=${key}
cia rfs:=/nfs${path};\
wire==cu-cs-hpsc;rhost:=cia;rfs:=/nfs${path}; \
rhost:=cia-fs;rfs:=/nfs${path};
saros.1 host==saros;type:=link \
host!=saros;rhost:=saros
saros.2 host==saros;type:=link \
host!=saros;rhost:=saros
This map looks very similar to the one we looked at before.
The first three sections are identical. One thing to note, however
is that in this case we can not re-write the cia rule as we
did above. The reason is that in the CSEL lab we use /nfs
as the autodir, which made the rfs:=/nfs${path} attribute specifications
unnecessary. However, on the research nets we use an autodir
called /tmp_mnt for historical reasons, therefore the ${autodir}${path}
regex (regular expression) that gets expanded from the /defaults section
would be /tmp_mnt/home/cia which won't work for cia!
Let's look more closely at the default map entry:
* host==${key};type:=link \
host!=${key}
The machine on my desk at CSops is named coatl and it exports
a homedir filesystem:
suod % showmount -e coatl
export list for coatl:
/tmp_mnt/home/coatl admin-net.cs.colorado.edu...
The name of the exported filesystem is /tmp_mnt/home/coatl.
On another machine I will cd into the homedir exported from coatl.
The above default rule will be facilitating:
suod % cd /home/coatl
suod % pwd
/tmp_mnt/home/coatl
suod % df .
Filesystem kbytes used avail capacity Mounted on
coatl.cs.colorado.edu:/tmp_mnt/home/coatl
2097151 64109 1830230 3% /tmp_mnt/home/coatl
To reiterate: /home is the automount-point and /coatl is the KEY.
The second of the two LOCATION selectors fired because our host was suod and
the key was coatl. To follow the behavior more closely, look back at the
attributes that get defined at runtime from the (top) /defaults section.
In particular, after the second LOCATION is chosen, the rfs will
be set to /tmp_mnt/home/coatl because the autodir is defined
to be /tmp_mnt (this is specified on the amd command-line when it
is started at boot) and the path (which consists of the automount-point
and the KEY) is /home/coatl. The rhost is set to coatl.cs.colorado.edu
again because the KEY is coatl.
The automounter figures out the name of the filesystem and the name of the server based on the rules in the amdmaps, and then causes the filesystem to be mounted if necessary. In the above case, the amd process on the machine suod caused the filesystem /tmp_mnt/home/coatl exported from the machine coatl to be NFS-mounted on suod using a local mountpoint of /tmp_mnt/home/coatl (which is not coincidental). Finally, a symbolic link is made from the automount directory to the nfsmount directory. If you actually do an ls -l in the automount directory you will see the truth:
suod % cd /home
suod % ls -l coatl
lrwxrwxrwx 1 root 19 Mar 22 00:36 coatl -> /tmp_mnt/home/coatl/
Now let's turn to the more complicated tools maps. These maps are more complex in order to maintain a uniform execution path for a given tool across the multiple hardware/OS combinations we support. Additionally, they have increased complexity to account for different toolset hierarchies, local caches of certain tools and redundant fileservers.
First we'll look at the tools map for the CSEL lab. Again, the default map is in /usr/local/etc/amdmaps and it is called map.tools.
Here is the entire map (as you can see for yourself if you care to look on a running lab machine):
# AMD map for /tools (see: /csops/RFC/tools.txt)
#
# This file is autogenerated by: gen_tools_map
#
# If you modify it by hand, your changes will be obliterated.
#
# If a default value for 'rfs' exists, amd is using it instead of 'fs'
# for mount of type 'link' which are children of 'auto' mounts, but
# only if another such mount exists and is happy. So 'rfs' can't be
# specified in the defaults list until this bug is fixed.
#
# These default values for 'rfs' and 'fs' are expected by some of the
# entries below, so don't mess with them unless you are sure you know
# what you are doing.
#
# There really isn't supposed to be a comma after ${NOCONN}. It is
# either defined to be empty, or as `noconn,'.
/defaults type:=nfs;fs:=/nfs/${rhost}${path/};\
opts:=rw,intr,spongy,grpid,${NOCONN}timeo=${NFS_TIMEO},\
retrans=${NFS_RETRANS},rsize=${NFS_RSIZE},wsize=${NFS_WSIZE}
#
# Package set = cs
#
cs type:=auto;fs:=${map};
cs/* -type:=nfs;rfs:=/local/tools/cs;fs:=/nfs/${rhost}/tools/cs;\
sublink:=${karch}/${/key}; \
rhost:=fiche-202;
#
# a hack so you can browse /tools/cs
#
cs/top -type:=nfs;rfs:=/local/tools/cs;fs:=/nfs/${rhost}/tools/cs;\
sublink:=${karch} \
rhost:=fiche-202;
# Special rules for set cs
#
# Package set = edu
#
edu type:=auto;fs:=${map};
edu/* -type:=nfs;rfs:=/local/tools/edu;fs:=/nfs/${rhost}/tools/edu;\
sublink:=${karch}/${/key}; \
rhost:=fiche-202;
#
# a hack so you can browse /tools/edu
#
edu/top -type:=nfs;rfs:=/local/tools/edu;fs:=/nfs/${rhost}/tools/edu;\
sublink:=${karch} \
rhost:=fiche-202;
# Special rules for set edu
#
# Package set = ugrad
#
ugrad type:=auto;fs:=${map};
ugrad/* -type:=nfs;rfs:=/local/tools/edu;fs:=/nfs/${rhost}/tools/edu;\
sublink:=${karch}/${/key}; \
rhost:=fiche-202;
#
# a hack so you can browse /tools/ugrad
#
ugrad/top -type:=nfs;rfs:=/local/tools/edu;fs:=/nfs/${rhost}/tools/edu;\
sublink:=${karch} \
rhost:=fiche-202;
# Special rules for set ugrad
#
# Package set = opt
#
opt type:=auto;fs:=${map};
opt/* -type:=nfs;rfs:=/local/tools/opt;fs:=/nfs/${rhost}/tools/opt;\
sublink:=${karch}/${/key}; \
rhost:=fiche-202;
#
# a hack so you can browse /tools/opt
#
opt/top -type:=nfs;rfs:=/local/tools/opt;fs:=/nfs/${rhost}/tools/opt;\
sublink:=${karch} \
rhost:=fiche-202;
# Special rules for set opt
#
# Package set = sa
#
sa type:=auto;fs:=${map};
sa/* -type:=nfs;rfs:=/local/tools/sa;fs:=/nfs/${rhost}/tools/sa;\
sublink:=${karch}/${/key}; \
rhost:=fiche-202;
#
# a hack so you can browse /tools/sa
#
sa/top -type:=nfs;rfs:=/local/tools/sa;fs:=/nfs/${rhost}/tools/sa;\
sublink:=${karch} \
rhost:=fiche-202;
# Special rules for set sa
#
# Package set = db
#
db type:=auto;fs:=${map};
db/* -type:=nfs;rfs:=/local/tools/db;fs:=/nfs/${rhost}/tools/db;\
sublink:=${karch}/${/key}; \
rhost:=bfs;
#
# a hack so you can browse /tools/db
#
db/top -type:=nfs;rfs:=/local/tools/db;fs:=/nfs/${rhost}/tools/db;\
sublink:=${karch} \
rhost:=bfs;
# Special rules for set db
#
# end
The first section should look familiar by now: the /defaults section.
The remainder of the map file consists of rules describing quite a few different toolset hierarchies. The first one described is for the cs set, the largest and most frequently accessed.
The first rule is a new one:
cs type:=auto;fs:=${map};
Remember that the automount point for this map is /tools.
The KEY for the rule is cs and its type is one we haven't
seen yet: auto. The auto type is a special one that
indicates another automount-point. The fs:=${map} directive
is also a special one that tells the automounter to reference this map
for rules regarding the new automount point. (i.e. ${map} always
expands to be the current map.)
The next two rules have to do with the next-level automount-point /tools/cs. (i.e. /tools is the root automount-point and /tools/cs is the next level automount-point.) Here is the first of these two rules:
cs/* -type:=nfs;rfs:=/local/tools/cs;fs:=/nfs/${rhost}/tools/cs;\
sublink:=${karch}/${/key}; \
rhost:=fiche-202;
This (above) rule causes no end of grief for unsuspecting users who are
looking for things in /tools/cs. For example, suppose
someone was newly logged into
a machine that had been inactive for a while and decided to look for some
new tool called fuziwat. The first thing she might do is to cd into
the /tools/cs directory and run ls to see what's there:
egret % cd /tools/cs
egret % ls
gnu_util@
egret %
Woah! The innocent user is shocked to find basically nothing present
in the place where so many tools have been found before. Not understanding
the situation, the user will simply try another command that usually works
just fine. She does the following:
egret % /tools/cs/netscape/bin/netscape &
[1] 5894
egret %
And to her surprise it works just fine! Even more confused, she looks
again at the /tools/cs/ directory:
egret % ls /tools/cs
Acrobat@ bin@ gnu_util@ netscape@ netscape.site@
TechExplorer@ conv30@ jpi@ netscape-4.7@ perl5@
egret %
"Pretty wierd," is what the user starts thinking at this point. Now she
tries something else:
egret % ls /tools/cs/fuziwat
/tools/cs/fuziwat@
egret %
What? the new tool suddenly appears as well..
All of the above is the result of that one AMD rule shown just above. Let's look at it again and examine some pieces of it:
cs/* -type:=nfs;rfs:=/local/tools/cs;fs:=/nfs/${rhost}/tools/cs;\
sublink:=${karch}/${/key}; \
rhost:=fiche-202;
First there is the KEY: 'cs/*'. Remember that cs itself is an automount
point: that is why nothing is present in /tools/cs if AMD has just
started or the machine has been unused for a while (i.e. AMD unmounts
filesystems after some period of inactivity.) The '*' in the KEY, as
we saw above in the homedir maps, will match anthing because it is
a wildcard. Unfortunately for the innocent users, it will match
things that don't exist like fuziwat!
The LOCATION(S) has three new features. The first is the leading - (dash) which is used to indicate defaults that apply to all subsequent LOCATIONS for the specific rule. In the above example there is only one subsequent location: rhost:=fiche-202; The second feature is the sublink:=${karch}/${/key}; setting. The sublink uses the concept of the volume that was discussed at the start of this chapter. The variable ${karch} always expands to the current OS+hardware combination. Finally, the third feature is the / (forward-slash) that is hiding in the ${/key} variable. This slash may appear at the front of the variable name or the end. Here are the semantics of it:
${key} expands to (e.g.) cs/netscape
${/key} netscape
${key/} cs
For example, run the ls -l command as follows:
ecks % ls -l /tools/cs/netscape
lrwxrwxrwx 1 root wheel 43 Apr 6 20:11 /tools/cs/netscape ->
/nfs/fiche-202/tools/cs/x86+Linux2/netscape/
When we access /tools/cs/netscape, two references to the amdmap are
made. The first reference is for the KEY cs and the second is for
the KEY cs/netscape. Let's look in more detail at what happens
for the second reference. The rfs (the remote filesystem)
is /local/tools/cs and it is exported from the machine fiche-202.
The local NFS mountpoint fs is /nfs/fiche-202/tools/cs. You can
verify that for yourself:
ecks % cd /tools/cs/netscape
ecks % df .
Filesystem 1024-blocks Used Available Capacity Mounted on
fiche-202:/local/tools/cs
35208072 29185320 6022752 83% /nfs/fiche-202/tools/cs
Finally, AMD makes a symlink from ${path} (i.e. /tools/cs/netscape)
that points to ${fs}${sublink}. In the above example,
the ${sublink} expands into x86+Linux2/netscape.
The reason that arbitrary symlinks like fuziwat will appear in the /tools/cs directory is because AMD will happily make symlinks into the ${fs} (i.e. /nfs/fiche-202/tools/cs) with the assumption that the ${/key} will be there. This default rule works very well because it doesn't need to be changed when we add new packages to the tools tree.
The third rule for the cs toolset is a 'hack' that we added so that folks
would be able to view the entire toolset by using the special key cs/top.
cs/top -type:=nfs;rfs:=/local/tools/cs;fs:=/nfs/${rhost}/tools/cs;\
sublink:=${karch} \
rhost:=fiche-202;
Notice that the sublink in the rule doesn't actually include the KEY at
all. This rule simply points the user at the top-level directory of
the toolset hierarchy for the given ${karch}. Note well that this
forces the user to view the tools that are resident on the remote fileserver.
If you use the top keyword in your path then you are guaranteed to
be accessing the tools from the remote fileserver. This is important to
realize because many commonly used tools are actually cached locally
(called the local tools cache). Accessing local tools is much
faster than getting them from a remote server and if you specify top
in your path (e.g. /tools/cs/top/netscape/bin/netscape) then you will
not be using the local tools.
So, how does AMD deal with locally-cached tools? Each locally-cached tools needs a special rule in the local AMD map. If we went around to each machine and manually updated the local tools cache and the local AMD maps then we would lose all the administrative benefits of using AMD in the first place. Let's first look at a 'local' tools map and then I'll briefly explain how they are automatically generated.
The tools map that lives in /usr/local/etc/amdmaps is a generic map, in the sense that no local tools are represented in it. We have a Perl script called gen_tools_map (in the same directory, you can look at it on any lab machine) which examines the local cache and creates a local map that lives in /local/etc/amdmaps. The local cache, if there is one, lives in /local/tools/${set}/${karch} (for example, /local/tools/cs/sun4+SunOS5/).
imix % cd /tools/cs/netscape
imix % pwd
/local/tools/cs/sun4+SunOS5/netscape-4.7
The amdmap /local/etc/amdmaps/map.tools is built from the map of
the same name in the /usr/local/etc/amdmaps directory. It contains
everything its parent has, with the addition of rules for each 'package'
in the local cache. The local rules follow the three generic rules that
we looked at above. Here is a snippet from the file on imix:
cs type:=auto;fs:=${map};
cs/* -type:=nfs;rfs:=/local/tools/cs;fs:=/nfs/${rhost}/tools/cs;\
sublink:=${karch}/${/key}; \
rhost:=fiche-202;
#
# a hack so you can browse /tools/cs
#
cs/top -type:=nfs;rfs:=/local/tools/cs;fs:=/nfs/${rhost}/tools/cs;\
sublink:=${karch} \
rhost:=fiche-202;
cs/scripts type:=link;fs:=/local/tools/cs/sun4+SunOS5/../common/scripts;
cs/perl5 type:=link;fs:=/local/tools/cs/sun4+SunOS5/perl-5.004_04;
cs/tcl-7.4 type:=link;fs:=/local/tools/cs/sun4+SunOS5/tcl-7.4;
cs/emacs type:=link;fs:=/local/tools/cs/sun4+SunOS5/emacs-19.34;
cs/gnu type:=link;fs:=/local/tools/cs/sun4+SunOS5/gnu_util;
cs/perl5.site type:=link;fs:=/local/tools/cs/sun4+SunOS5/perl5.site;
The rules that point at the locally cached tools use a type of link
and the fs tells where the symlink should point.
Finally, let's take a quick look at the tools map for the research networks
in the CS department. This map is more complicated to account for the
fact that we have two different fileservers, each with multiple network
interfaces (nine different subnets total!). The rules simply have
more LOCATIONS in each rule (the number of rules is the same - three per
set for the generic map). Here is a chunk of the map for the cs set:
cs/* -type:=nfs;rfs:=/local/tools/cs;fs:=/nfs/${rhost}/tools/cs;\
sublink:=${karch}/${/key}; \
wire==grad-net.cs.colorado.edu;rhost:=fiche-205 \
wire==cu-cs-grad;rhost:=fiche-205 \
wire==srl-net.cs.colorado.edu;rhost:=stool-241a \
wire==cu-cs-srl;rhost:=stool-241a \
wire==capp-net.cs.colorado.edu;rhost:=fiche-242 \
wire==cu-cs-capp;rhost:=fiche-242 \
wire==servers-net.cs.colorado.edu;rhost:=fiche-243 \
wire==cu-cs-servers;rhost:=fiche-243 \
wire==csops-net.cs.colorado.edu;rhost:=fiche-192d \
wire==cu-cs-csops;rhost:=fiche-192d \
wire==db-net.cs.colorado.edu;rhost:=stool-199a \
wire==cu-cs-db;rhost:=stool-199a \
wire==admin-net.cs.colorado.edu;rhost:=stool-236a \
wire==cu-cs-admin;rhost:=stool-236a \
wire==serl-net.cs.colorado.edu;rhost:=stool-242b \
wire==cu-cs-serl;rhost:=stool-242b \
rhost:=fiche-fs rhost:=stool-241a
Each of the wire== lines is a single LOCATION in the rule.
And, again, the wire==<something> is called a selector and it acts
like an if statement: "if we are on a given subnet, then contact the given
hostname for NFS exports."
Notice, though
that the same subnet is represented in two different ways (i.e. with two
different LOCATION specifications in the rule.)
I'm not sure why this is necessary, but I venture to guess that it has to
do with different implementations or versions of AMD (for different OS types
perhaps) needing to use the same map.
The last two LOCATIONS in the rule are worth looking at more closely:
rhost:=fiche-fs rhost:=stool-241a
Neither LOCATION has a selector associated with it. The point of these
two locations is two-fold. First, if a given machine is on none of
subnets that are represented by the wire== selectors, then it will
try to obtain tools from the host fiche-fs which is simply a 100
megabit interface on our primary tools fileserver 'fiche' (the -fs
postfix on the name for the interface indicates that it is the Fast Servers
subnet - the 128.138.242.192/26). Secondly, if for some terrible act
of gods, the machine 'fiche' is unavailable, then the host stool-241a
will be contacted. This second point is a form of redundancy which
increases the guaranteed availability of tools at all times, even if
one of the machines goes down.
The command-line invocation for AMD is not something you ever want to actually type by hand! Take a look at this ps output:
coatl % ps auxww | grep amd
tor 1956 0.0 0.3 548 340 p8 DV+ 10:30PM 0:00.00 grep amd (tcsh)
root 10179 0.0 0.4 232 548 ?? S 29Mar00 0:18.76 amd -k i386+OpenBSD2 \
-x fatal,error -rp -c 1900 -w 1900 -t 8.110 -l syslog -a /tmp_mnt /Tools /usr/local \
/etc/amdmaps/Tools -cache:=inc,sync /tools /usr/local/etc/amdmaps/tools -cache:=inc,sync \
/csops /usr/local/etc/amdmaps/csops -cache:=inc,sync /home /usr/local/etc/amdmaps/home \
-cache:=inc,sync /homes /usr/local/etc/amdmaps/homes -cache:=inc,sync /logs /usr/local/ \
etc/amdmaps/logs -cache:=inc,sync /local/config /usr/local/etc/amdmaps/local.config \
-cache:=inc,sync /local/sys /usr/local/etc/amdmaps/local.sys -cache:=inc,sync /cs/research \
/usr/local/etc/amdmaps/research -cache:=inc,sync /cs/ftp /usr/local/etc/amdmaps/ftp \
-cache:=inc,sync /src /usr/local/etc/amdmaps/src -cache:=inc,sync /srl /usr/local/etc \
/amdmaps/srl -cache:=inc,sync /rd /usr/local/etc/amdmaps/rd -cache:=inc,sync /discovery \
/usr/local/etc/amdmaps/discovery -cache:=inc,sync
The command line consists of options followed by any number
of automount-point, amdmap, -map-options combinations
(three space-separated items).
Here is a description of the options used above:
-k <arch> determines the value of ${karch} in the amdmaps
-x fatal,error a comma-separated list of SYSLOG levels
-r tells AMD to restart any existing NFS mounts (so AMD
can control them)
-p tells AMD to echo its PID to STDOUT (so it can be
saved in a file like '/etc/amd.pid' or something)
-c 1900 this is the time in seconds that a given mount will
remain cached (i.e. active) before being unmounted
(if it is not being used)
-t 8.110 this argument is used to control the timeout and
retransmit periods for NFS (and related) protocols
-l syslog tells AMD how to log; one can specify a filename or
simply 'syslog' to have it use that instead
-a /tmp_mnt used to tell AMD the default NFS mountpoint directory
Then you have a series of the three-part combinations; for example:
/tools /usr/local/etc/amdmaps/tools -cache:=inc,sync
There is one three-part set for each different automount-point and amdmap.
We use a Bourne shell script called startamd to fire up the AMD process at boot time. The script lives in the /usr/local/etc/amdmaps directory. Take a look at it if you interested in seeing how we choose which maps to use and so on..
The amq command is used to get status information about current the AMD process. For example,
coatl % amq
/ root "root" coatl:(pid10179)
/csops toplvl /usr/local/etc/amdmaps/csops /csops
/homes toplvl /usr/local/etc/amdmaps/homes /homes
/local/config toplvl /usr/local/etc/amdmaps/local.config /local/config
/src toplvl /usr/local/etc/amdmaps/src /src
/discovery toplvl /usr/local/etc/amdmaps/discovery /discovery
/cs/ftp toplvl /usr/local/etc/amdmaps/ftp /cs/ftp
/cs/research toplvl /usr/local/etc/amdmaps/research /cs/research
/rd toplvl /usr/local/etc/amdmaps/rd /rd
/Tools toplvl /usr/local/etc/amdmaps/Tools /Tools
/tools toplvl /usr/local/etc/amdmaps/tools /tools
/home toplvl /usr/local/etc/amdmaps/home /home
/logs toplvl /usr/local/etc/amdmaps/logs /logs
/local/sys toplvl /usr/local/etc/amdmaps/local.sys /local/sys
/srl toplvl /usr/local/etc/amdmaps/srl /srl
/home/trouble nfs suod:/tmp_mnt/home/trouble /tmp_mnt/home/trouble
/home/fcsk nfs fcsk.cs.colorado.edu:/tmp_mnt/home/fcsk /tmp_mnt/home/fcsk
/tools/cs auto /usr/local/etc/amdmaps/tools /tools/cs
/tools/cs/mh-6.8.3 nfs fiche-fs:/local/tools/cs /nfs/fiche-fs/tools/cs/i386+OpenBSD2/mh-6.8.3
/tools/cs/mh nfs fiche-fs:/local/tools/cs /nfs/fiche-fs/tools/cs/i386+OpenBSD2/mh
/tools/cs/bin nfs fiche-fs:/local/tools/cs /nfs/fiche-fs/tools/cs/i386+OpenBSD2/bin
By itself with no arguments, amq produces a list that includes toplvl automount-points
as well as NFS mounts and others. The first field is the name of the automount-point or
the active NFS mount. Column two is the type of the mount. Column three shows
aither the amdmap associated with the mountpoint or the location of the NFS server.
The last column shows either the automount-point (again!) or the local mountpoint
for the NFS filesystem.