The cron daemon is one of the servers that gets started at boot. Its job is to run periodically scheduled processes known as cron jobs. Every user can create their own crontab file detailing what processes they want run and exactly when to run them.
Run the crontab program to edit or list out a crontab file. For example,
% crontab -e
The above would load your own crontab file (if any) into a temp-file and
invoke your favorite editor on it for you (i.e. defined by the environment
variable EDITOR ). The crontab file consists of single-line entries
with (in general) six fields. The first five specify when to run the
job, the rest of the entry defines the job to be run. The first five fields
are minute, hour, day, *month*, and day-of-week in that order.
The values these fields may take are:
minute 0-59
hour 0-23
day 1-31
month 1-12
day-of-week 0-6 (0 = Sunday)
Another value that may be used is an asterisk which matches any value
that the field can take. Integer values may given as comma-separated
values (discreet) or dash-separated values (ranges, inclusive).
Here is an example crontab entry:
# delete old mail that's more than a week old
0 3 * * * find $HOME/Mail -name '#*' -mtime +7 -exec rm {} \;
(Note that comments are allowed in the crontab file.) The above crontab
entry runs every night at three in the morning. Its purpose is to
use the (powerful) find command to locate and remove messages from my
mailbox directories that have been previously deleted from the mail system
and are a week old. (Deleted mail messages simply get renamed from a number
[indicating its position in a given mail-folder] to have a hash-mark - # -
in front of the number).
When cron jobs are actually run, they do so with the UID of the owner of the crontab file. Root's cron jobs run as root. Commands specified in the crontab file as parsed with the Bourne shell, so be aware of that too.
The crontab files are typically stored in a directory somewhere in /var although the specific location varies by OS. OpenBSD puts them in the /var/cron/tabs directory, while RedHat puts them in /var/spool/cron. RedHat also has a bunch of crontab stuff in the /etc directory. Notably, there is an /etc/crontab file (this is root's crontab file on RedHat) as well as three special subdirectories that have files in them denoting jobs to run. The subdirs are cron.daily cron.weekly and cron.monthly and you can guess how often each of those directories' job files are processed.
Conr on various OS's has additional configurable behavior. Read the relevant manpages on your OS to find out the differences.
An important part of administering a machine is keeping track of the various diagnostic, error and informative messages that are generated by the kernel and many of the daemon services. Unix provides a general logging facility called syslog that other applications can use to log information in a consistent manner.
Applications make use of C library functions (see syslog(3) on OpenBSD) to send relevant messages to a special system logging device (normally /dev/log - this is actually just a Unix Domain Socket, similar in concept to a networking Socket, read a good C manual for details). The syslogd(8) daemon reads this device and based on its configuration (specified in /etc/syslog.conf ) writes the messages to various log files.
The basic format of an entry in /etc/syslog.conf is as follows:
selector <tab> action
The two fields must be separated by at least one <tab> character.
Selectors are typically given in the form facility.level
and the Action is typically either a filename or a loghost.
Here is a table of the usual facility types:
facility programs which use it
-------- ---------------------
kern the kernel
user "user" processes (no specific)
mail sendmail
daemon "system" daemons, such as 'routed'
auth security and authorization-related commands (e.g. SSH)
lpr BSD line-printer daemon
news usenet news system
uucp for the UUCP service
cron the cron daemon
syslog used by syslogd itself
ftp for ftp services
authpriv similar to the 'auth' facility, but logged to a secure file
local0 - local7 used for local/other daemons
* the '*' indicates 'all' facilities
And here is a table of the syslog severity levels:
Level Description
----- -----------
emerg for 'panic' situations
alert for urgent, less than panic, situations
crit for 'critical' conditions
err for other error conditions
warning for warning (non-error) messages
notice for non-error conditions that may require attention
info for informational messages
debug for debugging messages
none special level used to disable logging of a facility
And finally, a list of possible actions:
Action Description
------ -----------
<filename> Must begin with a '/', indicates a log file to which
the message must be written
@<hostname> Indicates that the messages should be forwarded over
the 'net to the specified host
@<ipaddress> Same deal as @hostname
user1[,user2] A comma-seperated list of users; if an indicated
user is logged in, then the message is WRITTEN to
their TTY (NOT sent in email as I lied to you)
* Indicates the message should be written (using WALL)
to ALL users logged in
Selectors may also combined in a number of different ways.
Here are a few examples:
facility.level action
facility1,facility2.level action
facility1.level1;facility2.level2 action
*.level action
*.level;badfacility.none action
Basically, the selectors present in a given entry are ORed
together. If a message matches a given selector, then the
associated action is taken.
A given selector may contain multiple, comma-seperated
facilities. Multiple selectors may combined on a single line
with semi-colons.
Messages may match multiple lines. This can be annoying, as you will end up with multiple instantiations of the same log message!
When a given level is specified, all levels upto and including the specified level are logged. For example, if the level were specified as warning, then the levels emerg, alert, crit, and err are included.
Okay, now let's look at some example /etc/syslog.conf files.
Both examples come from CS dept hosts. The first is a standard (client)
file that we use on bascially every machine. Each machine logs
some things locally but forwards everything to a central loghost
for security reasons. The central log host has a different conf file.
Here is the client file:
#
# CS Department syslog.conf file
#
# servers need mail debug
mail.debug /var/log/maillog
# we want to keep all the printer stuff local
lpr.debug /var/log/lpd-errs
# if there is an emergency, everyone should hear about it
*.emerg;user.none *
#
# log important stuff locally too, just because we're lazy
*.warning;auth.info /var/log/messages
# we want to send the following stuff to netloghost for
# central logging and netlog viewing
*.warning;mail,lpr,daemon,auth,local0,local1,local2,local7.none @netloghost
daemon,auth.info;mail.notice @netloghost
# sudo
local2.debug @netloghost
# named
local3.debug @netloghost
# tcpd
local7.debug @netloghost
# status and popper
local0.debug @netloghost
# cardd syslogs to local1 - direct to boulder for safe keeping
local1.debug @netloghost
And here is the master file:
##
# From everyone else including us via netloghost
#
# $Id: class04.txt,v 1.3 2001/02/11 00:37:45 tor Exp tor $
##
# main logs
#*.err;kern.debug;mark.debug;auth.notice /dev/console
#
kern.debug;mail.crit;auth,daemon.notice /var/log/messages
*.err;mail,auth,daemon,user.none /var/log/messages
#
user.warning /var/log/user
# csops main logging facility via netlog
*.warning;mail,lpr,daemon,auth,local0,local1,local2,local7.none netlog
auth.info;local2.debug;daemon,local3.notice;local7.err netlog
# cardd syslogs to local1
local1.debug /var/log/card
# auth stuff (including npasswd)
auth.info /var/log/auth
# log policy agreement signings
local4.info /var/log/policy
# cisco logs
local5.debug /var/log/cisco
# tcpd
local7.debug /var/log/tcp
lpr.debug /var/log/lpd-errs
mail.debug /var/log/maillog
# if there is an emergency, everyone should hear about it
#*.emerg;user.none;local2.none *
# netblazer (logs to local6, tagged with "syslog")
!syslog
local6.debug /var/log/netblazer
# popper (logs to local0)
!popper
local0.info /var/log/popper
# ssh (logs to auth)
!sshd
auth.debug /var/log/ssh
# named (logs to local3
!named
local3.debug /var/log/named
# sudo (logs to local2 or auth)
!sudo
*.* /var/log/sudo
Our master netlog host is running the OpenBSD syslogd which
has an advanced tagging system that is used to extend the
(rather) limited facility list. A tag is included in the
message sent via syslog; the syslogd looks for the specified
tag in the message.
# rotate log files every hour, if necessary
0 * * * * /usr/bin/newsyslog
Here is an example:
#
# configuration file for newsyslog, size is in KB, time is in hours
# This file is under RCS on suod:/local/etc/newsyslog.conf.i386+OpenBSD2
# Changes made elsewhere will be overwritten!
# $Id: class04.txt,v 1.3 2001/02/11 00:37:45 tor Exp tor $
#
# logfilename owner.group mode ngen size time [ZB]
/var/log/lpd-errs daemon.daemon 640 7 50 * Z
/var/log/maillog root.Trouble 640 7 * 24 Z
/var/log/syslog.nonet root.Trouble 640 7 50 * Z
/var/log/messages root.Trouble 640 7 100 * Z
#
# OpenBSD specific
/var/cron/log root.wheel 600 3 10 * Z
/var/log/wtmp root.wheel 644 7 * 168 ZB
/var/log/ipflog root.wheel 640 3 10 * Z
/var/log/aculog uucp.dialer 640 7 10 * Z
The headings for each column in this file should be fairly
self-evident. The owner.group and mode fields specify
what the permissions information for each logfile should be.
The ngen field tells newsyslog how many generations should
be kept for the given log file. That is, as logfiles are rotated,
this number of previous files will be kept around, while any older
ones will be deleted.
You may specify that logfiles should be rotated when they reach a given size (in KiloBytes) or after a certain number of hours have elapsed.
The final column specifies what sort of compression should be applied to the logfile after it has been rotated out of use.
A few more items to be aware of:
There are two main types of disks: IDE and SCSI . IDE, or Integrated Device Electronics, a.k.a ATA drives are currently the PC standard. They are cheaper to manufacture than SCSI drives, and for a single user workstation, the cheap IDE disk could easily out perform a cheap or old SCSI system. Newer SCSI though still out performs, especially under load, like a web-server.
Sun computers have traditionally been SCSI based, although this has changed slightly with Sun's Ultra-5 and -10 desktop series, which have more "off the shelf" PC hardware.
Let's look at the two interface types briefly:
Most modern PC motherboards have two IDE interfaces on them, a primary interface and a secondary one. Each interface can support two devices, commonly called master and slave devices. There is usually a jumper on the device that is used to determine whether it should behave as a master or slave device.
New IDE disks and old BIOSes don't get along. As all the PC boot-related manpages probably said also, there is a 1024 cylinder limit in the old BIOS standard, so that one could only keep a bootable kernel image within the first 1024 cylinders of the disk. The 10 Gigabyte Western Digital drive in saclass-3 has 16383 cylinders, but notice (below) that the BIOS reports it as 1313 cylinders. Some BIOSes will fudge the drive geometry to fit the old standard but represent more storage capacity. When we talk about the fdisk program, we'll look at the 1024 cylinder problem again.
There is a comprehensive IDE FAQ available from:
Now SCSI devices. While most PC motherboards have IDE interfaces on them, for SCSI, you need a separate controller card. Older SCSI cards were capable of supporting 8 devices (one of them being the controller card), while newer SCSI implementations can support 16 devices. There are a number of different styles of SCSI connectors, and many sorts of adapters to switch between the styles. SCSI devices also have jumpers that can be used to set the SCSI ID of the device. Older devices will have a set of three jumpers, which are designated as ON or off:
Jumper 1 J2 J3 SCSI ID
off off off 0
ON off off 1
off ON off 2
ON ON off 3
off off ON 4
ON off ON 5
off ON ON 6
ON ON ON 7
The newer devices will have a fourth jumper to enable device ID's 8-15 to
be selected in a similar fashion.
Many external SCSI devices will have some sort of thumb-wheel that can be used to set the SCSI ID much more easily, simply by moving the wheel to the correct ID.
Another important issue with SCSI is termination of the device chain. Many devices will have another jumper on them which will enable termination when the jumper is in place. Termination is achieved using resistors to provide impedence matching . Older SCSI drives actually came with small resistor packs that you could insert into the disk's circuit board. Improper termination of a SCSI chain can cause signal reflection and thus data corruption, and can even result in devices not being recognized at all.
SCSI disks may have other jumpers as well controlling a number of features like write-protect and motor-start . To get the specific info for you disk, you should visit the manufacturer's website and do a search.
There is an informative FAQ on SCSI and a few other interface types:
In order to use a disk at all, it and its controller must both be recognized by the kernel. In order for the controller to be recognized, there must be a driver for it in the kernel. The generic OpenBSD kernel comes with most common SCSI drivers enabled. In Linux land, drivers may be compiled directly into the kernel or they may be compiled as modules which are loaded either at boot time or simply when they are needed. RedHat uses the loadable modules for the installation and its default kernel. One reason modules are nice is that the kernel itself takes up less space in RAM, with only the drivers loaded that are necessary for your machine. For example, on my desktop machine coatl the difference in size between the generic kernel and one which doesn't have many extra drivers compiled in is quite large (around a Megabyte smaller):
coatl % ls -l bsd*
-rwxr-xr-x 1 root wheel 1890751 Dec 10 14:19 bsd*
-rw-r--r-- 1 root wheel 2832934 Oct 28 01:10 bsd.old
To see if your disk is recognized, you need to look at the dmesg output, or watch the messages that appear when your kernel is booting and doing the device probe. Here are the relevant bits from dmesg on saclass-1 (the OpenBSD sparc):
sbus0 at iommu0: clock = 25 MHz
dma0 at sbus0 slot 15 offset 0x400000: rev 2
esp0 at dma0 offset 0x800000 pri 4: ESP200, 40MHz, SCSI ID 7
scsibus0 at esp0: 8 targets
sd0 at scsibus0 targ 0 lun 0: <MAXTOR, P0-12S, JB21> SCSI1 0/direct fixed
sd0: 997MB, 1795 cyl, 15 head, 75 sec, 512 bytes/sec, 2042265 sec total
sd1 at scsibus0 targ 1 lun 0: <FUJITSU, M2654S-512, 010P> SCSI2 0/direct fixed
sd1: 1959MB, 2179 cyl, 21 head, 87 sec, 512 bytes/sec, 4014054 sec total
sd2 at scsibus0 targ 2 lun 0: <MAXTOR, P0-12S, JB21> SCSI1 0/direct fixed
sd2: 997MB, 1795 cyl, 15 head, 75 sec, 512 bytes/sec, 2042265 sec total
sd3 at scsibus0 targ 3 lun 0: <SEAGATE, ST32430W SUN2.1G, 0508> SCSI2 0/direct f
ixed
sd3: 2049MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sec, 4197405 sec total
st0 at scsibus0 targ 6 lun 0: <EXABYTE, EXB-8200, 2618> SCSI1 1/sequential remov
able
st0: density code 0x0, 1024-byte blocks, write-enabled
You can see that this system is SCSI based. The first four lines above show the controller details. It is an 'esp' controller (you can read about it on saclass-1: man esp(4)) - this is an NCR chipset found on lots of older Sparc hardware. You can see also that the controller is assigned scsi ID 7, the highest priority ID. Following that are details about five other scsi devices, four disks and one tapedrive. It is important to note that the tag out front (to the left), like sd0 for example is NOT the SCSI ID, the scsi ID is shown further in the line and it is called a targ (for target). The lun is a further subdivision that is not commonly used. Other information about each device is also given, including size and geometry. Two other drivers that are being used in this example have associated manpages as well; they are sd(4) and st(4) .
The difference between the SCSI ID of the device and the id number assigned to the device by the operating system is an important one to note. There isn't necessarily any correlation. For OpenBSD, the rule is: the first scsi device found by the kernel is sd0, the second is sd1 and so on. For example, if the first scsi target found is target 3, it will still be mapped to sd0 because there are no lower target IDs used... (This is not the case in the above dmesg snippet from the saclass-1 machine.)
The issue is further complicated by the fact that (for arcane, historical reasons) old Sparc hardware maps scsi id 0 to be id 3 and id 3 to be id 0. Basically, you just need to pay attention to what the actual devices are and what your operating system wants to refer to them as. Dmesg output is the most useful source to figure that sort of thing out.
An example of a scsi device that has more than one lun is a multi-disc CDROM. Here is some dmesg output from a SunOS4 box that has a Nakamichi 7-CD changer on it:
sr1: Unrecongized Vendor 'NRC', product 'MBR-7' sr1 at esp0 target 5 lun 0
sr2: Unrecongized Vendor 'NRC', product 'MBR-7' sr2 at esp0 target 5 lun 1
sr3: Unrecongized Vendor 'NRC', product 'MBR-7' sr3 at esp0 target 5 lun 2
sr4: Unrecongized Vendor 'NRC', product 'MBR-7' sr4 at esp0 target 5 lun 3
sr5: Unrecongized Vendor 'NRC', product 'MBR-7' sr5 at esp0 target 5 lun 4
sr6: Unrecongized Vendor 'NRC', product 'MBR-7' sr6 at esp0 target 5 lun 5
sr7: Unrecongized Vendor 'NRC', product 'MBR-7' sr7 at esp0 target 5 lun 6
The format of the SunOS4 dmesg is a little different, but you can see that this
is probably another Sparc (because it has the esp driver and SunOS4 wasn't
really ever ported to anything else). The device has scsi ID 5 and 7 luns,
one for each CD slot.
Now how about an OpenBSD PC machine:
pciide0 at pci0 dev 7 function 1 "Intel 82371SB (Triton II) IDE" rev 0x00: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility
atapiscsi0 at pciide0 channel 1
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <MITSUMI, CD-ROM FX600S !B, P01> SCSI0 5/cdrom removable
cd0: can use 32-bit, PIO mode 3, DMA mode 1
pciide0: channel 1 interrupting at irq 15
cd0(pciide0:1:0): using PIO mode 0, DMA mode 1 (using DMA data transfers)
ahc1 at pci0 dev 20 function 0 "Adaptec AHA-2940" rev 0x00: irq 11
ahc1: aic7870 Single Channel, SCSI Id=7, 16 SCBs
scsibus1 at ahc1: 8 targets
ahc1: target 0 synchronous at 10.0MHz, offset = 0xf
sd0 at scsibus1 targ 0 lun 0: <SEAGATE, ST51080N, 0943> SCSI2 0/direct fixed
sd0: 1030MB, 4826 cyl, 4 head, 109 sec, 512 bytes/sec, 2109840 sec total
ahc1: target 3 synchronous at 10.0MHz, offset = 0xf
sd1 at scsibus1 targ 3 lun 0: <MICROP, 2217-15MQ1001901, HQ30> SCSI2 0/direct fixed
sd1: 1685MB, 2372 cyl, 15 head, 96 sec, 512 bytes/sec, 3450902 sec total
Notice that the PC has an IDE interface but that it also has a SCSI
controller as well; there is something else strange here too: the ATAPI CDROM
which hangs off the IDE interface is actually treated like a SCSI device.
ATAPI CDROM devices actually use a command set that is derived from the
SCSI command set, and although the drive has an IDE interface, most unix
systems I have seen use a SCSI driver to talk to them.
The scsi controller on this PC is ahc(5) , and you can see that it is
an "Adaptec AHA-2940". Notice that the PC dmesg gives you some additional
information such as the IRQ used by the device.
Let's look more closely at the SCSI id numbers. Notice that the two SCSI disks on this system have ID's (targets) 0 and 3. The OpenBSD kernel maps these drives to sd0 and sd1 respectively. This can cause you a headache if, for example, you have only one drive on your system, and it is SCSI ID 3, which OpenBSD maps to sd0 ; and your system also boots off of this drive. Now suppose you add a new drive to the system with SCSI ID 1. When the system reboots, the disk with ID 1 will now be mapped to sd0 , while the original drive gets mapped to sd1 . Now, the kernel gets loaded properly, but then the system can't find its filesystems because it expects them to be living on sd0! We'll discuss this a bit more when we look at the /etc/fstab file.
Now how about an IDE only PC. This one is a RedHat PC:
VP_IDE: IDE controller on PCI bus 00 dev 39
VP_IDE: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings: hda:DMA, hdb:pio
ide1: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdc:pio, hdd:DMA
hda: Maxtor 91080D5, ATA DISK drive
hdd: ATAPI CDROM, ATAPI CDROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
hda: Maxtor 91080D5, 10300MB w/512kB Cache, CHS=1313/255/63
hdd: ATAPI 40X CD-ROM drive, 128kB Cache
This machine has one IDE disk on the primary IDE interface and on CDROM
on the secondary interface. The RedHat Linux dmesg output is different
from the BSD dmesg, but you can see that a lot of the same information is
present include disk sizes and geometry, as well as relevant IRQs.
Note also that the disk is the master device on the first IDE interface,
while the CDROM is showing up as a slave device on the second interface
even though there is no master device. Most likely the cdrom has been
jumpered to be a slave, even though it is the only device on the interface.
Generally, it is prefereable to make it the master device if there is only
one, but fortunately this configuration works too!
Traditionally, filesystems in the unix world were called ufs partitions standing for Unix File System. OpenBSD today uses a new version of the ufs partition that is called ffs or Fast File System. Most versions of unix that run on PC hardware today are also capable of mounting NON unix filesystems like the ISO9660 filesystem that is used on most CDROMs and the FAT filesystem (and its relatives) from the DOS world. Linux uses its own filesystem type that is known as ext2 (This is version two of the linux 'extended' filesystem which derives from minix ).
There are a number of steps necessary to creating a filesystem under unix. If you are using a PC you first need to use fdisk(8) to create at least one fdisk partition. Subsequent steps depend on the operating system you are using, but many of the concepts are similar. For most BSD style systems the next step is the disklabel(8) program where you set up another kind of partition table. SunOS and Solaris call this program format(8) .
Following the creation of usable partitions, either with just fdisk in the Linux case, or with fdisk and disklabel in the PC / BSD case, you need to create the actual filesystems using the newfs(8) program, or mke2fs(8) under Linux.
Now the newly created filesystem can be mounted using the mount(8) command. Typically the root filesystem is called / (slash). Within that filesystem are mountpoints for other filesystems, again, typically these are just directories. Another common filesystem is /usr and its mountpoint is just the directory /usr. When you use the df(1) command you are shown A list of currently mounted filesytems:
saclass-1 % df
Filesystem 512-blocks Used Avail Capacity Mounted on
/dev/sd3a 63198 37090 22950 62% /
/dev/sd3d 253402 10558 230174 4% /var
/dev/sd3e 991790 316990 625212 34% /usr
/dev/sd3f 909690 61096 803110 7% /local
The filesystem is listed on the leftmost column, the mountpoint on the
rightmost column. Notice the default block-size is 512 bytes. To
get the output in 1024 byte blocks on OpenBSD, use the -k argument to
the df command:
saclass-1 % df -k
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/sd3a 31599 18545 11475 62% /
/dev/sd3d 126701 5280 115086 4% /var
/dev/sd3e 495895 158495 312606 34% /usr
/dev/sd3f 454845 30548 401555 7% /local
The fdisk program derives from the DOS/PC world. Nowadays each OS that runs on a PC has its own version and they're all different. The program is necessary because the BIOS needs to have the disk organized in a certain way in order for it to boot an operating system.
First is the MBR or Master Boot Record. You probably read a lot about it in /usr/doc/lilo-0.21/README on saclass-[34]. If you think of the disk as one long array of bytes, the MBR takes up the first 512 bytes of that array. Contained in it is the fdisk or DOS partition table . This table has four slots in it. If you are running OpenBSD, you can get away with defining only one fdisk partition that spans the whole disk. If you are dual-booting your PC and want to run Win 9X and some version of unix then you need at least two fdisk partitions, one for each operating system. Each entry in the partition table defines the starting and ending points of that partition, as well as the type of the partition. Let's look at some examples from running fdisk on an OpenBSD PC.
For these examples, I will be using the machine saclass . First, recall from dmesg what disks we have (I actually added a new one since the dmesg we talked about earlier, so that I could use the fdisk program):
saclass % dmesg | grep ^sd
sd0 at scsibus1 targ 0 lun 0: <SEAGATE, ST51080N, 0943> SCSI2 0/direct fixed
sd0: 1030MB, 4826 cyl, 4 head, 109 sec, 512 bytes/sec, 2109840 sec total
sd1 at scsibus1 targ 1 lun 0: <MICROP, 1924-21MZ1077810, HZ2P> SCSI1 0/direct fixed
sd1: 2000MB, 2280 cyl, 21 head, 85 sec, 512 bytes/sec, 4096656 sec total
sd2 at scsibus1 targ 3 lun 0: <MICROP, 2217-15MQ1001901, HQ30> SCSI2 0/direct fixed
sd2: 1685MB, 2372 cyl, 15 head, 96 sec, 512 bytes/sec, 3450902 sec total
sd3 at scsibus0 targ 3 lun 0: <SEAGATE, ST32430W SUN2.1G, 0508> SCSI2 0/direct f ixed
sd3: 2049MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sec, 4197405 sec total
You can look at the output of df to see which disks are being used:
saclass % df
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/sd3a 64607 21924 39453 36% /
/dev/sd3f 79431 4504 70956 6% /var
/dev/sd3g 577399 3634 544896 1% /home
/dev/sd3h 1162798 194826 909833 18% /usr
Now, by default, fdisk will simply output the current fdisk partition table:
saclass % sudo fdisk sd0
Disk: sd0 geometry: 131/255/63 [2104515 sectors]
Offset: 0 Signatures: 0xAA55,0x0
Starting Ending
#: id cyl hd sec - cyl hd sec [ start - size]
-------------------------------------------------------------------------
0: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
1: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
2: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
*3: A6 0 1 1 - 130 254 63 [ 63 - 2104452] OpenBSD
Here we see that only one partition is defined, and its type, surprise,
is OpenBSD. The starting and ending points are given in Cylinder, Head,
Sector geometry . You also never want to start a partition on
Cylinder 0, Head 0, Sector 0 because it will overwrite the MBR !
If you instead want to use fdisk interactively and, perhaps, write a new fdisk label in the MBR on the disk, then you need to use an inactive disk. The program might let you do it anyway, though, so you MUST be careful. That's why you check the df output or the /etc/fstab file (see below) first. If you mess up the fdisk partition table of a running disk you are in serious trouble, and may end up having to re-install the entire system! So always be careful. Notice from the df output above, the sd0 disk contains most of the operating system, not to mention the kernel itself in / . One disk that isn't being used is sd1 so we'll use that now. (And once you get sudo permissions going, I encourage you to do this on your own as well!).
To start up fdisk in interactive, or edit mode, use the -e argument. This time we'll use sd1 which is a 2 Gigabyte Maxtor drive. (It's an old 5+1/4" full height drive, that would barely fit in modern PC :).
saclass % sudo fdisk -e sd1
Enter 'help' for information
fdisk: 1> print
Disk: sd1 geometry: 255/255/63 [4096575 sectors]
Offset: 0 Signatures: 0xAA55,0x0
Starting Ending
#: id cyl hd sec - cyl hd sec [ start - size]
-------------------------------------------------------------------------
0: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
1: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
2: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
*3: A6 0 1 1 - 254 254 63 [ 63 - 4096512] OpenBSD
fdisk:*1> help
help Command help list
manual Show entire OpenBSD man page for fdisk
reinit Re-initialize loaded MBR (to defaults)
disk Edit current drive stats
edit Edit given table entry
flag Flag given table entry as bootable
update Update machine code in loaded MBR
select Select extended partition table entry MBR
print Print loaded MBR partition table
write Write loaded MBR to disk
exit Exit edit of current MBR, without saving changes
quit Quit edit of current MBR, saving current changes
abort Abort program without saving current changes
fdisk:*1>
One thing to note is the huge number of fdisk partition types that are
possible. To change any of the four partitions, use the edit command
with a partition number as an argument (0 - 3):
fdisk: 1> e 3
Starting Ending
#: id cyl hd sec - cyl hd sec [ start - size]
-------------------------------------------------------------------------
*3: A6 0 1 1 - 254 254 63 [ 63 - 4096512] OpenBSD
Partition id ('0' to disable) [0 - FF]: [A6] (? for help) ?
Choose from the following Partition id values:
00 unused 01 DOS FAT-12 02 XENIX / 03 XENIX /usr
04 DOS FAT-16 05 Extended DOS 06 DOS > 32MB 07 HPFS/QNX/AUX
08 AIX fs 09 AIX/Coherent 0A OS/2 Bootmgr 0B Win95 FAT-32
0E DOS FAT-16 10 OPUS 12 Compaq Diag. 40 VENIX 286
50 DM 51 DM 52 CP/M or SysV 54 Ontrack
56 GB 61 Speed 63 ISC, HURD, * 64 Netware 2.xx
65 Netware 3.xx 75 PCIX 80 Minix (old) 81 Minix (new)
82 Linux swap 83 Linux files* 93 Amoeba file* 94 Amoeba BBT
A5 FreeBSD A6 OpenBSD A7 NEXTSTEP A9 NetBSD
B7 BSDI filesy* B8 BSDI swap DB CPM/C.DOS/C* E1 Speed
E3 Speed E4 Speed EB BeOS/i386 F1 Speed
F2 DOS 3.3+ Sec F4 Speed FF BBT
Partition id ('0' to disable) [0 - FF]: [A6] (? for help) 0
Partition 3 is disabled.
fdisk:*1> print
Disk: sd1 geometry: 255/255/63 [4096575 sectors]
Offset: 0 Signatures: 0xAA55,0x0
Starting Ending
#: id cyl hd sec - cyl hd sec [ start - size]
-------------------------------------------------------------------------
0: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
1: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
2: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
3: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
fdisk:*1>
The partition types are represented in hexidecimal (base sixteen).
Now let's add a new partition in slot 0, this one will be another OpenBSD
partition and only use half the disk:
fdisk:*1> e 0
Starting Ending
#: id cyl hd sec - cyl hd sec [ start - size]
-------------------------------------------------------------------------
0: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
Partition id ('0' to disable) [0 - FF]: [0] (? for help) A6
Do you wish to edit in CHS mode? [n]
Partition size [1 - 4096575]: [0] 2048287
fdisk:*1> print
Disk: sd1 geometry: 255/255/63 [4096575 sectors]
Offset: 0 Signatures: 0xAA55,0x0
Starting Ending
#: id cyl hd sec - cyl hd sec [ start - size]
-------------------------------------------------------------------------
0: A6 0 0 1 - 127 127 31 [ 0 - 2048287] OpenBSD
1: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
2: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
3: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
Editing in CHS mode allows you to specify the exact Cylinder/Head/Sector
boundaries. You can instead simply enter the number of sectors .
Notice that the starting point leaves the the first sector (0) alone.
Most versions of unix have their own concept of partition that will be explained in the next section on the disklabel command. Linux, however, just uses the fdisk partitions directly. The Linux fdisk program is quite a bit different from the OpenBSD version. Basically, though, you have similar functionality. Key differences are that the OpenBSD fdisk has such a wide range of partition types and it is very robust. I use an OpenBSD install floppy frequently just because I can get a usable shell from which I can run fdisk.
Another difference that is important for linux is that it lets you edit the partition tables in extended partitions as well. The extended partition is one of the basic fdisk partition types (its hex code is 05) and it has a special property: it has its own fdisk partition table, very similar to the MBR, except it resides at the very start of the extended partition, and not at the very start of the disk. You can define a great number of logical partitions within a single extended partition. Linux uses each of these logical partitions as well as primary fdisk partitions for its filesystems. Linux has two fdisk partition types that can be seen in the list above. They are: 82 Linux swap, and: 83 Linux filesystem (meaning an ext2 filesystem, in particular).
Sometimes the MBR of a disk gets messed up but the rest of the disk is actually fine. If you have a printout of what the partition table looked like, you can use a boot floppy that will allow you to run fdisk to recreate a new MBR.
Non-Linux versions of unix use a different approach to disk partitions. The reason for this is that the Unix File System structure has been in use since long before DOS and fdisk. As we saw above, all of the saclass machine's filesystems lie within a single fdisk partition. Sometimes people call these unix filesystem partitions slices in order to distinguish between them and fdisk partitions (notably Solaris and FreeBSD). I don't like the term 'slice' for some reason, so I will always distinguish by calling unix partitions 'partitions' and fdisk partitions 'fdisk partitions'. NON fdisk partition tables are sometimes called BSD partition tables. They are also refered to disk labels .
The disklabel(8) program is used in OpenBSD to edit the BSD partition table. In certain respects the process is similar to that described above for the fdisk program: you are still dealing with disk geometry and defining the beginning and ending points of each partition. One convention that survives from the early days is that the third partition is always defined to represent the entire disk. BSD systems refer to their partitions by letter, so partition 'c' is always the whole disk.
Like the fdisk program, you need to be careful that you don't stomp on a running disk. Always compare the dmesg information with df or /etc/fstab to be safe.
Let's look at our two OpenBSD machine's disk labels. Also like the fdisk program, with no additional arguments disklabel just outputs the current label.
saclas2 % sudo disklabel sd0
Password:
# using MBR partition 3: type A6 off 63 (0x3f) size 2104452 (0x201c84)
# /dev/rsd0c:
type: SCSI
disk: SCSI disk
label: ST51080N
flags:
bytes/sector: 512
sectors/track: 109
tracks/cylinder: 4
sectors/cylinder: 436
cylinders: 4826
total sectors: 2109840
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 131173 63 4.2BSD 1024 8192 16 # (Cyl. 0*- 300)
b: 262036 131236 swap # (Cyl. 301 - 901)
c: 2109840 0 unused 0 0 # (Cyl. 0 - 4839*)
d: 262036 393272 4.2BSD 1024 8192 16 # (Cyl. 902 - 1502)
e: 524072 655308 4.2BSD 1024 8192 16 # (Cyl. 1503 - 2704)
f: 930460 1179380 4.2BSD 1024 8192 16 # (Cyl. 2705 - 4839*)
If you compare the label to the output of the df command, you can see
how the filesystem device names correspond to the partitions in the label:
saclass % df
Filesystem 512-blocks Used Avail Capacity Mounted on
/dev/sd0a 126578 33988 86262 28% /
/dev/sd0d 252882 2 240236 0% /tmp
/dev/sd0e 505796 9890 470618 2% /var
/dev/sd0f 898262 344852 508498 40% /usr
/dev/sd2a 3342800 4 3175656 0% /local
saclass-1:/local/home 909690 61106 803100 7% /local/home
The root partition /dev/sd0a is partition a in the label, and so on.
You can also see the traditional c partition which spans the whole disk.
Most of these partitions have an fstype of 4.2BSD which is another
name for the Fast File System, that is the default for OpenBSD.
Partition b has a different fstype than the others, being a swap
partition. Swap partitions do not have a defined structure in the way
other filesystems do. Instead, they are used by the
kernel's virtual memory system as an extension of the machine's RAM.
Before we look at saclass-1's disklabel, I want to mention that the disklabel program will let you define boundaries however you like, so you need to be careful that you don't have overlapping partitions in use. You also want to be careful that you don't have unused cylinders between partitions. Saclass-1 seems to have a problem with its disk label. Here it is:
saclass-1 % sudo disklabel sd3
# /dev/rsd3c:
type: SCSI
disk: SCSI disk
label: MXT-1240S
flags:
bytes/sector: 512
sectors/track: 68
tracks/cylinder: 15
sectors/cylinder: 1020
cylinders: 2368
total sectors: 2415360
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 65280 0 4.2BSD 1024 8192 16 # (Cyl. 0 - 63)
b: 131580 65280 swap # (Cyl. 64 - 192)
c: 2423457 0 unused 0 0 # (Cyl. 0 - 2375*)
d: 262140 196860 4.2BSD 1024 8192 16 # (Cyl. 193 - 449)
e: 1024080 459000 4.2BSD 1024 8192 16 # (Cyl. 450 - 1453)
f: 939420 1483080 4.2BSD 1024 8192 16 # (Cyl. 1454 - 2374)
disklabel: warning, partition c: size % cylinder-size != 0
disklabel: partition c: partition extends past end of unit
disklabel: partition f: partition extends past end of unit
saclass-1 %
Let's also recall what dmesg told us as well:
saclass-1 % dmesg | grep ^sd3
sd3 at scsibus0 targ 3 lun 0: <MAXTOR, MXT-1240S, I3.6> SCSI2 0/direct fixed
sd3: 1183MB, 2368 cyl, 15 head, 68 sec, 512 bytes/sec, 2423457 sec total
Here we see a conflict between what the kernel sees the device as and
what geometry is specified in the BSD disk label. The kernel reports the
disk as having 2423457 sectors, while the disklabel is
reporting 2415360 sectors. In this case, the kernel's figure is probably
the correct one, so the c partition is really OK. You can use the
disklabel command to change its notion of the geometry, and in this
case we probably should. To do so, however, would require taking
the machine offline.
The OpenBSD disklabel program is quite user friendly. It also accepts a -E argument to put it into interactive edit mode just like the fdisk program. The lower-case -e argument also edits the diklabel, but it throws you into a text editor and you enter in the changes that way (like with vi ).
Let's fire it up on sd0 which is one of saclass-1's unused disks:
saclass-1 % sudo disklabel -E sd0
This platform requires that partition offsets/sizes be on cylinder boundaries.
Partition offsets/sizes will be rounded to the nearest cylinder automatically.
Initial label editor (enter '?' for help at any prompt)
> ?
Available commands:
p [unit] - print label.
M - show entire OpenBSD man page for disklabel.
e - edit drive parameters.
a [part] - add new partition.
b - set OpenBSD disk boundaries.
c [part] - change partition size.
d [part] - delete partition.
g [d|b] - Use [d]isk or [b]ios geometry.
m [part] - modify existing partition.
n [part] - set the mount point for a partition.
r - recalculate free space.
u - undo last change.
s [path] - save label to file.
w - write label to disk.
q - quit and save changes.
x - exit without saving changes.
? [cmnd] - this message or command specific help.
Numeric parameters may use suffixes to indicate units:
'b' for bytes, 'c' for cylinders, 'k' for kilobytes, 'm' for megabytes,
'g' for gigabytes or no suffix for sectors (usually 512 bytes).
Non-sector units will be rounded to the nearest cylinder.
Entering '?' at most prompts will give you (simple) context sensitive help.
> p
device: /dev/rsd0c
type: SCSI
disk: SCSI disk
label: Maxtor P-12S cyl
bytes/sector: 512
sectors/track: 74
tracks/cylinder: 13
sectors/cylinder: 962
cylinders: 2076
total sectors: 1997112
free sectors: 0
rpm: 3600
16 partitions:
# size offset fstype [fsize bsize cpg]
c: 1997112 0 unused 0 0 # (Cyl. 0 - 2075)
d: 333814 0 4.2BSD 1024 8192 16 # (Cyl. 0 - 346)
e: 333814 333814 4.2BSD 1024 8192 16 # (Cyl. 347 - 693)
f: 333814 667628 4.2BSD 1024 8192 16 # (Cyl. 694 - 1040)
g: 333814 1001442 4.2BSD 1024 8192 16 # (Cyl. 1041 - 1387)
h: 333814 1335256 4.2BSD 1024 8192 16 # (Cyl. 1388 - 1734)
i: 328042 1669070 4.2BSD 1024 8192 16 # (Cyl. 1735 - 2075)
>
This disk has a bunch of partitions already defined. I'm going to delete
them and create a new one:
> d i
> d h
> d g
> d f
> d e
> d d
> p
...
16 partitions:
# size offset fstype [fsize bsize cpg]
c: 1997112 0 unused 0 0 # (Cyl. 0 - 2075)
> a a
offset: [0]
size: [1997112] 32m
Rounding to nearest cylinder: 65416
FS type: [4.2BSD]
fragment size: [1024]
block size: [8192]
cpg: [16]
>
Notice that the disklabel editor let's you specify sizes in m for
megabytes. It takes other units as well, see the manpage for details.
Don't forget to write the label, if you need to:
> p
...
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 65416 0 4.2BSD 1024 8192 16 # (Cyl. 0 - 67)
c: 1997112 0 unused 0 0 # (Cyl. 0 - 2075)
> w
> q
No label changes.
saclass-1 %
Some brief explanation. I deleted all the partitions except, of course,
for the c partition. Then I added a new 32 megabyte a partition.
The quit command q does a write by default, that is why we saw the
"No label changes" message, because we had already written the label
explicitely with the w command.
Once you have created a disklabel (under OpenBSD) or finished with fdisk (under Linux) then you are ready to actually create a filesystem. Most flavors of unix have a newfs(8) command for this purpose. Linux users have the mke2fs(8) command.
Running the command is fairly straight forward, although the command has numerous options. Let's newfs the partition we just created and then look at the results.
saclass-1 % sudo newfs /dev/rsd0a
Password:
/dev/rsd0a: 65416 sectors in 68 cylinders of 13 tracks, 74 sectors
31.9MB in 5 cyl groups (16 c/g, 7.52MB/g, 1856 i/g)
super-block backups (for fsck -b #) at:
32, 15504, 30976, 46448, 61920,
saclass-1 %
A number of things about that require explanation. First, note that
the newfs command is using the device /dev/rsd0a instead of /dev/sd0a .
Look at the two of them using ls:
saclass-1 % ls -l /dev/*sd0a
crw-r----- 1 root operator 17, 0 Feb 4 00:01 /dev/rsd0a
brw-r----- 1 root operator 7, 0 Jan 18 10:41 /dev/sd0a
Instead of file sizes we see a perhaps strange notation: '17, 0'
All devices in unix appear as files in the filesystem but have these major
and minor device numbers appear instead of size in ls -l output.
The major number is an index in the kernel into a list of drivers;
the minor number is used by a given driver to distinguish amongst various
similar devices like multiple scsi disks. Look at all four scsi disks
on saclass-1 (only the a partitions of those disks, mind you):
saclass-1 % ls -l /dev/*sd[0123]a
crw-r----- 1 root operator 17, 0 Feb 4 00:01 /dev/rsd0a
crw-r----- 1 root operator 17, 16 Jan 18 10:41 /dev/rsd1a
crw-r----- 1 root operator 17, 32 Jan 18 10:42 /dev/rsd2a
crw-r----- 1 root operator 17, 48 Jan 18 10:42 /dev/rsd3a
brw-r----- 1 root operator 7, 0 Jan 18 10:41 /dev/sd0a
brw-r----- 1 root operator 7, 16 Jan 18 10:41 /dev/sd1a
brw-r----- 1 root operator 7, 32 Jan 18 10:42 /dev/sd2a
brw-r----- 1 root operator 7, 48 Jan 18 10:42 /dev/sd3a
Anyway, each partition has two devices associated with it. One is
the raw device (e.g. /dev/rsd0a ), this device is used by lowlevel
system commands like the newfs command as well as the fcsk file system
repair command. The other is known as a block device
(e.g. /dev/sd0a ) and this is used for higher level I/O. For example,
when you create a small file with only a few bytes in it, this file is
actually taking up more space. The amount of space taken up is determined
by the filesystem's block size. This could be 512 bytes or it could
8096 bytes. The basic block size you choose for a filesystem depends on
a number of factors. Before I answer that though, let's talk about
some important filesystem concepts that relate back to the output of
the newfs command we saw above. Notably, the superblock .
An excellent book on the structure of the BSD operating system is
called "The Design and Implementation of the 4.4 BSD Operating System",
written by Mckusick, Bostic, Karels, and Quarterman. (Addison Wesley, 1996,
ISBN 0-201-54979-4). I highly recommend it if you are at all interested
in the internels of BSD. At any rate, they have this to say:
A traditional UNIX filesystem is described by its superblock, which
contains the basic parameters of the filesystem. These parameters
include the number of data blocks in the filesystem, a count of the
maximum number of files, and a pointer to the 'free list', which is
a list of all the free blocks in the filesystem. [p. 269]
Every data file in a unix filesystem has an inode or 'index node'
that descibes it. Included in the inode are the permissions, the id numbers
of the owner and group of the file, its size and pointers to datablocks
in the filesystem in which the actual data of the file is stored. So
you can see that, even if your file only has a few bytes in it, it is
still using at least one data block. (Two different files can not share
one data block.)
Reasons to have a smaller block size (like 512) could be, for example, a news server which will have lots of small files. If you had a large block size like 8096, but most of your files were only 1024 or less, then you are losing a lot of space. Conversely, if you have only large files, but they do change often, then a large block size makes more sense to avoid fragmentation . Fragmentation is the idea that the various datablocks that make up a given files are spread all over the filesystem, instead of being (physically) contiguous on the disk. One reason that is important is speed of data retrieval.
When the newfs command is used to create a filesystem it stores the superblock information at a number of different locations in the partition. It outputs the sector location of each of these backup superblocks (as you saw in the example above). If you save this information somewhere, you can use it to recover the partition if, for some reason, the primary superblock gets trashed.
After newfs'ing the filesystem, it can now be mounted. A standard mountpoint that is used for temporary things is /mnt and I'll use it for this example:
saclass-1 % sudo mount /dev/sd0a /mnt
Password:
saclass-1 % df
Filesystem 512-blocks Used Avail Capacity Mounted on
/dev/sd3a 63198 37090 22950 62% /
/dev/sd3d 253402 10566 230166 4% /var
/dev/sd3e 991790 316990 625212 34% /usr
/dev/sd3f 909690 61106 803100 7% /local
/dev/sd0a 62902 2 59756 0% /mnt
saclass-1 %
and you can also see that it shows up in df .
There is a corresponding command to unmount a filesystem:
saclass-1 % sudo umount /mnt
saclass-1 %
The mount command has options that allow it to mount any filesystem
that your kernel can deal with, for example:
% sudo mount -t msdos /dev/fd0 /mnt
would mount a DOS floppy on the mount point /mnt.
The fsck command is used to repair the filesystem. The OpenBSD fsck_ffs (which is called by fsck) performs the following functions:
Unreferenced inodes
Link counts in inodes too large
Missing blocks in the free map
Blocks in the free map also in files
Counts in the super-block wrong
Basically, the program goes through every block in the filesystem and
makes sure that inodes point to what they are
supposed to, that all blocks are either used in files, or on the
free list etc.
Sometimes fsck will find a block or even a chain of data blocks that are no longer referenced in the superblock. These are called unreferenced inodes, and fsck will re-reference the data and create a file with the name being the original unreference inode number. It places these files in a directory called 'lost+found' that is created for this purpose in base of the filesystem.
Occasionally the primary super-block for the filesystem gets corrupted (very rare and yucky). Use the -b command to fsck (or to the filesystem-specific version of fsck, e.g. fsck_ffs ) to specify an alternate superblock. There is always a backup super-block at offset 32 and you can use the -N flag to newfs to tell you the locations of all the alternate superblocks.
The file /etc/fstab is a list of all filesystems that are to be mounted when the machine boots. (On Solaris machines this file is called /etc/vfstab). Here is the /etc/fstab from saclass-1:
saclass-1 % cat /etc/fstab
/dev/sd3a / ffs rw 1 1
/dev/sd3d /var ffs rw 1 2
/dev/sd3e /usr ffs rw 1 2
/dev/sd3f /local ffs rw 1 2
Both the mount and fsck commands will reference this file by default.
Please see the manpage for a complete description of each field.
The idea of the flags directory is to implement a crude FIFO queue for you all to be able to access certain resources without stepping on each others toes.
Each flag represents a resource that can only be used by one person at a time. For example, running fdisk/disklabel/newfs/mount on a particular disk is something only one person can do at one time; if two folks were to try to, things could get nasty.
To make the queue work effectively, everybody has to play by the rules:
If you notice that a resource has been claimed, but the person is no longer logged in (i.e. check the w who and last commands to see if they are logged in or how long ago they disconnected); If it looks like the resource has simply been abandoned, then it is OK to claim it, but please mail both me or david clements as well as the offending user.
Please be careful and have fun!