Lecture 3


The Unix file structure is (modulo some complications that we'll deal with momentarily) a tree. The base of the tree—the root—is /. Files can be named either by absolute paths like /bin/sh—this is the sh file in the bin directory of the root. Note again that DOS/Windows uses \ as a path separator (FWIW, Classic Macintosh used : internally, although this was hidden from ordinary users). Why? Gratuitous incompatibility.

Often, however, relative paths are used. What's happening here is that paths are often named in a context where there is an implicit directory. In this case, you simply describe how to get there from here, omitting the first slash. Thus, if your working directory is /bin, you can give the relative path sh, which is just /bin/sh.

Let's consider a slightly more complicated situation. Professor Kurtz's home directory on UCCS machines is /home/stuart. He has a simple shell script that removes all those obnoxious emacs backup files at /home/stuart/bin/clean.

$ pwd /home/stuart $ cat bin/clean #!/bin/bash rm -f *~ $

Having a script to do this is much safer than typing in by hand any command that begins with rm ... *. Experience is a hard teacher. Effective though.

He can use the relative path bin/clean from /home/stuart to name /home/stuart/bin/clean, which is useful if he wants to edit it.

What if we want to use a relative path to describe a file that is not a descendant of my working directory? We can use .. to mean “parent,” and . to mean “self.” Thus, e.g.,

$ ls ..

will list the files contained in the parent of the current working directory. These pseudo-files (..) and (.) can appear as ordinary path components. Thus,

$ cat ../README

or even

$ cat ../../../README

are perfectly plausible.

Your PATH environment variable (we'll talk more about environment variables later) tells the shell where to look for executable programs. For security reasons, you should never have . on your current path, but this creates a problem: if you're writing and debugging a program, you probably don't have the build directory in your path. How can you execute it? Well, if you provide a nontrivial path to a binary, it will execute. So...

$ ./fib 10 55 $

There's a special abbreviation ~ which specifies a user's home directory (i.e., the current value of the shell variable HOME). Thus, if Professor Kurtz's current working directory is something other than /home/stuart, he can still refer to the file above via the path ~/bin/clean. This is often useful, e.g., ls ~/bin. One caveat here is that tilde expansion is something that is done by the shell, not by the system's path-based file handling routines.

Links and inodes

What is really happening with the file system is a bit more complicated. A real system may have multiple device-based file systems associated with it. One of these file systems is the root system. Other file systems are mounted onto the root system, at particular nodes. This differs from the DOS/Windows approach in which the file systems remain distinct—a forest—and are mapped to letter designators, e.g., the ubiquitous C:. Somewhat idiosyncratically, even though it is a Unix-based system, MacOS X's Finder presents separately mounted partitions as a forest, a rare case where an operating system provides the end user with a model that's closer to the hardware than what it provides to a programmer.

The particular path to a file on a mounted file system will depend where that file system was mounted. For example, it is not only possible for the same file system to be mounted at distinct points on different machines, it is fairly common.

Each file system contains an array of inodes. An inode can be thought of as having an index (its index in the file system dependent array), the length of the file, and some sort of hardware dependent way of describing where to find its bytes (e.g., a sequence of sector-offset-length triples). Note that a file system can be exhausted either by running out of places to store bytes (e.g., if you have a few really big files) or by running out of inodes (e.g., if you have a lot of little files). Part of system tuning is in figuring out how many inodes to allocate so that you run out of bytes and inodes more or less at the same time. Unixes usually make a conservative underestimate of average file size, so that you'll tend run out of bytes before inodes. It's easier to explain this policy to end users.

The files that constitute directories contain a representation of a map from trailing path components to inodes on their file system. Thus, it is not possible to have a hard link cross a file-system boundary. If this mapping was 1-1, then the Unix file system really would be a tree. It's not. The same inode may be accessible via multiple paths. This has some deep consequences. In particular, consider file deletion. When we delete a file, we can't simply add its inode to a free list—after all, what we delete is actually a reference from a file (in the guise of a directory) to the inode that represents our file, and there may be other links—even from the same directory!

So what actually happens is that an inode maintains a reference count—how many paths are associated with it. When a path is removed, the reference count associated with that files inode is decremented. If the reference count reaches zero, the file can be freed on the physical media. Reference counts are incremented when a new path to the file is created, either via the command line ln (link) command, or its underlying system call. One of the ways a file system can become inconsistent is if the reference count stored in the inode is incorrect. For this reason, directory files are now special, the remarks of K&P notwithstanding. Another problem, endemic to reference counting schemes, is that circular structures can become inaccessible without having their reference count go to zero. To make this less likely to happen, only superusers can create new links to a directory, and this is very rarely done.

A further (post K&P) nuance is the existence of a special class of file called a symbolic link. This is a “that's me, over there” kind of file, but it keeps track of the “over there” part by path, rather than by inode. Thus, a symbolic link can refer to different files (i.e., files associated with different inodes, and perhaps even on different file systems if the mounts have changed) over its lifetime. Indeed, there may be times when the path is invalid. Symbolic links are created with ln -s source target. Symbolic links are not a luxury—they're the only way to create links that (appear to) cross a file system boundary.

Access Control

Each inode contains additional information in the form of a user ID (UID), group ID (GID), and access control bits. Historically, the /etc/passwd and /etc/groups files provided a map from symbolic user and group names to numeric IDs. These days, considerations of security and efficiency have typically resulted in somewhat different solutions, but the principle remains the same.

A group can be thought of as a collection of users. A user can belong to many groups, but files (and processes) are always associated with a single user and group.

The traditional UNIX access control system divided the universe into three classes: the user of a file/directory, the group of a file/directory, and others (everyone else). Access granularity consisted of separate read, write, and execute permissions. For a directory, execute means the right to traverse that directory.

Some systems today extend the traditional Unix system with access control lists (ACLs), which are a more flexible and fine-grained means of controlling access than the old user-group-other scheme, but the UGO model is adequate for our immediate uses, and generally speaking ACLs act as modifiers to access control granted by the traditional 3x3 system.

Access control is manipulated using the chmod program. Experienced UNIX users typically use the three-digit codes for setting permissions, e.g.,

$ chmod 644 foo

sets the read and write access bits for the file's owner (4 for read, 2 for write = 6, and if the execute bit had been set, it would be 1), and the read bits for group and world access (two more fours).

On the other hand, updating a single bit is more easily accomplished using the symbolic notation, e.g., to add user and group execute access to a script, you might use

$ chmod ug+x foo

It is important to realize that Unix began life as a research project for a small group of computer scientists at AT&T Bell Labs. As a result, the defaults in the permission scheme are biased towards sharing: directories are by default world readable, as are newly created files. These defaults can be changed, but doing so runs against the collaborative spirit that underlies the system. Privacy requires a conscious opt-in.

Exercise 3.1 Create a small text file, and then set its permissions so that you can't read it. Suppose you set the file permissions so that the user does not have read permission, but the user's group does. Can you read the file then?

Turn in a commented transcript.