O/S overview

Overview

6.828 goals:
- Understand operating systems in detail by designing and implementing a small O/S
- Hands-on experience with building systems ("Applying 6.033")
What is an operating system?
- software that turns the hardware into something useful
- layered picture: hardware, O/S, applications
- Four main functions:
  - Abstract the hardware for programmer convenience
  - Multiplex the hardware among multiple applications
  - Isolate applications to contain bugs
  - Allow sharing among applications
Examples:
- OS-X, Windows, Linux, *BSD, ... (desktop, server)
- PalmOS, Windows/CE (PDA)
- Symbian, JavaOS, OS-X? (Cell phones)
- VxWorks, pSOS (real-time)
O/S typically provides abstract resources:
- processes
- files
- directories and file names
- users + security
- interprocess communication
- networking
- time
- terminals
What does an O/S abstraction look like?
- They are usually implicit: no class Process
- Applications only see them via system calls, for example:
- fd = open("/dev/tty", 1);
- write(fd, "hello\n", 6);
- pid = fork();
- execl("/bin/echo", "echo", "hello", 0);
Why is O/S design hard/interesting?
- The abstractions interact (fd = open(); ...; fork();)
- You want simplicity via a few unifying ideas (e.g. file descriptors)
- You want the minimum number of mechanisms (is IPC alone enough?)
- Convenience vs efficiency (most of what fork() copies is wasted)
- Convenience vs fine control (fork() vs clone()/rfork())
- Portability vs efficiency (files vs disk extents)
- Security gets in the way of everything
Why might 6.828 be useful?
- Perhaps you will build an O/S
- Or write a device driver, network stack, or file system
- Or need to diagnose application bugs or performance
- Or use ideas like concurrency / synchronization

Class structure

Lectures (20%)
- 45min lecture
- 45min case study
- xv6 (a traditional O/S) then more modern topics
- homework
Lab: small O/S for x86 in an exokernel style (50%)
- kernel interface: hardware + protection
- unprivileged library: fork, exec, pipe, ...
- applications: file system, shell, ..
- development environment: gcc, bochs
- lab 1 is out
Two quizzes (30%)
- mid-term (in class)
- final (during exam week)

Case study: the shell (simplified)

interactive command execution and a programming language
Nice example that uses various O/S abstractions. See Unix paper if you are unfamiliar with the shell.
Final lab is a simple shell.

Basic structure:

      
       while (1) {
	    printf ("$");
	    readcommand (command, args);   // parse user input
	    if ((pid = fork ()) == 0) {  // child?
	       exec (command, args, 0);
	    } else if (pid > 0) {   // parent?
	       wait (0);   // wait for child to terminate
	    } else {
	       perror ("Failed to fork\n");
            }
        }

The split of process creation into fork and exec turns out to have been an inspired choice, though that might not have been clear at the time; see the assigned paper for today.

Example:
```
        $ ls
```
how does ls know which directory to look at? how does it know what to do with its output? how does the shell know ls has finished?
why call "wait"? to wait for the child to terminate and collect its exit status. (if child finishes, child becomes a zombie until parent calls wait.)
I/O: file descriptors. Child inherits open file descriptors from parent. By convention:
- file descriptor 0 for input (e.g., keyboard). read_command:
```
     read (0, buf, bufsize)
```
- file descriptor 1 for output (e.g., terminal)
```
     write (1, "hello\n", strlen("hello\n"))
```
- file descriptor 2 for error (e.g., terminal)
How does the shell implement:
```
     $ ls > tmp1
```
just before exec insert:
```
    	   close(1);
	   fd = creat("tmp1", 0666);  // fd will be 1
```
The kernel will return the first free file descriptor, 1 in this case.

What if you run the shell itself with redirection?

     $ sh < script > tmp1

If for example the file script contains

     echo one
     echo two

Is that equivalent to:

     $ echo one > tmp1
     $ echo two > tmp1

how do programs communicate?

        $ sort file.txt | uniq | wc

	$ sort file.txt > tmp1
	$ uniq tmp1 > tmp2
	$ wc tmp2
	$ rm tmp1 tmp2

A pipe is a one-way communication channel. Here is a simple example:

        int fdarray[2];
        char buf[512];
        int n;

        pipe(fdarray);
        write(fdarray[1], "hello", 5);
        n = read(fdarray[0], buf, sizeof(buf));
        // buf[] now contains 'h', 'e', 'l', 'l', 'o'

file descriptors are inherited across fork(), so this also works:

        int fdarray[2];
        char buf[512];
        int n, pid;

        pipe(fdarray);
        pid = fork();
        if(pid > 0){
          write(fdarray[1], "hello", 5);
        } else {
          n = read(fdarray[0], buf, sizeof(buf));
        }

How does the shell implement pipelines (i.e., cmd 1 | cmd 2 |..)? We want to arrange that the output of cmd 1 is the input of cmd 2. The way to achieve this goal is to manipulate stdout and stdin.

The shell creates processes for each command in the pipeline, hooks up their stdin and stdout, and waits for the last process of the pipeline to exit. Here's a sketch of what the shell does, in the child process of the fork() we already have, to set up a pipe:

	    

	    int fdarray[2];

  	    if (pipe(fdarray) < 0) panic ("error");
	    if ((pid = fork ()) == 0) {  child (left end of pipe)
	       close (1);
	       tmp = dup (fdarray[1]);   // fdarray[1] is the write end, tmp will be 1
	       close (fdarray[0]);       // close read end
	       close (fdarray[1]);       // close fdarray[1]
	       exec (command1, args1, 0);
	    } else if (pid > 0) {        // parent (right end of pipe)
	       close (0);
	       tmp = dup (fdarray[0]);   // fdarray[0] is the read end, tmp will be 0
	       close (fdarray[0]);
	       close (fdarray[1]);       // close write end
	       exec (command2, args2, 0);
	    } else {
	       printf ("Unable to fork\n");
            }

Who waits for whom? (draw a tree of processes)
Why close read-end and write-end? ensure that every process starts with 3 file descriptors, and that reading from the pipe returns end of file after the first command exits.
How do you create a background job?
```
        $ compute &
```
How does the shell implement "&", backgrounding? (Don't call wait immediately).
More details in the shell lecture later in the term.