6.828 Fall 2008 Lab 7: Final Project

Handed out Friday, November 14, 2008
Tarball Due Friday, December 5, 2008
Private Demos on December 4, 5, 11 and 12, 2008
In Class Demos on December 3, 2008


In this lab you will flesh out your kernel and library operating system enough to run a shell on the console. You will then do the core of the lab: the final project.

Lab Requirements

Unlike in previous labs, in this lab you may work in pairs. Here are some ideas for you final project: To complete the assignment, you will turn in a tarball of your code as usual. In addition, you will have to demonstrate your final project for the TAs in person on December 4th or 5th. Dates and a sign-up form will be posted soon. Every group should also be prepared to do a 5 minute in-class demo on Wednesday December 3rd, 2008.

Late policy. If you are working in a pair, you may combine your late days, so if one of you has one day and the other also has one day, you can turn in the finished code up to two days late without penalty. However, the assignment must be turned in by the evening of Friday, December 12, even if you have enough late days (or want to use penalty days) to be able to hand it in later. Labs received after Friday, December 12 will receive an F. Groups that turn in their project late will meet with the TAs on Friday, December 12th instead. Regardless when you turn in your code, you must still be prepared to demo your project in class on December 3rd.

Getting Started

Use Git to commit your Lab 6 source, fetch the latest version of the course repository, and then create a local branch called lab7 based on our lab7 branch, origin/lab7:

athena% cd ~/6.828/lab
athena% git commit -am 'my solution to lab6'
Created commit 734fab7: my solution to lab6
 4 files changed, 42 insertions(+), 9 deletions(-)
athena% git pull
Already up-to-date.
athena% git checkout -b lab7 origin/lab7
Branch lab7 set up to track remote branch refs/remotes/origin/lab7.
Switched to a new branch "lab7"
athena% git merge lab6
Merge made by recursive.
 kern/env.c |   42 +++++++++++++++++++
 1 files changed, 42 insertions(+), 0 deletions(-)

Shared Repository

Since most of you will be working in groups for this lab assignment, you may want to use git to share your project code between group members. To set up a shared git repository in AFS, one of your group members should run the following commands to create the repository in their home directory:
athena% cd ~/6.828
athena% mkdir project-repo.git
athena% cd project-repo.git
athena% fs sa . groupmember1 rlidwk
athena% fs sa . groupmember2 rlidwk
athena% git --bare init
Initialized empty Git repository in /afs/athena.mit.edu/user/g/r/groupmember0/6.828/project-repo.git/
Now you will need to decide on whose source code you will use as a starting point for your group project. That group member should run the following commands to place his or her lab7 source code into the group repository:
athena% cd ~/6.828/lab
athena% git remote add group /afs/athena.mit.edu/user/g/r/groupmember0/6.828/project-repo.git
athena% git checkout -b project lab7
Switched to a new branch "project"
athena% git push group project
Counting objects: 3, done.
Writing objects: 100% (3/3), 221 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /afs/athena.mit.edu/user/g/r/groupmember0/6.828/project-repo.git
 * [new branch]      project -> project
athena% git config branch.project.remote group
athena% git config branch.project.merge refs/heads/project
The other members of your group should configure their git checkouts to track the project branch from this shared group repository, as follows:
athena% cd ~/6.828/lab
athena% git remote add group /afs/athena.mit.edu/user/g/r/groupmember0/6.828/project-repo.git
athena% git fetch group
From /afs/athena.mit.edu/user/g/r/groupmember0/6.828/project-repo.git
 * [new branch]      project    -> group/project
athena% git checkout -b project group/project
Branch project set up to track remote branch refs/remotes/group/project.
Switched to a new branch "project"
At this point, all of your group members should be able to pull and push commits from the shared repository by running git pull and git push on the project branch, respectively. Keep in mind that git only tracks files that you have explicitly committed, so you may want to run git status periodically to see if there are any lingering files in your checkout that you haven't committed. You can use git add filename to tell git about files you would like to commit with the next git commit command. If there are files that you will never want to commit, and you want to prevent them from showing up in the output of git status, you should create a file named .gitignore and add the filenames that you'd like git status to ignore to that file, one per line.

Part 1: sharing library code across fork and spawn

We would like to share file descriptor state across fork and spawn, but file descriptor state is kept in user-space memory. Right now, on fork, the memory will be marked copy-on-write, so the state will be duplicated rather than shared. (This means that running "(date; ls) >file" will not work properly, because even though date updates its own file offset, ls will not see the change.) On spawn, the memory will be left behind, not copied at all. (Effectively, the spawned environment starts with no open file descriptors.)

We will change fork to know that certain regions of memory are used by the "library operating system" and should always be shared. Rather than hard-code a list of regions somewhere, we will set an otherwise-unused bit in the page table entries (just like we did with the PTE_COW bit in fork).

We have defined a new PTE_SHARE bit in inc/lib.h. This bit is one of the three PTE bits that are marked "available for software use" in the Intel and AMD manuals. We will establish the convention that if a page table entry has this bit set, the PTE should be copied directly from parent to child in both fork and spawn. Note that this is different from marking it copy-on-write: as described in the first paragraph, we want to make sure to share updates to the page.

Exercise 1. Change duppage in lib/fork.c to follow the new convention. If the page table entry has the PTE_SHARE bit set, just copy the mapping directly. (Note that you should use PTE_USER, not PTE_FLAGS, to mask out the relevant bits from the page table entry. PTE_FLAGS picks up the accessed and dirty bits as well.)

Use gmake run-testpteshare to check that your code is behaving properly.

You should see lines that say "fork handles PTE_SHARE right" and "spawn handles PTE_SHARE right".

Exercise 2. Change the file server so that all the file descriptor table pages and the file data pages get mapped with PTE_SHARE.

Use gmake run-testfdsharing to check that file descriptors are shared properly. You should see lines that say "read in child succeeded" and "read in parent succeeded".

Part 2: the keyboard interface

For the shell to work, we need a way to type at it. bochs has been displaying output we write to the printer port, but there is no good way to give it input. Instead, we'll use the X11-based interface and use CGA output and a keyboard driver. We've written the keyboard driver for you in kern/console.c, but you need to attach it to the rest of the system.

Exercise 3. In your kern/trap.c, call kbd_intr to handle trap IRQ_OFFSET+IRQ_KBD.

We implemented the console input/output file type for you, in user/console.c.

Test your code by running gmake xrun-testkbd and type a few lines. The system should echo your lines back to you as you finish them. Make sure you type into the X window bochs brings up, not the console.

The Shell

Run gmake xrun-icode. This will run your kernel inside the X11 Bochs starting user/icode. Icode execs init, which will set up the console as file descriptors 0 and 1 (standard input and standard output). It will then spawn sh, the shell. You should be able to run the following commands:
	echo hello world | cat
	cat lorem >out
	cat out
	cat lorem |num
	cat lorem |num |num |num |num |num
	cat script
	sh <script
Note that the user library routine printf prints straight to the console, without using the file descriptor code. This is great for debugging but not great for piping into other programs. To print output to a particular file descriptor (for example, 1, standard output), use fprintf(1, "...", ...). See user/ls.c for examples.

Run gmake run-testshell to test your shell. Testshell simply feeds the above commands (also found in fs/testshell.sh) into the shell and then checks that the output matches fs/testshell.key.

Challenge! Do an awesome final project!

This ends the lab. As usual, you can grade your submission with gmake grade and hand it in with gmake handin. By December 3rd, you should be prepared to show off your challenge to the class. December 4th and 5th you will present to the TAs.