This lab is a short lab designed to make you familiar with the SFS software infrastructure. In the previous assignments you have used SFS's async lib. In this assignment you will be exposed to a functional distributed file system, remote procedure call, and the NFS 3 protocol. We expect that most of you wil do a final project in the area of distributed file systems and use the SFS software for your final project; you don't have to but it will save you a lot of time.
This assignment is simple, in principle: you are supposed to fill in some missing blanks in an user-level NFS3 server that we will hand you. At the end of the assignment you will have an operational, usable, secure, distributed file system. The assignment should take you a couple of hours (we did most of the work for you). Once you have completed the assignment, you should start thinking of what you want to do for your final lab. The final lab is essentially to build a cool system of your own design and write a short paper about it. (If you don't have any specific ideas for a project yourself, we will suggest a number of possible projects.) Details will follow shortly.
Although this assignment is easy, you will be handed an arsenal of sophisticated weapons and it also easy to shoot yourself in the foot (i.e., you may have to reboot your workstation). We (that is David Mazieres, the principle author of SFS) went through great length to make the software robust. Programming errors in the client and the server of the file system shouldn't crash your local operating system.
athena% uname -a SunOS all-night-tool.mit.edu 5.7 Generic_106541-09 sun4u sparc SUNW,Ultra-60Fetch sfsusrv.tar with your web browser and place it in your Athena home directory. This tar file contains just the source for a basic file server; it depends on library files which we've pre-compiled for you in /mit/6.894/sfs.
Run these commands:
athena% add -f gnu athena% add 6.894 athena% tar xvf sfsusrv.tar athena% cd sfsusrv athena% makeThe make may take five or ten minutes.
You need to create a public/private key pair for sfsusrv to use to prove its identity to clients. Run these commands in your sfsusrv directory:
athena% setenv SFS_RUNINPLACE `pwd` athena% /mit/6.894/sfs/agent/sfskey gen -P sfs_host_keyType return when sfskey prompts you for the Key Name.
In order to avoid conflicts between different 6.894 students running sfsusrv on the same dialup machine, you should choose yourself a random port number between 1024 and 65536. Let's call it my-port.
Open a new window and ssh to athena.dialup.mit.edu. Make sure you're on a Sun. Then:
athena% cd ~/sfsusrv athena% mkdir /tmp/export-$user athena% rm sfsusrv_config athena% echo 'keyfile sfs_host_key' > sfsusrv_config athena% echo "export /tmp/export-$user" >> sfsusrv_config athena% setenv SFS_RUNINPLACE `pwd` athena% setenv SFS_PORT my-port athena% ./sfsusrv -f sfsusrv_configIf all went well, you should see output like this:
sfsusrv: version 0.5, pid 8493 sfsusrv: serving /sfs/biohazard-cafe.mit.edu:dgn84hqianfvqpvh7a42wgubqz9pdy9g sfsusrv: No sfssd detected, running in standalone mode.The /sfs/biohazard-cafe.mit.edu:dgn84... is a self-certifying pathname. It contains enough information for an SFS client to contact your SFS server and verify the server's identity: the biohazard-cafe.mit.edu part names the server, and the bytes after the colon are a hash of the server's public key and server name. You can find out more about self-certifying pathnames and SFS at http://www.fs.net; we will also read a paper about SFS (see schedule).
sfsusrv is now acting as an SFS file server for the directory /tmp/export-$user. Now you need to start an SFS client.
athena% cd /tmp athena% mkdir sfs-$user athena% cd sfs-$user athena% mkdir etc athena% cp ~/sfsusrv/etc/* etc athena% add 6.894 athena% ln -s /mit/6.894/sfs/nfsmounter . athena% tellme root athena% su (Type the root password, which the tellme root command told you.) athena% setenv SFS_RUNINPLACE `pwd` athena% setenv SFS_PORT my-port athena% /mit/6.894/sfs/sfscd/sfscdNow open another window on your workstation. You'll need the self-certifying pathname that sfsusrv printed, so copy it with the mouse. Then type
athena% cd PASTE_SELF_CERTIFYING_PATHNAME_HERE athena% lsYou should see a list of the files in /tmp/export-$user on the sfsusrv machine.
athena% cd PASTE_SELF_CERTIFYING_PATHNAME_HERE athena% echo hi > junk-$user athena% cat junk-$userYou can't read the file because sfsusrv doesn't implement the read operation. Your job is to implement the NFS3 read RPC. The file that implements the NFS3 RPCs is client.C. Open client.C and you will see how the other NFS3 RPCs are implemented using standard file system system calls; your job is to create a similar implementation for read, perhaps named client::nfs3_read().
If you do this correctly, you will only have to make changes to client.C, but we encourage you to understand how the client and server work. The parts of relevance are the client code, the RPC files, and sfsusrv. You have your copy of the sfsusrv source; everything else (including libraries that sfsusrv uses) is in subdirectories of /mit/6.894/sfs.
The code that runs on the client machine is stored in the directories sfsmounter, sfscd, and sfsrwcd. sfsmounter is the mounter for /sfs. sfsmounter contains very sophisticated code; you don't really have to understand it. The cool thing about it is that you can kill sfscd or sfsrwcd and restart it without having to reboot your computer.
sfscd makes an encrypted, authenticated connection with the server specified in the self-certifying pathname using the FS protocol. The SFS protocol is specified in svc/sfs_prot.x. The makefiles run this file through an RPC compiler to generate stubs and marshalling code; the RPC code is in the directory arpc. The crypto code is in the directory crypt.
If sfscd succeeds in setting up a connection, the remote server will tell it to handoff the connection to sfsrwcd. sfsrwcd relays NFS3 calls issued by the local kernel to the server on the other side of the secure connection (sfsusrv in this case). It does the same for replies. The RPC file for NFS3 is stored in svc/nfs3_prot.x. SFS extends NFS3 slightly to include an expiration field for file attributes to help with cache consistency; this only affects RPC responses, whose definitions are in svc/nfs3exp_prot.x sfsusrv sets the expiration field (called expire) to zero, which causes the SFS client to act just as it would for standard NFS3.
Here are some hints:
Stop your sfsusrv with control-C, then re-start it with these commands:
athena% setenv ASRV_TRACE 10 athena% ./sfsusrv -f sfsusrv_config >& sfsusrv.traceMake sure you're still running sfscd. Type these commands on the sfscd machine:
athena% cd /sfs/PASTE_SELF_CERTIFYING_PATHNAME_HERE athena% rm junk-$user athena% echo hello > junk-$user athena% cat junk-$user athena% cat junk-$userStop sfsusrv. Now look at the RPC queries and replies recorded in sfsusrv.trace. Which correspond to the creation of junk-$user? Which to the first cat? Which to the second cat? Explain any differences between the RPCs caused by the two cat commands.
% cat trace-explanation.txt client.C | Mail -s '6.894 lab 4' 6.894-submit@pdos.lcs.mit.eduIf you have any problems about submission, please contact jinyang@lcs.mit.edu. The lab is due by the beginning of class on Tuesday, October 24.