CSCI 333

Storage Systems

Home | Schedule | Labs | Williams CS | Piazza

Lab 3b: FUSE FS, part II

Objectives

In lab 3b, you will (1) finish the prototype filesystem you began in lab 3a, and (2) add enough functionality to your prototype filesystem that it can be used to store actual data in mutable files.

The Assignment

Your assignment is to finish the "Simple FS"-like filesystem that you began in lab 3a. Your finished filesystem should have the following features:

• The general structure of the filesystem is similar to the "Simple FS" design (see lab 3a's description).
• In addition to all of the calls supported in lab 3a (getattr, access, readdir, and mkdir), you must support the following additional operations: close (i.e., release), create, fgetattr, mknod (only for plain files, so you can assume mode will always be S_IFREG and ignore dev), open, read, rmdir, statfs, truncate, unlink, and write.
• You are not required to support readlink, symlink, chmod, chown, fsync, link, rename, or utimens. Note that many of these operations require additional filesystem design (e.g., chown requires that you track ownership).
• Your write operation must support full Unix semantics. It must be possible to:
  • Extend a file by writing at the end,
  • rewrite arbitrary bytes in the middle of a file, and
  • extend a file by a large amount by seeking far past the end and writing a single byte.
• You are not required to support sparse files (a sparse file is a large file that does not explicitly represent all of the "holes"; instead it creates "0"s on demand).
• Directories should be variable-sized and should grow as files are created within them. However, it is not necessary to shrink the size of directories when files are removed. In other words, as long as you mark the deleted contents of the directory invalid and ensure that files are logically inaccessible, you do not need to reclaim fragmented space within the directory's data blocks.
• The statfs operation should correctly report the amount of free space in the data region of the file system. In particular, if it says that n blocks are left, it should be possible to write n additional blocks of data to an existing file, and impossible to write n+1 blocks.
• Note that if you decide to implement symnolic links, your path-evaluation code doesn't need to do anything special for symbolic links; FUSE will do that for you.

Testing your File System

Many of your features can be tested using standard Unix utilities (the dd utility is especially useful—see `man dd`). I strongly recommend that you write special test code or scripts. I will provide some tests, but I also encourage you to share any tests that you write with the group. So as soon as you add functionality, create a unit test. Then run all of your existing unit tests to verify your new "feature" didn't break anything! I recommend creating a separate folder of test scripts.

Here are some things that I expect to work, so you are encouraged to create at least tests for these features:

• Subdirectories (and sub-sub, and…)
• Files and directories with (modestly) long names (e.g., over 16 characters, but I won't push the limits if you have set reasonable upper bounds)
• Directories with many entries (enough that the directory requires more than one disk block to store the (inode number -> path) mappings)
• ls -la, including for files of various sizes, and including . and ..
• Zero-length files
• Very large files (e.g., 15+ data blocks)
• Extending files (e.g., with `echo xxx >> foo`)
• Extending a file while overwriting a few bytes at the end
• Truncating files to zero length
• Truncating files to nonzero length
• Using truncate to extend files by adding zeros
• Changing a few bytes in the middle of a file (hint: invoking dd with the conv=notrunc switch is one way to do this)
• statfs (hint: the df command uses that call)

The above is not an exhaustive list. Try to think of any useful functionality that has been omitted.

Important Notes

Refer back to the important notes in lab 3a to make sure you don't do anything silly. In addition:

• Your implementation must be (reasonably) efficient. It should be possible to create a 1-MiB file in well under a second.
• You should not waste memory. It's OK to store the bitmaps and the entire inode table in memory. But you may not store a copy of the entire disk in memory, nor may you store entire files or directories in memory. An obvious exception is 1-block files, which of course may be assumed to fit…

Submission

Commit your code to your git repository. Your repository's README.md file should describe how to compile and run it. If you implement any additional features, describe them prominently in your README.md file so that you receive credit. So that I know that you have finished, please use the commit message "lab 3b submission" when you commit your final version. If you later realize that you were not done, you can continue to make additional commits, but it should be clear from the output of `git log` which version of your code is your submission.

Ideally, the final version of your lab will be completed by Sunday, May 10. That gives one week before the start of the final exam period. I realize that this has been a "non-traditional" semester in almost every imaginable way, and it may have been an uphill battle to stay ahead, stay focused, and communicate. If you find yourself lost or falling behind your target schedule, I would like you to consider the following options:

• Email me to schedule a one-on-one or group code review/design meeting
• Post a question on Piazza (remember, they are anonymous!)
• Message me on the course slack, either directly or in a public channel
• Message a classmate on the course slack, either directly or in a public channel
• Collaborate with a classmate, and go over each other's designs
• Come to office hours, even if you don't have any specific questions

Hints and Help

I have shared a framework to help with part I. I will continue to post updates and tests to that repository. When I do, I will post an update on Piazza so that you are aware.

If there is anything about the repository's code that is unclear, please let me know. My goal is to get us all across the finish line and learn something in the process. So you can borrow as much or as little of that code as you'd like and use it in your own implementations. The only requirement is that you understand and can explain to me all of the code that you submit.

This lab borrows from an assignment created by Geoff Kuenning.