CSCI 237

Computer Organization

Home | Schedule | Assignments | Links | Williams CS

What is the Autograder?

CS237 uses a system for automatically grading your projects. This isn't out of laziness; the autograder* is designed to help you do the projects correctly and provide feedback in a timely fashion.

To help you validate your programs, your submissions will be tested automatically, and the result will be emailed back to you. You may then continue to work on the project and re-submit.

Be warned that the results from the auto-grader will not be very illuminating; they won't tell you where your problem is or give you the test programs. The main purpose of the auto-grader is to help you know to keep working on your project (rather than thinking it's perfect and ending up with a 0). The best ways to debug your program continue to be generating your own test cases, using appropriate debugging strategies (e.g., gdb), and comparing your program's output to expected (correct) answers. Debugging is sometimes challenging, but it also one of the best ways to learn the concepts in the projects.

You may submit your program as many times as you like. However, only the results from the first submission of each day will be emailed back to you. Later submissions on that day will be graded and cataloged, but the results will not be emailed back to you.

Beyond this one-per-day policy, you will be given 5 bonus submissions per project that also provide feedback. Bonus submissions will be used automatically: each submission you make after the first one of the day will count as one of your bonus submissions, until they are used up. After your bonus submissions are used up, the system will continue to provide feedback once per day.

In addition to the auto-grader's evaluation of your programs' correctness, we will evaluate your programs in other ways. These criteria may include clarity and completeness of your documentation, coding style, efficiency, and understandability of your code, etc., as well as evaluations of post-hoc explanations of your code. Your final score will be the product of the auto-grader's score and these other measures.

Submitting Your Work

Use the submit237 script to submit your files from any one of the department's lab machines. The full name of the script is /usr/cs-local/237/bin/submit237. You will need to type this full name everywhere that you see the command submit237 until you add the location of this script to your $PATH environment variable. Once you do add /usr/cs-local/237/bin to your path, you won't have to keep typing in the full name (instead, you will just type submit237). Ask your instructor or TA for help editing the $PATH environment variable if you don't know how to do it. A strategy that should work is to execute the following command:
   $ echo "PATH=$PATH:/usr/cs-local/237/bin" >> .local_bashrc

submit237 submits the set of files associated with a lab assignment and is called as follows:

   $ submit237 <projectnum> <file1> <file2> <file3> ... <fileN>

Note that all C programs should end in ".c".

For example, to submit the single file bits.c for Project 1 (bit puzzles), you will execute:

   $ submit237 1 bits.c

The script will package up your project and send it to the autograder. Within a few hours at most (depending on the server's load) you should receive an email telling you how you did. If you do not receive a response within 4 hours of submission, please email the instructors at cs237staff@williams.edu. The official time of submission for your project will be the time the last submission of your project is received at the server. If you send in anything after the due date, your project will be considered late (and any associated penalties may apply).

Helpful Hints

The key to using the autograding system is to:

1) "Submit early and often." Your submissions are limited to one per day (plus bonus submissions). This means the earlier you start on the projects, the more chances you have to get it right. You may wish to submit your starter files the day you receive them just to see what the feedback will look like.

2) Recognize that the autograder is not a debugger. The autograder will not tell you what you did wrong or what it is using for its testcases.

3) Testing is the only way to figure out what is actually wrong with your program. Writing effective test cases is a central component of writing good software.

4) Do not be afaid to "throw one away". Starting over is hard to do, but sometimes it is the best option.

*CREDIT: Many parts of the autograder were developed at the University of Michigan. We are very grateful to the creators!