Usage¶

Before following this guide, make sure you’ve installed zucchini as described in Installation.

The following section is written as a zucchini workflow that would be used by a TA in a course that already has a zucchini farm set up (as an example, we will use the sample zucchini farm), that an instructor or TA has already prepared the assignment and linked to it on the farm (as an example, we will use the sample zucchini JUnit assignment), and that student submissions are available in directory (as an example, we will use sample submissions on a git repo).

Note that this tutorial expects that you are on either Linux or OSX, that you have access to the terminal (Terminal.app on OSX), that you have installed a Python distribution that’s either 2.7 or >3.4 (we recommend Anaconda for beginners), that you have git installed, that you have JDK 1.8 or higher installed and linked to your path, and that you have gradle installed.

Let’s start by installing zucchini

pip install zucchini


We set up our workspace by entering our identity details:

zucc setup


Then we add the farm for the metadata repository created by our instructor. We name it cs1337-fall1970:

zucc farm add https://github.com/zucchini/sample-farm.git cs1337-fall1970


Then we make a new directory for our grading and change into it.

mkdir zuccsample && cd zuccsample


We list the assignments on our farms to find the one we’re looking for:

zucc list


From the output of this, we find that our assignment is called junit/stacks-queues. We use zucc init to make zucchini pull the assignment configuration into a new directory which will have the assignment’s name. We use our farm’s name as well as the assignment’s name on the farm. Note that detailed information about this assignment, which tests a Stack and Queue implementation using JUnit, can be found on the repository page for the assignment.

zucc init cs1337-fall1970/junit/stacks-queues


git clone https://github.com/zucchini/sample-assignment-submissions.git


Now we change into our assignment directory, and make zucchini load the submissions we just downloaded. Note that in a real workflow, submissions would likely be loaded through LMS integration modules such as Canvas. Also note that the -d flag for the path loader is used to make zucchini use the directory name (e.g. Alice) as the submitting student’s name as well.

cd stacks-queues


Then, we start the grading process. This will grade each submission separately and save their results in their folders into the submissions’ meta.json files. Once the grading is done, a text editor will open to show the newly updated grades. Hit :q close it.

zucc grade


Now that we’re done grading, we want to exports the grades our students received. Note that in a real workflow, this would also likely be done through LMS integration modules such as Canvas, which allow for grades to be saved directly onto students’ accounts.

zucc export csv > grades.csv


And we’re done! The grades can be found in the CSV file.

Creating an Assignment¶

Anatomy of an Assignment¶

A Zucchini assignment consists of a list of components, each of which itself consists of a list of parts. Like this:

Zucchini aims to streamline the process of converting a student’s submission to a grade in the gradebook, and an assignment instructs Zucchini how to perform this conversion. Indeed, Zucchini downloads submissions, posts grades, and checks due dates for entire assignments, even if they consist of multiple components.

Components represent the smallest pieces of an assignment that Zucchini can grade independently. Usually, this means each independent file in the submission has its own component. Examples of components:

• A test class which tests a particular class in the submission in a JUnit-based grader
• A test suite in a Libcheck-based grader
• A subcircuit in a CircuitSim circuit
• A set of prompts in a prompt grader

Parts represent the smallest result in grading a component that deserves its own weight. We generalized parts because we noticed all of our backends had them. Examples of parts:

• A test method in a JUnit-based grader
• A test in a test suite in a Libcheck-based grader
• A test of a subcircuit in a CircuitSim circuit
• A prompt in a prompt grader

Now, here is a concrete example of the diagram above for a homework with a CircuitSim circuit fsm.sim and a Java file BitVector.java:

Weights¶

Zucchini weights components and parts relatively. That is, a component $$i$$ is worth $$\frac{\text{weight}_i}{\sum_k \text{weight}_k}$$ of the grade.

So for the following assignment:

the rubric is actually:

Component Percent
fsm.sim (One-hot subcircuit) 50%
fsm.sim (Reduced subcircuit) 16.67%
BitVector.java 33.33%

Parts have the same relationship with their parent components. So a part $$j$$ of a component $$i$$ is worth $$\frac{\text{weight}_i}{\sum_k \text{weight}_k} \times \frac{\text{weight}_j}{\sum_l \text{weight}_l}$$ of the grade.

Don’t let the decimal points above mislead you: Zucchini calculates grades with rational numbers internally, so you you don’t need to worry about floating point screwing up or perfect submissions getting a 99.99 or anything like that (lc3grade had this problem).

We added relative weighting because we didn’t enjoy twiddling with weights until they summed to 100. If you do, you can make all the weights add up to 100:

Assignment Configuration¶

The directory structure for an assignment my_assignment looks like:

my_assignment/
zucchini.yml
submissions/
Sood, Sanjay/
meta.json
files/
fsm.sim
Lin, Michael/
meta.json
files/
fsm.sim


You need to create only zucchini.yml and optionally grading-files/. Zucchini will generate submissions/. zucchini.yml looks like

name: Homework X # required
author: Michael Lin # required
due-date: 2018-06-24T18:00:00-04:00
canvas:
course-id: 2607
assignment-id: 8685
penalties:
- name: LATE
backend: LatePenalizer
backend-options:
penalties:
- after: 1h
penalty: 25pts
components: # required
- name: Finite State Machine # required
weight: 2 # required
backend-options:
test-class: FsmTests
files: [fsm.sim]
parts: # required
- {test: clockConnected,  weight: 1}
- {test: resetConnected,  weight: 1}
- {test: enableConnected, weight: 1}
- {test: outputA,         weight: 5}
- {test: transition,      weight: 10}
- name: Fully reduced
weight: 1
backend-options:
command: "java -cp hwX-tester.jar com.ra4king.circuitsim.gui.CircuitSim fsm.sim"
files: [fsm.sim]
parts:
- text: "banned gates?"
weight: 2
- text: "number of incorrect SOP expressions"

You can find a full list of graders at zucchini.graders.