Autograder.io

Autograder.io is an open-source automated grading system that lets programming instructors focus on writing high-quality test cases without worrying about the details of how to run them. Autograder.io is primarily developed and maintained at the Computer Science department of the University of Michigan, where it supports 5000 students per semester spread across a dozen courses.

This documentation is intended for instructors. If you are a student, please contact your instructor for assistance.

Autograder.io offers:

• State-of-the-art security using Docker containers as a foundation.

• Insulating sensitive instructor files from student code and preventing information leakage.

• A sophisticated, highly customizable feedback system.

• Students can work and submit in groups.

• Elegant displaying of test results, including output diffs.

• Flexible grading policy and deadline options.

This documentation is a work in progress. Contributions are welcome through pull requests to our GitHub repository.

Getting Started

• Setting Up Your Course
  • First-time course creation
  • Copying a Course from a Previous Term
  • Add Admin, Staff, and Student Users
• Creating Your First Project
  • Set the Hard and Soft Deadlines
  • (Optional) Enable Group Work
  • (Optional) Limit How Many Times Students Can Submit Per Day
    • Set the timezone for “Reset submissions per day at” to your local timezone
  • Choose a Grading Policy (Best or Most Recent Submission)
• Adding Test Cases to Your Project
  • Upload Instructor Test Case Files
  • Configure Files Students Should Submit
  • Add Test Cases
    • Create a Test Suite with Python Unit Tests
      • Create the First Test
      • “Test” the Test Case Settings
      • Clone the Test Case to Complete the Suite
    • Create a Test Suite with C++ Output Tests
      • Create and Configure the Test Suite
      • Create the Test Cases
• Final Steps
  • Publish the Project
  • Downloading Grades and Submitted Files
  • Showing Students their Final Graded Submission

Topics

• How-tos and FAQ
  • Clone a course or project for a new term
  • Let students submit even if they’re not in the roster
    • 1. Restrict guests to your email domain (Optional, once per course)
    • 2. Enable the “Anyone with the link can submit” project option
  • Grant an extension
  • Edit group members
  • Rerun a test case or submission
    • Rerunning a “stuck” or “errored” submission
  • Why do I get “permission denied” when I run my script with ./my_script.sh?
  • Can I use MATLAB?
  • Why does my sandbox image build time out?
  • What happens if a student has multiple “best” submissions?
• Hard & Soft Deadlines: Recommendations and Implications
• Building Custom Sandbox Images with Docker
  • Tips and Tricks
• “Bonus Submission” and “Late Day” Tokens
  • Bonus Submission Tokens
    • Giving Extra Bonus Submission Tokens
  • Late Day Tokens
    • Giving Extra Late Day Tokens
    • Edge Cases
• Grading Style and Code Quality by Hand
  • Create and Configure the Handgrading Rubric
    • Configure the Basic Settings
    • Add Checkboxes
    • Add Annotations
  • Grade Submissions
  • Publish Grades and Rubric
• Submission Email Receipts
  • Verifying Submission Emails with the Verification Link
  • Verifying Submission Email Receipts Manually
• Rejecting Submissions that Fail Required Checks
  • Caveat: Late Day Tokens