We modified UIUC's PrairieLearn service to add fully-configurable, server-side autograding to student submissions. All our contributions are currently on a fork called PrairieLearnHack.
- Tejasvi Nareddy
- Nathan Walters
- Genevieve Helsel
- Jordi Paris Ferrer
- Matt West
- Dave Mussulman
- Jake Bailey
- Kevin Wang
We spend most of Friday night and Satuday morning understanding the codebase. The github repo had extensive documentation on how the code was structured, so we read through that. Additionally, one of PrairieLearn's mentors helped us understand how all the different files and machines were interconnected.
We spend most of Saturday brunch and afternoon designing the autograding service. We created a full network diagram on a chalkboard and decided on the program flow. Even though we spend 3+ hours on this, we felt that this was important, as we all knew what to do from the beginning.
We split the work up into sections and created pull requests to the fork for each one.
- Documentation that defines enabling autograders: https://github.com/PrairieLearn/PrairieLearnHack/pull/1
- Created some dummy autograders that execute dummy student code: https://github.com/PrairieLearn/PrairieLearnHack/pull/2
- Created the grading service to run in a seperate machine and spawn docker containers for each student submission: https://github.com/PrairieLearn/PrairieLearnHack/pull/3
- Hooked up the PrairieLearn server to the grading service over sockets: https://github.com/PrairieLearn/PrairieLearnHack/pull/4
- Bug fixes while testing for 12 hours (yay): https://github.com/PrairieLearn/PrairieLearnHack/pull/5
- Documentation that defines running the grading service: https://github.com/PrairieLearn/PrairieLearnHack/pull/6
- Professors can add autograders to PrairieLearnHack. They can fully configure which environments and which tests are run with each autograder.
- Students can trigger an autograder by submitting code online
- Grading runs in a seperate one-time docker container for each student submission using a master-child configuration. Grading happens securely to prevent student code from comprimising the docker container.
- Students are notified of results in real-time using queuing services.
We extensively documented our changes to PrairieLearnHacks. All of our defined standards for json sending and receiving between the two servers (PrairieLearn and the Grader) are defined in markup files such as this one. All the changes that professors need to make to upload autograders is also documented throughout the rest of the documentation. All these markup files are linked to the repo's Readme.md.
We also made sure to comment our code! All our pull requests have comments too.
Currently, this only runs on localhost through sockets. We did not have time to implement AWS cloud functionality, so our code is not live on the PrairieLearn website. We planned on using AWS SQS as our queuing service, S3 for storage of autograders and student submissions, ECS for running the graders in docker containers, and EC2 for the main PrairieLearn server. All the components work locally by just running different scripts at the same time.
