Teacher Test Repository

Official Test Respository: fork/clone me

Yes. Just use this as your starting point. We even included a basic .hgignore for you.

Test Repo Structure

top-level-repo-folder
    - config.yaml
    - runner.py
    - test_<name>.py
    - any other files
    - any other folders/*

config.yaml

This is Aurum’s test repo configuration file. It describes how Aurum uses your test code to run against students’ input.

See config.yaml Configurations for configuration details.

Here is an example:

 student_file: student.py
 run:
     student_soln_exe: student.py
     test_runner_file: runner.py

 nose_report_file: nosetests.xml
 run_student_file: True

full_scores:
    test_it_runs:
        full_score: 1
        description: Does it run without crashing?
    test_image_found:
        full_score: 1
        description: Did you save out an image?
    test_image_matches:
        full_score: 2
        description: Is the image the correct image with colorbar?

Warning

Do not use TAB in any YAML file because it will raise error when YAML tries to parse the file. Use spaces!

runner.py

This file is required because this is the main program Aurum runs. It basically tells Aurum to execute nosetest and make a report afterward.

You can find this in the official repository at the end of this page.

test_<name>.py

This contains the test code written in Python using the unittest module. We then use nosetest to run the test code and generate a report from the test result.

The sample teacher test repo above has one test file called test_c_exercise.py:

Warning

The current Aurum system is running on Python 2.6.5. Therefore, certain unittest features are not present in that module. You have to use unittest2. We have this module installed but to make things simpler, if you need to run on both 2.6.5 and 2.7+, you should consider doing:

try:
    import unittest2 as unittest
except ImportError:
    import unittest

If you use our official test repo above, we include this for you.

Full Examples

We have compiled some fully working, non-trivial examples for you to look at.

Python:

find_evens: bitbucket

Best Practices

We consider these as best practices to avoid problems such as cheating in a smart way. The things we mention here will reference options in config.yaml and writing unittests.

1. If you want to commit your solution in the respository, avoid using common file names such as soln.py or solution.py (or .cpp, you get the point here).

   This is a big issue in Python because you can import the solution. There are some workarounds such as adding directories so students have to guess where the actual solution lives. This is obviously tedious work. If you plan on making the test code public, you have to work around by deleting (or even mocking out) the import at run time.

   We actually do encourage you to commit your solution code because you can version control it. Just don’t use common names!

2. Use try: import xx except: ... unittest method we recommend in a section or two above. Use that.

3. Avoid committing large files such as data. They should live somewhere else. For example, Dropbox or Google Drive. If you have to commit a few images, that’s fine. If you have more than a few images or very complex csv files, try hosting them somewhere else. Or talk to us. We can sort out the most practical way to do this with you.

4. Do not forget your .hgignore file. See Include .hgignore.

5. Do not forget to perform some cleanup in your tests. Very often the assignments may involve some file generations that are useless afterward (i.e. download a sample csv file so student code can run against this file) because they don’t count toward student’s result, then you should perform tearDownClass in your unittest test case.

Note

We recommend you fork / clone our oficial respository on Bitbucket.