CS 146: Elementary Algorithm Design and Data Abstraction (Advanced Version)

Using Marmoset to submit assignments

Basic Logistics

  1. Go to https://marmoset.student.cs.uwaterloo.ca/ and log in using your WatIAM info. This should be the same as the info you use to log in to Quest.
  2. Click the "as" button under "Authenticate". You should have only one choice here.
  3. Click "CS146 (Winter 2020):".

    Note: If you don't see what you're supposed to see in steps 2 or 3, then you are probably not yet added as a student to CS 146 on Marmoset. Contact the course staff to resolve this issue.

  4. You should now be able to see the assignments that have been set up on Marmoset for CS 146. You can submit your assignment files to Marmoset via the "web submission" column for each assignment question.

Submitting Assignments

If only one file is required for an assignment question, you can submit only that file. If multiple files are required for an assignment question, you must zip all of the required files and submit the zip file. Make sure that all of the required files are named correctly; otherwise, you will receive "did not compile" error message.

Tests on Marmoset

There are two types of tests that we will use on Marmoset in CS 146: public tests and release tests.

Public tests are designed to test whether your program works in the most basic sense possible. They are simple tests that test for trivial cases.

Release tests are designed to test whether your program is correct. You can expect these tests to test all the dark corners and special cases that can potentially break your code.

After Submitting a Program

When you make a submission to an assignment question on Marmoset, your submission will be automatically tested on a CSCF server. After a while (a few seconds to a few minutes, depending on the state of the server, the program and test complexity), the tests should finish and a result will be available.

Interpreting Test Results

If your submitted program does not compile or run successfully on its own, your submission will receive a result of "did not compile" and the detailed test results will contain something similar to the error message you get if you ran your program yourself. In this case, your submission will not be tested with any of the tests.

If your submitted program runs successfully on its own, it will be tested with all of the public tests.

If it fails any public test, the detailed test results will display an error message for that public test. In this case, your submission will not be tested with any of the release tests.

If it passes all of the public tests, you will have the option to view the result of the release tests. If you do so, you will use up one of your "release tokens" for that question. Now, for every assignment question, you are given 3 release tokens (this number may be higher for early assignments). If you use up one or more of them, one release token will regenerate once every 12 hours, until you have 3 release tokens again. Start your work early if you want to have more chances to see the results of the release tests. If you run out of tokens and you don't expect them to generate before the deadline, you can still resubmit, although you will not be able to tell how your submission did on the release tests.

Marmoset automatically tests each submission with all of the release tests, in some order specified by the course staff. If your submission fails a release test and you use a token to see the results, you will only see that test and one more test in the detailed test results. If your submission passes all the release tests, you will not see any release tests in the detailed test results, but you will be credited with full marks for that question.

If you fail a release test, you will get a very small amount of information about what went wrong. You will not be given details of the test case that you failed. Do not attempt to guess what that test case might be; do not ask about it on the forum, and do not speculate about test cases on the forum. The correct action when failing a release test is to re-examine your own test suite and redesign it to find the error in your code or your assumptions.

You can continue to submit and see the result of release tests after the deadline has passed, though post-deadline submissions do not affect marks. It's a good idea to finish questions on which you ran out of time, to make sure that you've done all the learning.

Common Marmoset Test Messages

Below are some possible release test results:

"Success: test X passed", where X is the name of the test. You can pat yourself on the back for this one.

"Error: wrong output for test X". This means that some function you were required to write did not produce the value we expected, or (later in the course) that some part of your program did not produce the output that we expected.

"Error: program ran out of resources while running test X". This means that your program took too much time or too much memory during the test. The "short result" report should be ignored in the case of a timeout because we use our own timeout mechanism, not Marmoset's.

"Error: program encountered error while running test X". This means that your program halted with an error during the test.

"Error: program ran out of time while initializing". This happens if your program takes too much time when it is evaluated by Marmoset, before any of our tests are run.

"Error: program ran out of memory while initializing" (same as above, but with memory).

"Error: program failed to initialize". This happens if you use the wrong language level, if you have non-text elements in your program (such as values snipped out of the Interactions window and pasted into the Definitions window), or if you are doing something that is not permitted, such as file I/O.

Grading

There is no penalty for multiple submissions. Your best submission counts. The only thing stopping you from spamming Marmoset with many submissions is your own conscience, so don't do it. Please remember that the server is a shared resource; out of courtesy to your fellow students, do not do anything that overloads it, especially close to deadlines.

 

Last modified on Monday, 06 January 2020, at 09:50 hours.