Creating and Editing Websheets

To get started, we recommend diving in by opening these two links, which demonstrate an exercise called TwoPlusTwo:

public static void main(String[] args) {
\[
   System.out.print(2+2);
]\
}

Moreover, the Template / Reference Solution specifies a correct "reference" solution, which is to fill in that blank with System.out.print(2+2);.

(The grader actually runs the code, so any other solution that behaves the same way is just as good. So System.out.print(4); is correct too.)

Here are the sections of this manual, which describe how to write each component in more detail:

Examples

You can access a list of all Websheets (login required). There, you can see the definition of about 200 sample exercises in Java and C++.

Short Answer / Multiple Choice

• short answer:
• multiple true/false:

Template / Reference solution

Once you fill in a working program, you need to decide on what parts you want to blank out for the student. Websheets supports two kinds of blanks:

• inline blanks which start and end on the same line; they can expand horizontally, but can never contain newlines
• multi-line blanks which take up the entirety of several lines; they can expand vertically, but never can start or end in the middle of a line

Both kinds of blanks are indicated by using \[ to denote the start of the blank, and ]\ to denote the end of the blank. The interior region will be left blank and editable for the user.

The rules for how these delimiters must appear in the reference solution / template are:

• inline blanks are defined by \[ and \] on the same line
• multi-line blanks are defined by \[ on a line by itself, followed some lines later by ]\ on a line by itself.

For example, look at how Distance defines both kinds of blanks.

(In the multi-line case, additional whitespace on the lines containing the delimiters is currently ignored, although it could be meaningful in a later version, say where Python is used and we want to force indentation of a region.)

Hidden code, fake code, default contents for blanks

Feel free to skip this section if you're reading the documentation for the first time.

The simple \[...]\ delimiters are sufficient to describe a large variety of exercises. But to allow the exercise writer more flexibility, websheets allow a few additional kinds of regions.

You can use \hide[...]\ to enter code that will be hidden from the student's UI. It will be included in both the reference solution and the student's combined solution that gets executed upon submission. For examples, see

• , where it is used in-line to hide a call to .clone()
• , where it is used multi-line to include an inner class

You can use \fake[...]\ to display read-only text in the user interface that looks like it is part of the program, but that won't actually be included. For examples, see

• , where it is used in-line to show pseudocode in the middle of a program
• , where it is used similarly, but to contain multiple lines

You can use \show: to have a blank initially populated with some text. (Think of providing a buggy program fragment for a student to fix.) To do this, change \[ blanked-out text ]\ to \[ blanked-out text \show: default text here ]\. For example, see

• , where it is used in multi-line syntax to give a strawman constructor
• (No current example uses the in-line syntax.)

Cheatsheet: What elements appear where?

The exercise description (HTML)

LaTeX. All websheets have MathJaX processing turned on. This means that you can include LaTeX using $inline format$ or $$big centered display format$$. Look at for examples of both.

Links. Sometimes you'll want to add a link to another websheet inside of the description. One way to do this nicely is to write a link with a URL like

<a href="javascript:websheets.load('java/00-intro/SquareSwap')">SquareSwap</a>

Diagrams. You can use any software and any formats you like, but the example exercises using diagrams (e.g. ; ) were created using IPE (which uses PDFs as an editable format) and rasterized to PNG using render.sh.

Java test suite

Because tests will be spliced in to a Java program, any valid Java code can be used. For example, instead of writing

testMain("AA");
testMain("BB");
…
testMain("ZZ");

for (char ch='A'; ch<='Z'; ch++) testMain(""+ch+ch);

Basics: testing static methods

testMain(String... args)

call main on these arguments.

E.g. see

test(String methodName, Object... args)

call a static method named methodName on this list of arguments

E.g. see

note: if there's exactly one argument, which is an array, Java requires you use

test(methodName, (Object) /* your array here */)

due to inherent ambiguity in varargs. E.g. see . The ambiguity doesn't exist if there are multiple arguments of which some are arrays, e.g. see .

If you're reading the manual for the first time, now would be a good time for you to try creating some sample exercises; you've now read the documentation for the essential mechanics.

Basics: testing instance methods

saveAs = "attendance"; // (1)
testConstructor(); // (2) i.e. attendance = new Clicker();
testOn("attendance", "curr"); // (3) i.e. attendance.curr(); 

• (2) call the zero-argument constructor defined by the student,
• (1) save the result as a variable named attendance,
• (3) call the instance method curr with no arguments on attendance object.

saveAs = String  (default null)

save the value returned by the next test as a named object. (implementation note: we maintain two parallel namespaces for student and reference solutions.)

Note: you can save anything, not just constructor calls. E.g. in , we save a method's return value:

saveAs = "scaled";
testOn("x", "times", 10.0);

testConstructor(Object... args)

call the constructor with this list of arguments

testOn(String instanceName, String methodName, Object... args)

on the named instance, call the named instance method with this list of arguments

var(String)

once an object is saved, you may want to pass it an argument to a method, rather than just calling its instance methods. For example in , after constructing two objects x and y, we call x.plus(y) like so:

testOn("x", "plus", var("y"));

basically, the var() wrapper is needed to distinguish the saved variable y from the string literal "y"

Exceptions and standard input

expectException = boolean  (default false)

indicates that the student code should throw an exception on the following test. (it will catch whatever exception the reference code throws, and expect the student code to throw the same class of exception)

E.g., .

stdin = String  (default null)

for the next test, set the contents of standard input to the following String

E.g., .

stdinURL = String  (default null)

same as stdin, but fetch this webpage and use its contents. Useful when there is a long data set posted online, and you prefer not to copy a long text into the websheet definition

E.g., .

note: use simple string literals only. e.g. stdURL = "http://goo.gl/" + "robots.txt"; is bad, it will confuse the prefetcher that grabs the data before starting the sandbox.

Further methods

remark(String s)

Print out a remark to the user explaining what the tests are doing. Useful when you still want them to see the automatically-generated description of the test, but you also want to explain more context to the student. s should be valid HTML.
c.f. title = which replaces the automatically-generated test description.

E.g., .

E.g., .

store(Object o)

Copy an object to the user and reference namespace. Used for checking shallow copying in .

construct(String pckg, String clss, String type_prms, Object... args)

Use this to create a new instance of some other class. pckg should be a fully qualified package or null to use student/reference code; clss should be a simple class name; type_prms should be "" if you are not constructing a generic, or a list of type arguments for a generic if you are (this is just shown to the student in the description of the test since generic erasure happens at runtime); and args are the consturctor arguments. It is best seen by these two examples:

In we call

construct("stdlibpack", "Queue", "<Integer>")

to mimic a call to new Queue<Integer>() (the package stdlibpack has the COS 126 "standard" classes, and we want the Queue from that package)

In we call

construct(null, "Person", "", "Tony")

to mimic a call to new Person("Tony") — Person is in the dependencies of that websheet. Leaving pckg as null means that the student's code uses the Person from the student package while the reference uses the one from the reference package.

testNamedMain(String methodToCall, String fakeClassName, String... args)

Call some hidden internal method, but pretend you are calling main(args) on some other testing class. Used in at testNamedMain("exampleClientMain", "ExampleClient"); to run the hidden method exampleClientMain but to tell the student we are running java ExampleClient.

Further options

title = String  (default null)

replace the automatically-generated description of the next test with the given HTML. Useful if you are using a lot of fake or hidden code and the automatically-generated description the student would see in testing would make no sense.
c.f. remark() which adds extra commentary before the next test

E.g. .

E.g. .

quietOnPass = boolean  (default false)

Don't show the next test unless the student fails it. Especially useful if you have a lot of tests.

E.g. .

E.g. .

maxOutputBytes = int  (default 10000)

Change the maximum number of bytes the student code can print per test before it is terminated.

E.g. .

E.g. .

dontRunReference = boolean  (default false)

Don't run the reference solution. Instead, rely on either template code or hidden code that will determine if the student passes, and which will throw an exception if they failed, but not if they passed.

E.g. .

E.g. .

Note that hidden code can make thrown exceptions look a bit nicer by using websheets.Grader.FailException like FTPLimiter.

cloneForStudent = boolean  (default true)

Before passing arguments to the student, clone them. So if the student mutates the arguments, it doesn't affect the grader.

This is turned off to check shallow copying in .

cloneForReference = boolean  (default true)

Same as above, but for reference code. No current example.

realTolerance = double  (default 1E-4)

Relative error accepted for outputs. (Except if reference answer is 0, this is the absolute error accepted.) No current example.

ignoreRealFormatting = boolean  (default true)

If false, ignore realTolerance, also don't accept 5.00 as the same as 5.0. No current example.

ignoreTrailingSpaces = boolean  (default true)

Ignore trailing spaces at the end of each line. No current example.

Misc

defaultOptions.quietOnPass = true;

Java-specific optional websheet properties

classname

when you want the visible and executed class name not to be the same as the name of the overall exercise (the Python module name, referred to internally as the slug).

E.g. .

dependencies

require student to complete some other websheet before this one; the student's most recent correct solution to the dependency will be made available when compiling executing this one.

E.g. .

Note: if instead you want to give student extra reference code without requiring them to actually complete the dependency, you can use \hide instead:

E.g. vs .

imports

a list of classes to import (will be made visible in UI and added to reference solution)

E.g. .

C++ test suite

C++ (call main with stdin and args)

For each run, you specify the standard input and the command-line arguments, both of which are optional. Specifically, for our editor, each test is a JSON dict with optional keys stdin (a string) and args (a list of strings), and the test suite is a list of tests.

Take a look at which uses both stdin and args. Many other C++ examples use just one of stdin or args but not both.

You can add "hidden":true to a test to make the input now be shown. You can add "title":"your title here" to label a test with a title. Thanks to dz0 for this feature!

A test that runs with no inputs is an empty dict, so a test suite that just runs one test with no inputs would have test suite [{}]. See .

C++ (call functions directly)

A test suite for this engine is a list of commands, where each command has one of two types:

1. ["check-function", "function name", "return type", ["arg 1 type", "arg 2 type", …]]
   which checks that a function of this name and exact signature exists
2. ["call-function", "function name", ["arg literal 1", "arg literal 2", …]]
   which calls the function and checks that its return value, when printed, is correct

C++-specific optional websheet properties

Is example?

Used to indicate a chunk of code that is being included (perhaps by embedding) just for demonstration purposes.

E.g. .

Examples can be editable, e.g. which is used to kick off an in-class discussion. Sometimes this is also useful when your exercise can't really be autograded properly due to technical constraints. See which uses randomness and which involves timeouts.

Multi-language optional websheet properties

epilogue

show a message or explanation after problem is completed.

E.g. .

Forbidden substrings

Resrtict students from including these words in their code.

E.g. .