Assignment 5: Property-Based Testing

Due Monday, April 13

Goals for This Assignment

By the time you have completed this work, you should be able to:

• Use basic generation and testing functionality of an existing property-based testing library
• Write properties, and test code with these properties

Step-by-Step Instructions

Step 1: Choose a Language and Property-Based Testing Library

For this assignment, template code in Java has been provided, which uses the junit-quickcheck library. However, you do not have to use this code; you may use any language and any existing property-based testing library. If you choose a different language or library, you'll need to port the template code over, though most of this should be straightforward (there isn't much implementation code). If you choose to port the code to another language or testing library, you will get a 30% bonus on this assignment. This bonus is intended as compensation for the extra work involved in porting the code, and a reward for exploring something we didn't cover in class.

If you're interested in working with another language/library, a good starting point is the Wikipedia page on QuickCheck. All the listed languages are annotated with citations corresponding to property-based testing libraries written in those languages. QuickCheck itself is a property-based testing library for the Haskell programming language, and introduced the idea of property-based testing in 2000. While the Wikipedia page equates property-based testing and QuickCheck, QuickCheck is an instance of a property-based testing library, not property-based testing in and of itself.

If you choose to work with a different language / library, be sure to have a README.txt file which explains how to run your properties, ideally with a single command. I should not have to understand how to use your specific language and property-based testing library in order to run your code, and a single command for me to run should give me everything I need.

The rest of these steps assume you're using the provided template code, as opposed to porting it. You may need to tweak these steps for your own port. If you're not sure what you need to change, don't hesitate to ask.

Step 2: Download, Compile, and Test Needed Code

Download the template code here. This template code encapsulates a project that encodes natural numbers in Java, without ever using any of the built-in primitive types (i.e., no int, long, double, etc.). If you're interested, the particular encoding is based on the Peano arithmetic, though you shouldn't need this information.

The provided template code comes with the following:

• A complete, (hopefully!) bug-free implementation, underneath src/main/java/arithmetic/*
• A series of unit tests, in src/test/java/arithmetic/NatUnitTest.java. These tests make use of JUnit.
• A generator of natural numbers, in src/test/java/arithmetic/NatGenerator.java. This generator is written for the junit-quickcheck library.
• A single property-based test in src/test/java/arithmetic/NatPropertiesTest.java. This again makes use of junit-quickcheck, along with the aforementioned natural number generator.

The provided code should already compile with Maven (mvn compile). Similarly, the code should already be testable (mvn test), which includes the aforementioned property-based test. Make sure the above commands work before moving on. If the commands don't work, you may need to first edit pom.xml, as described in Assignment 2.

Step 3: Write Properties, With Limitations

Write additional properties in src/test/java/arithmetic/NatPropertiesTest.java, so that the code is much better tested than it currently is. The following restrictions apply:

• You may not simply convert Nat into a more Java-friendly type like int. Your properties need to work directly with values of type Nat.
• Each property must take at least one Nat as an input, and this input must somehow be used in an assertion (e.g., assertEquals, assertTrue, etc.). You cannot simply write unit tests under the guise of properties.
• Collectively, your properties must get you 100% code coverage. If you're writing good properties, this should not be difficult to acheive. From my own code, 1-3 properties involving a given method were sufficient for this, and the same property usually used multiple methods. If you're not using Java, you can skip this requirement (I don't want to put you through the extra trouble of finding an appropriate code coverage library).

Step 4: Submit Everything

Zip up everything in your arithmetic directory, including the target directory. Be sure your target directory holds up-to-date coverage results. Name your zipfile arithmetic.zip, and submit it on Canvas. In the comments for the submission, list everyone you worked with, if applicable.

If you did not use Java or junit-quickcheck, be sure to submit a README.txt, as specified in Step 1.