Requirements on Instances and Test Programs

Here is some information about the instances and test programs required for a particular benchmark.

We need five instances for the test and installation phase, and at least ten for the competition itself. Not all instances provided for the competition phase will be used during the competition phase. Just as in the previous ASP competition, too hard or too easy instances will be deleted. The instances for the competition phase will be drawn randomly from the provided instances, so please send enough to ensure a fair selection. For more information on the required format of the instances, please consult the subpage on "Input and Output format".

For each benchmark problem we need a test program. For a decision problem, the input of this program will consist of the concatenation of the input for the problem instance and the witness produced by a solver. This input will be read from standard input, and will consist of a collection of atoms of input and output predicates. The output on standard output will be either the string "CORRECT" or the string "INCORRECT".
Notice that the test program will not be called when a solver generates UNSATISFIABLE for an instance.

A test program for an optimisation problem will be called with a similar input, namely the concatenation of the atoms representing the instance problem and the atoms representing the last witness generated by the solver for this instance. This will be read from standard input. The output will consist of three strings separated by a period "."
1) "CORRECT" or "INCORRECT".
2) an integer representing the cost value.
3) "OPTIMAL" or "NON-OPTIMAL" or "OPTIMALITY-UNKNOWN".
Note that the option of returning "OPTIMALITY-UNKNOWN" gives the writer of the test program the freedom not to include an optimality check in the program, in case this will be too big an effort. In practice, your checker should not check for optimality unless the check is tractable. (The competition software assigns 30 seconds to the checker.)