Metamorphic testing is a well-established technique for testing software for which no oracle is readily available. The idea is to exploit known properties of an application domain to check consistency of the behaviour of a software application for particular inputs, without knowing what the right behaviour to be expected is for these inputs. The simplest example of metamorphic testing is that to test the trigonometric function “sin”, we could check whether, for a given value x, sin(x) and sin(x + 2π) are found to have the same value (modulo round-off error). If they do not, “sin” has been implemented incorrectly and the mismatch provides a starting point for debugging.

In recent work my group others have investigated applying metamorphic techniques for testing compilers, and are now exploring ideas for how to apply metamorphic testing to program analysers more generally. In particular, we are interested in exploring the use of semantics-preserving program transformations as a basis for checking the consistency of analysis results. In the talk I will demonstrate this in action for automated testing of compilers for GLSL, the OpenGL shading language, which is difficult to test in a traditional manner due to its deliberate lack of strict floating-point semantics. I will then talk about ideas for how to use program transformations as a basis for “analysing the analyser”, testing both correctness and performance characteristics of program analysis tools.

Alastair F. Donaldson is a Senior Lecturer in the Department of Computing at Imperial College London, where he leads the Multicore Programming Group. Prior to joining Imperial, he was a Visting Researcher at Microsoft Research Redmond, a Postdoctoral Research Fellow at the University of Oxford, a Research Engineer at Codeplay Software Ltd., and a PhD student at the University of Glasgow.