Surveys and questionnaires are widely used for both exploratory and confirmatory data analysis. However, the design of survey artifacts remains more art than science. In particular, we know that the survey design itself can bias participants’ responses in ways that may be difficult to detect post hoc. Consequently, there are few systematic ways to address flaws in the design of the survey instrument.

This talk will present the SurveyMan language and runtime system. SurveyMan is a randomization-first tabular language for designing, debugging, and deploying surveys on the web. We will walk through the construction of example surveys to illustrate how SurveyMan’s language features address potential biases. We will then show how the language features allow us to apply constraints to the survey, and how a particular survey structure implies or preclude analyses.