Parsers are everywhere. Novel and not-so-novel parsing tools (and occasionally new ideas) emerge routinely. We have a jungle of techniques, summarized ably by Grune and Jacobs [2], but little guidance as to the quality of implementations. In fact, judging the utility and performance of tools is hard for experts, and can be bewildering for users, who mostly want a black-box parser that ‘just works’. We suggest a community effort to create a set of standard challenges with two goals: (a) helping users identify the right tool for their use-case, and (b) helping us improve the utility and performance of our implementation by providing an element of competition. These kinds of standard challenges have proved successful in developing the research agenda of areas such as graph drawing (http://www.graphdrawing.org), grammar induction (http://www.grammarlearning.org) and in broad areas of data science (http://www.kaggle.com)