Competitions for (automatic) verification tools compare tools against a common set of benchmarks and typically compare results based on how many of the benchmarks can be verified within a given time. The need to have a common input/output language and the creation of a growing set of benchmarks helps drive a research area forward.
(There are also competitions for [auto-active verification] tools and interactive verification tools but these also test the expertise of the humans driving the tools.)
Papers related to Verification tool competition
- Benchmarking solvers, SAT-style [nyxbrain:sc2:2017]
- Evaluating general purpose automated theorem proving systems [sutcliffe:ai:2001]