M. Ian Graham, David Kitchin

A hard AI problem can be defined as a problem in artificial intelligence that has gone unsolved for a long period of time, even under intense research. In other words, a hard AI problem is a problem which is solvable by humans but not readily solvable by computers. A CAPTCHA is a program which generates and grades hard AI problems in order to automatically verify that it is communicating with a human and not another program. (www.captcha.net)

A program can pass a CAPTCHA test (thereby thwarting the CAPTCHA) through one of two methods. The first is solving the relevant hard AI problem, something which is unlikely to occur. The second is having access to a "human oracle"--that is, the program must be able to query a human resource and use the answer in its evaluation.

A human oracle is not only a tool for breaking CAPTCHA tests; it provides an extremely powerful augmentation to the computational abilities of any program which may query it. With access to a human oracle, hard AI problems become solvable. A computer will suddenly be able to handle issues such as emotion, symbolism, and general recognition much more easily.

We explore the problem of creating a human oracle system, under the assumption that paying participants for their time is financially prohibitive. We propose that an economical human oracle may be fashioned by embedding a hard AI problem in an online game. Specifically, we create a human oracle system for general image description and classification, with associative, symbolic, and emotional qualification. The game in which the problem is embedded is entertaining for players, who might not even know their answers are used as resources. The system receives an image as input and returns an intelligent description as output.

To our knowledge, this is the first implementation of a practical and economical human oracle system.