Publication: Beyond independent agreement: A tournament selection approach for quality assurance of human computation tasks
Date
02-11-2011
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Quality assurance remains a key topic in human computation research field. Prior work indicates independent agreement is effective for low difficulty tasks, but has limitations. This paper addresses this problem by proposing a tournament selection based quality control process. The experimental results from this paper show that the human are better at identifying the correct answers than producing them themselves. Copyright © 2011, Association for the Advancement of Artificial Intelligence. All rights reserved.