Determining review coverage by extracting topic sentences using a graph-based clustering approach

Reviews of technical articles or documents must be thorough in discussing their content. At times a review may be based on just one section in a document, say the Introduction. Review coverage is the extent to which a review covers the “important topics” in a document. In this paper we present an approach to evaluate the coverage of a submission by a review. We use a novel agglomerative clustering technique to group the submission’s sentences into topic clusters. We identify topic sentences from these clusters, and calculate review coverage in terms of the overlaps between the review and the submission’s topic sentences. We evaluate our coverage identification approach on peer-review data from Expertiza, a collaborative, web-based learning application. Our approach produces a high correlation of 0.51 with human-provided coverage values.