The Wikipedia Corpus

From Wikipedia Quality
Jump to: navigation, search

The Wikipedia Corpus
Jeff Pasternack
Dan Roth
Publication date

The Wikipedia Corpus - scientific work related to Wikipedia quality published in 2008, written by Jeff Pasternack and Dan Roth.


Wikipedia, the popular online encyclopedia, has in just six years grown from an adjunct to the now-defunct Nupedia to over 31 million pages and 429 million revisions in 256 languages and spawned sister projects such as Wiktionary and Wikisource. Available under the GNU Free Documentation License, it is an extraordinarily large corpus with broad scope and constant updates. Its articles are largely consistent in structure and organized into category hierarchies. However, the wiki method of collaborative editing creates challenges that must be addressed. Wikipedia’s accuracy is frequently questioned, and systemic bias means that quality and coverage are uneven, while even the variety of English dialects juxtaposed can sabotage the unwary with differences in semantics, diction and spelling. This paper examines Wikipedia from a research perspective, providing basic background knowledge and an understanding of its strengths and weaknesses. Authors also solve a technical challenge posed by the enormity of text (1.04TB for the English version) made available with a simple, easily-implemented dictionary compression algorithm that permits time-efficient random access to the data with a twenty-eight-fold reduction in size.


Wikipedia Quality

Pasternack, Jeff; Roth, Dan. (2008). "[[The Wikipedia Corpus]]".

English Wikipedia

{{cite journal |last1=Pasternack |first1=Jeff |last2=Roth |first2=Dan |title=The Wikipedia Corpus |date=2008 |url=}}


Pasternack, Jeff; Roth, Dan. (2008). &quot;<a href="">The Wikipedia Corpus</a>&quot;.