(This refers to a Wikipedia Review blog post.)

Look, I have no real axe to grind, but if you're going to quote froma study, it is better to quote the study than the summary. Have you actually read the study? Have you considered the limitations of the study as noted in the summary? It states "More research must be undertaken to analyze Wikipedia entries in other disciplines in order to judge the source's accuracy and overall quality. This paper also shows the need for analysis of Wikipedia articles' histories and editing process." This study hardly appears to be the smoking gun that the post claims. It just seems intellectually dishonest to complain about the poor quality on Wikipedia when the argument is based on on a summary of a study rather than the study itself. This is tabloid sensationalism, not real criticism. As to the coloring of the demise of Quid as "a legitimate, credible body of work has become the first major conquest in Jimbo Wales’s cultural war", wow, that's some hyperbole. Looking at the Independent article, I note sales slumped from 450 000 to 100 000 over possibly a ten year period, it's hard to pinpoint because the highpoint is labelled only as the 90's. That's not that bad when looked at in relative terms across publishing in Europe, where publishing sales are looking like entering a recession, for example compare newspaper or magazine sales. The article certainly presents a sensational reading of the demise, but there are so many other factors to be considered. Did the company over-expand in the boom years? Is there mis-management? Is the publisher trying a marketing tactic, trying to appeal to France's fierce national pride and love of tradition? It's plausible. Attack Wikipedia all you want, but do it on your own terms. As To Jimmy Wales and his cultural war, I don't think it's so much that. I think a lot of it just fell into Wales lap. He's now rather busy trying to work out how to make money out of it, something the Wikipedia community has amazingly stymied. Whatever sins you can paint on Wikipedia's doors, I think there should be some recognition of the fact that the community as a whole is opposed to advertising. That will probably kill the project in the long term, but a lot of the criticism of Wikipedia seems to focus on one aspect of the project, ignoring many other aspects which are instructive. Yes, the power structure sucks, and yes the shambolic nature of the system is far too prone to gaming. But somehow sometimes things emerge from the site in spite of that. And sometimes the criticism is very off kilter.

(This refers to the latest blog post.)

When two reports state "France's favourite encyclopaedia falls victim to Wikipedia" and "This study does support the claim that Wikipedia is less reliable than other reference resources", how is it intellectually dishonest not to repeat those statements and draw a fairly obvious conclusion?

Are you asserting that Wikipedia had no hand in Quid's demise, or that Wikipedia is as reliable as traditional established encyclopedias?

You write, "Attack Wikipedia all you want, but do it on your own terms."

I just did.

I prefer to base my opinions on more than headlines and summaries, I like to judge issues for myself. If you want to base your opinion on someone else's headline, that's up to you, but that's not making an attack on your own terms. That's being someone else's attack dog. The article on Quid is misleadingly headlined and the study is a study of 9 articles. Do you happen to know which nine articles? And do you happen to know how they were compared, what weighting was used? Until you know that, you don't actually know what you are saying. As I stated above, look at the limitations the study itself declares; More research is needed. You don't use a study based on such a small sample to declare anything. The people who wrote the paper know that. Why pretend otherwise. It's dishonest to make claims the authors of those claims do not make. The authors did not say 8 out of 9 articles are bad. They stated 8 out of the 9 articles reviewed were of a comparably lower standard when compared to comparable articles in Encyclopaedia Britannica, The Dictionary of American History and American National Biography Online.

I quoted the reports findings verbatim. I didn't say "8 out of 9 articles are bad". I included this quote:



...and then highlighted two statements in these reported findings and placed them in quotation marks, so your claims here have no merit.

I guess we could go into the report itself. But what are likely to find? A set of data that differs from the reported findings?

I apologise for saying you wrote 8 out of 9, I misread the second but one sentence and didn't see the word examine, that is my fault.



From that sentence I infer that you have not read the report itself. I've pointed out why I believe one should survey all the evidence before making an assertion. You are more than free to disagree, I just think it is intellectually dishonest to pronounce upon something from a position of relative ignorance. I also don't believe the two sources you make justify the assertion that the "collapse of further credible sources on the horizon" means "our future looks to be dominated by a leviathan". Perhaps if we had an example of any other field in which a monopoly exists without state intervention, your assertion would have more merit. You seem to be ignoring the many other possibilities for Quid's demise. The article itself places more weight on the internet as the culprit in that affair than Wikipedia. Wikipedia is most certainly flawed. Wikipedia is quite possibly flawed. But Wikipedia is not having only "a negative impact on our culture". Life is more complicated and nuanced than that allows for. I apologise for my misreading and the attack based upon it, but I still reject your posting as being sensationalised tabloid journalism. Do you not agree that one study, by a sole researcher, who "used a modification of a stratified random sampling and a purposive sampling to identify a variety of historical entries and compared each text in terms of depth, accuracy, and detail" and concluded that "More research must be undertaken" and that "generalizations cannot be made from this paper alone" is pretty definitive in stating it cannot in any way be read as definitive? I think my claims are pretty well founded, at least as well as your own.

Two questions:

1. Is Wikipedia having an impact on the sales of traditional reliable encyclopedias?
2. Is Wikipedia less accurate and less reliable than mainstream encyclopedias?

I believe the answer to those two questions is Yes. In which case, Wikipedia is having a negative impact on our culture.

We don't need Peer Reviewed criticism or Roland Barthes to make this point. It is crystal clear.

I don't see the problem on relying on summaries, it's the purpouse of abstracts to give the overal conclusion of the studies. Just restating what an author says in a summary is not manipulating anything, unless you want to reevaluate the study itself.

As for the success, there are success in Wikipedia, but those success are not because of Wikipedia itself but the fact that the infection hasn't still spread in those particular wikiregions.

No it isn't. For starters point one isn't crystal clear, and even if it is, doesn't mean it's impact is negative. One could spin the point the other way and posit it is a positive impact, because it is further entrenching the concept of free information. Your second point is also uncertain since studies currently conflict. Using one study against a competing and conflicting one, and using one newspaper report which may not necessarily make the case you state it does means your point is not crystal clear. You have started from the position of having your mind made up. I have started from the position of having an open mind. You have not convinced me, because you haven't substantiated your opinion on anything other than your opinion. I'm sorry, I just expect more from critics.

Since Whatever started off with "Look, I have no real axe to grind" and followed up with a long-winded, emotional rant without paragraph breaks, it is clear that they are, like many Wikipedia editors, reacting emotionally to evidence of Wikipedia failure.

There can be cognitive dissonance over this issue, for experienced editors.

Imagine, for instance, if you contributed to Wikipedia for years: Now imagine what it'd be like to suddenly realize that what you did didn't really have that much value, because the website you contributed to actually doesn't help human knowledge or even hinders it.

There's a contradiction there, between your long-standing beliefs and the evidence. That creates mental tension, which manifests as anger. It is alleviated cognitively temporarily through effective rationalization and continued denial ("Wikipedia Review are just assholes!!"). Ultimately, however, the only emotional relief is to recognize the truth.

My suggestion: Let Whatever stew in their juices. If you engage them, you'll only give them fuel for their further delusions. By leaving 'em alone and denying giving their soapbox an audience, it gives them the opportunity to thoughtfully self-reflect on Wikipedia failure.

I think I've already addressed this, but here goes. If you want to summarise and generalise, you're better off using a better abstract than one which states "More research must be undertaken" and "generalizations cannot be made from this paper alone". And if I'm honest, I'll trust the opinion of the person who read the book rather than the person who read the cover.

You aren't addressing it, "More research must be undertaken" is a golden standard, cautioning the reader when there are not many research being conducted on the subject covered. It's the language they use because they have no enough historiographical material published to compare and claim that the research supports such, such, such established conclusions according to such, such and such prior study. So your criticism does not make sense considering that they could not have written otherwise by confirming absolutly the unreliability of Wikipedia. A properly written paper peer reviewed will never include such direct conclusions where there are very few studies published about the issue.

As for trust, again, we're not reassessing the value of the study, we're only reporting it's conclusion, the study concludes Wikipedia less reliable, but it caution, since it's one of the only on the subject and that it would take more studies to confirm. It's science, studies must be falsifiable, what you expect that it say: ''Wikipedia is unreliable, get over it''? Even if true, you can't make such a statment of fact because you would present your conclusions as unfalsifiable. You're not doing science then.

Also, I haven't read the study itself, but from the tone of the summary and from the subject it deals with, it seems more primilary studies type (even if not intended) did they just decide randomnly the sample size or according to the expected precision. I'm intriged, because I have yet to know how they determinated some of the parametters. I've been interested in assessing the credibility of Wikipedia myself, but doing it like Britannica, restraining on science where generally articles could be of better quality given the type of editors they attract. I for one would not be interested to conduct a study by selecting a category of article where you know right there the result will be a failure (from experience, as low as 40%). Studies should be conducted by selecting the few spots of Wikipedia where we know there is something good being done. This way we know the maximum quality it could achieve. If in spite of that they find something like over 5% of differences, and the sample is enough big and replicated. Then the governments should have laws to restrict access to Wikipedia and considers it as some environmental polutant.

I'd just like to point out the existance of the "carriage return" button, otherwise known as the "enter" button. Makes reading large blocks of text easier.

Yeah, fucking retarded. The Foundation should be ashamed of themselves.

Choice quotations are manipulations. I can't find the term Kato placed in quotations, “inaccuracies in eight out of nine examined entries” in the summary of the study at all. It is also an interesting wording. The summary is clear that only nine articles were studied. Kato's wording can give the impression that more were studied and that 8/9ths were inaccurate. It's that sort of sensationalism I object to.



That's an interesting argument, but I'm not sure it applies generally across Wikipedia. It may be possible that there are areas where the Wikipedia model will work and areas where it will not. It appears that in areas where people have strong opinions the Wikipedia model as yet has not shown an ability to produce. It also appears that where there is no strong opinion the model can work. Which interestingly mirrors other aspects of society. Wikipedia is far more useful a tool for study than simply as an encyclopedia. I'm interested in how you suppose the successes in Wikipedia are not because of Wikipedia. What are they then put down to? The ghost in the machine?

I'm also minded of that old adage, that 90% of whatever is shit. Is it any wonder Wikipedia is the same? Wikipedia is produced by people. Should it not therefore evince the same flaws that anything else so produced exhibit? Some people might say that producing something only 80& accurate isn't too bad considering the method of production, when you consider the other sources are compiled by experts. And some might ponder the finding that those experts are only right 95-96% of the time.

I appreciate that's a standard Wikipedian apologist argument, but I'm not interested in making apologies, I'm only interested in pondering the possibilities, and working out exactly what it is the Wikipedia experiment tells us. I really do think Wikipedia will be of more use to sociologists than anybody else in the years to come. Although the psychologists might build a good case to dispute that.



I apologise if I too have overplayed my hand, but I believe I know how far the study can be used and within which reasonable caveats. My point is that I believe Kato has gone beyond the caveats of the study and also integrated tabloid sensationalism to build his case. I don't reject Kato's hypothesis out of hand, I just think it needs stronger evidence if it is to carry the weight Kato is asserting. At the minute it is rather like an acorn falling on one's head and assuming the tree is falling. I'd like to check the roots first, if that is okay. Has anyone actually read the study?

Kato has however stated his position for me far more clearly in this thread, for which I have not offered thanks, and I think we have chosen to agree to disagree. Or perhaps that there are areas where we agree and areas where we disagree. I don't reject the fact that the study can support the claim that Wikipedia is less reliable than other reference resources. I am intrigued that this is news, and also worried if there is evidence that the wider population are not aware of this fact.

This is perhaps where me and Kato agree, but where we have different ideas of the meaning. For example, if an encyclopedia written by the people, reflects the people and is supported by the people, should we be surprised? Kato's ultimate assertion is that Wikipedia is a negative impact upon our culture. My assertion is that it is possible it is a reflection.



But you're not, and that's my major point of contention. You're quoting a summary of the conclusion. It's like quoting the summary of a book to make a point. If you really want to criticise Wikipedia and have that criticism taken seriously, you have to do more than sensationalise recent news reports and studies.

If this was somebody else doing this, and you asked them if they had read the study in question, and they replied no, how seriously would you take them?



This I find an intriguing idea. Would you apply the same solution to bad journalism, which I hope we can all agree exists?

I favor an uptick in accuracy, excellence, and ethics in online media.

I agree, but I don't see that happening on Wikipedia because there appears to be either no ability or no will to build acceptance of those ideals into the system.

It can't (and won't) happen on Wikipedia because the system is not designed to value or pursue those objectives.

The obvious meaning of "nine examined entries" is that nine entries were examined. As a piece of elementary statistics, "inaccuracies in eight out of nine examined entries" implies that our best estimate is that around 89% of entries will have errors (provided the nine were chosen at random), though of course that figure has a high degree of uncertainty.

One needs a sample size greater than about 30 to produce a mean with a decent confidence interval.

I will reply to only this, as your replies aren't really about my answers.

The answer to this will be yes.



Not necessarly in this case, each articles are not just collectible data they contain materials. It depend how you calculate the confidence interval, taking each article as data information, or going deeper by taking elements of the article as the collectible data.

It's interesting to study the accuracy of Wikipedia because there is no standardized method and those who conduct them have free choice on comming up with a good methodology.

Let's be fair, though - Wikipedia is a pioneering process that's only been in existence for 6 or 7 years, and really only significantly popular for the last 2 or 3. To expect perfection out of any new process off the bat is unrealistic and improper, regardless of how you feel about Wikipedia's chances for success or "responsibility."

Some process need to be replaced because they can not be improved. In Wikipedia's case that it will become worst is predictible.

Sure, Wikipedia is unsustainable long-term in its current form due to its coddling of disruptive characters and POV pushers over content-creators and knowledgable people. With that said, 80% is not a bad number to be at for a small random sampling for a new service like Wikipedia - I'd be more interested to see a significant sample of the FAs and GAs, as that's where most articles should be attempting to end up. I'd assume that Wikipedia's best content is closer to the 95% threshold, perhaps even better, as FA/GA writers are the types who create content and believe in the project, wanting it to succeed.

Part of the problem with making comparisons is that the subject has a great influence on the quality of the information in the article. From what I can tell, articles that can be constructed around narratives are generally not too bad, no matter how far they are filled/padded out, if the subject is uncontroversial and there isn't anything woo-woo trying to attach itself. (As an example of the latter, right now I'm trying to get rid of an RC apocrypha about George Washington converting to Catholicism on his deathbed.) Once one steps over any of those lines, the quality is likely to go down, with bonus points for crossing two or more. Articles on Marriage and Wedding are dreadful, for instance, because of the push from the LGBT project (whose existence is a big bias problem anyway) and because, lacking a narrative, neither of them has an obvious organization.

At the moment I think most Wikipedia effort is going into sports/music/film/geographical trivia or into stuff that is only one step above that. Those articles are non-comparable to traditional encyclopedias because real encyclopedias don't bother with them. They are comparable to sites like IMDB or lighthousefriends or any number of specialist catalogs. On the Big Subjects I suspect that only the FAs or articles that have a lot of competent amateurs standing over them are really of acceptable quality. But at any rate, I suspect that it would be more illuminating to look at several different types of articles (or several different fields) and start from the real encyclopedia side in the selecting.

80% is very bad.

The problem in this case is that brute accuracy is not all, so any such test will bias the results in the profit of Wikipedia. Wikipedia is more prone to have biased editors, and they may decide to add materials and exclude others without the material being presented being wrong in the first place.

The same could be the case with true encyclopedia's, but they have true peer reviewing processes. I do believe that collaborative works could beat true encyclopedia's, and the only solution I found (I just came up with this yesterday) is for true encyclopedia's to form a community, where editors can come and provide relevent materials and suggestions but leaving the true encyclopedia'S staff do the editing. Online encyclopedia's like Britannica should be free to be accessed, we'll have to find a way to finance them, consider knowledge as cultural good. Wikipedia should definitly disapear. What good has been done to the project could be used by true encyclopedia's.

Indeed. That's really bad. So one in five facts were false? Imagine that in biography terms, just for starters.

If WP was some low-key academic experiment in collaboration, viewed largely by analysts keen to test the experiment, that would be fine. But as we know, WP is becoming the number one source of information for people on the planet. And it is putting reliable sources out of business to boot. (Don't keep denying that point, it is blatantly apparent that WP is impacting on the sales of real encyclopedias)

I'm just wondering how many lazy journalists will be crying when Wikipedia disappear.

80% isn't as bad as one thinks, especially considering what's considered a bad fact - a mistake in a date? An error in a listed habitat?

90% is 1 fact in 10, not too much better, truly.

90% is bad, I don't think why that'll excuse 80%. Anything under 95% is bad.

Yes, that's a very insidious form of bias. Even good information cited from encyclopaedias and serious reference works is often deleted because it's somehow not relevant. (How can anything from a serious reference work not be relevant to the sum of all human knowledge?)

Even more insidious, whole articles get deleted because they don't fit peoples prejudices (or are written and kept because they do fit).

I am not blatantly denying that point. You think it, I am not convinced. I think the internet is definitely having an impact on the sales of publications full stop. I am not sure that Wikipedia is solely to blame for the demise of even the one real life encyclopedia whose demise is not yet confirmed, let alone any others which have yet to be brought to my attention. I think Wikipedia is certainly impacting upon other encyclopedia and that the Wikipedia model is itself flawed, but I am not prepared to state with 100% certainty that Wikipedia is ultimately a bad thing and has only a negative impact. That is the point upon which we differ.


The worrying part is that they are starting to entrench bias into their deletion and inclusion process through the slow creep of notability. Wikipedia appears to slowly be defining itself as Brittanica without the experts.

The trouble is, without reading the study we have no idea if we are talking about one in five facts. For the record, the study stated that it "did reveal inaccuracies in eight of the nine entries and exposed major flaws in at least two of the nine Wikipedia articles. Overall, Wikipedia's accuracy rate was 80 percent compared with 95-96 percent accuracy within the other sources. This study does support the claim that Wikipedia is less reliable than other reference resources. Furthermore, the research found at least five unattributed direct quotations and verbatim text from other sources with no citations."

So one entry is okay. Eight are flawed with 2 of those flawed majorly. But there's no idea what the flaws are, how they are weighted and even what constitutes a flaw? Does a typo count? Enquiring minds should wonder. Still, at least we know that Wikipedia is less reliable than other reference resources, although we are left to define what those other reference resources are. Does anyone fancy chipping in to buy access to the study? A dollar each, say?

I was thinking about just forking over the money and seeing what's in the entire study, I'll probably go ahead and do that today.

See this Voice of the Prokonsul blog post

http://prokonsul.blogspot.com/2008/02/for-...bells-toll.html

That's a lot more useful. What also disturbs me is the amount of repackaging that is being done to Wikipedia content, such that it is cropping up everywhere. I'm never clear on how Alexa works though. Are there any comparisons of Wikipedia versus any of its more relevant competitors?

I'm making an assumption that it is the internet that is potentially dealing the blow to the established encyclopedias rather than Wikipedia itself on its own. The google algorithms and the established print encyclopedias reactions and actions regarding the internet have played as much a part as anything Wikipedia has done. I've read reports that the newspapers with the largest web traffic are those that made the leap to the web first and provided their content for free.

It makes you wonder whether, had one of the established print encyclopedias taken a different strategy towards the internet, Wikipedia may not have such a dominant position. Because what we, culturally need, and possibly what Wikipedia needs as well, to keep it more honest, is better competition. The google project probably has the most potential in that regards. I think Wikipedia's success has been a third of its problems. Another third I would attribute to the contributors. The final third is at the founders and the systems they implemented.

I'm not sure it is possible to salvage much from where it is now, but it needs better hands on the tiller than exist now. The mozilla foundation offers an interesting comparison of how it could have been taken forward. Had they pushed towards a partnership with google rather than answers all that time ago... and squared the circle of bad management from administrators upwards and worked out a way to limit contributions from an individual to one account who knows. But those are things for someone else to work out. I tend to agree with the assessments in other threads that wikis don't work in areas of contention. I think they don't work where people are more interested in protecting their position than the communal one. Like I say, it really is a fascinating social experiment. It is a shame that you can't track editors geographically, it would be interesting to see how the altruism/individualism synergy tracks across nations.

There was a popular book on economics written a few years ago by a World Bank (I think) economist which included a chapter on why, in some places, poor quality restaurants drive out good ones. I wish I could remember the title of the book or the author's name, because it was a fun read, but the argument was basically:

Where the main determinant of what choice is made in consumption is convenience, and where the feedback loop for poor service is minimal, firms that cheaply offer poor goods will drive out firms that try to offer quality goods.

The main case studied was restaurants in tourist areas, where most patrons can be expected to visit only a small number of times without a very long interval in between, and where they are likely to start with the cheapest place. Hence, boardwalk pizza sucks all around the world. Every once in a while, someone might make good boardwalk pizza, but most people will go next door to save a quarter, not realizing that the boardwalk pizza next door sucks. And then people like me just figure all boardwalk pizza sucks, and so don't patronize either.

And so Wikipedia, as long as it isn't obviously wrong even to someone who has referred to it because he doesn't know very much about a subject (in which case, "so fix it!" right? ), will draw traffic away from better sites that actually put hard project resources into making sure their information isn't crap, and therefore have fees or advertising or slow servers, etc.

That's really sad, any more laim that Wikipedia does not do damage to true encyclopedia's?

Gresham's Law applies in many contexts.

Gresham's Law of UseNet was that "Bad Postings Drive Out Good Ones."

On Wikipedia, bad editors drive out good ones.

I think it would have been less misleading to say "eight of the nine examined entries".

More accuratly, Wikipedia attract bad editors who drive out good ones.

You're wording does not involve sufficiently Wikipedia as the culprit.

Literally.

New York Times : Start Writing the Eulogies for Print Encyclopedias

http://www.nytimes.com/2008/03/16/weekinre...gin&oref=slogin

Interesting article, and fairly well balanced I thought. It evaluates Wikipedia's influence both positively and negatively, noting that "Encyclopedia publishers, while taking swipes at Wikipedia’s unreliability since it can be edited by anyone, have clearly adopted some of its lessons. They are incorporating more photographs and suggestions from readers to improve online content, and they are committed to updating material as facts change."

If only there was similar swiping by Wikipedia of better practises employed elsewhere. Sadly the Wikipedia model doesn't allow for innovation of practises in such a way. There's no way of implementing or even discussing implementation practically. Wikipedia is fast demonstrating that the crap also floats.