Loren Cobb posts a sophisticated vandalism study. His results confirm what I learned from personal experience in the trenches, reverting both short-term and long-term vandalism.

http://en.wikipedia.org/wiki/Wikipedia:Wik...06-22/Vandalism

Spread the word. Tell the media to revise their silly "most vandalism gets reverted in 5 minutes" nonsense from years ago. (I recall Somey made this point a while back.)

Is it subtle vandalism or creative editing?

Given that many admins on Wikipedia indulge in creative editing does that make them vandals?

The Wiki model pretty much encourages creative editing, it's probably this that initial attracts people to Wikipedia.

Wikipedia needs creative editing, all of its articles should be creatively edited and dispersed to the poor african children.

This study is meaningless as statistics without a precise definition of 'vandalism'. The fact that there is a category of 'subtle vandalism' is very telling. Presumably not so subtle that the study did not detect it. Are there even more subtle forms that it did not detect?

There are whole articles that (in my view) count as 'subtle vandalism'. When will they be reverted?

Also 'vandalism' is not the same as 'errors' or 'libel' or whatever. Do most 'errors' get corrected within minutes? Surely not.

Then there are the articles created just for POV purposes. They can be around for a very long time. And of course the articles deleted on purely POV grounds which may never be recreated causing permanent damage. As if we care!

The study methodology is painfully, grievously flawed. By taking 100 articles, then tracing backward in step-wise fashion through edits, until a vandal's edit is discovered, it does something really abominable.

Suppose I entered a nice, juicy, devious vandalized edit on January 1, 2009 on the article about [[Yellowfin tuna]]. Suppose, then, Jon Awbrey comes along on June 24, 2009 and makes a blunt, obvious, profane edit to the article. Jon's edit is discovered by the "Recent Change Patrol", and it is reverted in one minute. However, my edit from January remains safely in place.

The data point for [[Yellowfin tuna]] in Dr. Cobb's nifty chart is going to register along the "one minute" mark along the horizontal axis, rather than the "246,000-minute" mark, or the "never" mark.

So, Dr. Cobb thinks he has "proven" from this study that the median time for vandal correction on Wikipedia has dropped from 6 minutes, to 5 minutes, to 4 minutes, over each of the past three years. I would contend that all he has actually proven is that Wikipedia is being vandalized more often by "easy-to-spot" vandals.

I am going to contact Dr. Cobb and ask him about a particular article I have in mind, and I'm going to bet that his methodology will have missed one of my great not-so-recent vandalisms. I don't want to disclose it here, as it would soil the sample.

Greg

The study makes a huge swath of unjustified assumptions about Wikipedia vandalism. The biggest one seems to be the assumption that vandalism is randomly distributed amongst articles.

The problem is that our illustrious idiot used an inappropriate methodology to select edits for review. The only statistically meaningful way to do a vandalism study is to select all edits, or a random subsampling of edits, over a particular timeframe, and review each one independently to determine what sort of edit it is. Once you've identified each of the subject edits, then and only then can you step forward in the history of the relevant articles.

Article-based sampling will almost always bias the results to favor apparent stability, because most Wikipedia articles are barely ever edited (and thus barely ever vandalized).

The "double whammy" is one of the most common ways that persistent vandalism gets inserted into Wikipedia. I've even seen vandalism patrollers fight IPs to keep double-whammy vandalisms in because they can't be bothered to actually read whatever it is they're reverting. I suspect that some vandals have taken to using multiple accounts (or IPs) just so they can take advantage of this particular foible of the Twinkle-heads.


Indeed; the typology of individual edits is more complex than "vandalism" or "not vandalism". I'd probably have categories such as "Garbage edit", "Apparent blanking", "Apparent test edit", "Technical maintenance", "Copyedit", "Minor addition of information", "Substantial addition of information", and "Substantial rewrite", and then additional axes to these valuing the edit as "positive", "negative" or "neutral" to the overall value of the content.

I'd say the major division is between 'rubbish typed in with the full knowledge it is rubbish' and 'rubbish typed in without realising it is rubbish'. The 'vandal fighters' rarely pick up the second type (often because they contributed it in the first place).

E.g. this article

http://en.wikipedia.org/wiki/Nondual

by my definition is mostly vandalism of the second type, but it has been there for years.



Precisely.

I agree with Kelly that the more accurate study would review X randomly selected edits from the past Y ago, and then characterize said edits as Vandalism/Not Vandalism (where "Vandalism" is defined), and then review time-to-live. "Random Article" puts far too much weight on vandalism that is likley to be longer lived and more subtle than random edit, contrary to Greg's statement, which ignored the fact that a long-lived vandalistic edit is more likley on a lesser-trafficed article which is oversampled by this methodology. I suspect the study done right would demonstrate that vandalism is actually, on (mean/median), reverted in less than 1 minute, but with long-tail for "subtle" vandalism on articles that few people watch/care about like yellowfin tuna.

Yellowfin tuna is prone to 13,000 views per month.

The actual article that I've asked privately for Dr. Cobb to review has well over 60,000 views per month. That's significantly more traffic than obtained by the article about Cherry, or Lemon, or Saab, or Aaron Burr, or Ben Bernanke, or Federal Deposit Insurance Corporation. The naughty edit was made in early May 2009. Since then, the article has been viewed approximately 100,000 times, and it has been edited more than 40 times, including by such talented editors as Nopetro and Erik9.

Would you say that the statistics I've just provided are "likley" to be characterized as a lesser-"trafficed" article?

I'd love to see what the average error rate for vandalism patrollers is: that is, the percentage of edits characterized as vandalism removal that are reverting edits that are not actually vandalism. I bet it's higher than the Twinkleheads would like to admit.

It would appear that Dr. Cobb is quite the optimist.

Perhaps someone who is not on a 2:1 edit restriction by the ArbCom could notify him on the study's discussion page, that we are discussing his study here.

It would appear that none of the Wikipediots have mentioned the "double whammy" effect to him yet. Nor have they bothered to ask "what constitutes vandalism?"

"Watch/care about" does not equal "read."

No... not equal. But certainly highly correlated. Or, are you so dense (or blinded by your dislike of me) not to understand correlation?

I disagree that they are so highly correlated. I think that lots of non-editors read about Tunafsh and very few editors, while lots of editors read about, say, Ancient Egyptian race controversy, but very few non-editors. I suspect that juvenile vandalism that falls through the RC sieve on a Tunafish article would not get caught by the people who have the fish on their watchlist, while people who have Ancient Egyptian race controversy on their watchlist are on that article like hawks, and so any vandalism that is not instantly reverted by RCers is quickly reverted by POV warriors.

Also, please try to assume that I'm reading what you're writing and I'll assume you're doing the same. I'm not holding a grudge - in fact, I was the one that added your requested link to this discussion on the relevent talk page - obviously you are. Stop it.

Well, at least you provided two data points to formulate your sweeping generalization. I'll credit you that.

However, were I to have the time, I assure you that on 100 randomly selected articles, the correlation coefficient between page views and either edits-per-time or editors-per-time would be very high.

For example, Yellowfin tuna has been edited 28 times this year, by what appear to me to be 21 distinct editors.

Being that Ancient Egyptian race controversy has only about 40% of the traffic of Yellowfin tuna, but easily several thousand edits this year, you have selected a very unusual outlier in the data to support your thesis, I am willing to bet ($20).

One of the metrics for that is looking at talk pages of anonymous users that have a single author and are blank. (This presumes that an IP was warned about something and the warning was later rescinded.) It's not a perfect method, but I check every few months or so; there are usually about 1,000 results.

As an additional note, while I'm not a statistics expert, I question the use of the Pareto distribution, as I suspect that, with good data (second-scale rather than minute-scale), the vandalism reversion function would be a Rayleigh distribution, but I have now reached the end of my statistics skills.

Well this appealed to my competitive spirit so I just put in (under a different account obviously) a completely bizarre edit on an 80k article. Let's see how long it lasts.

(I will of course change it back this time tomorrow if it is still there)

I've reviewed your vandalistic edit, Peter, and it is approximately 12 or 13 times less funny than my vandalistic edit from early May, which still persisted up until Eva had to ruin the fun about 20 minutes ago.

OK that is quite funny, but only 2-3 times as funny, moreover my article is so incredibly serious that I get a multiplier of 10, so mine is at least 5 times funnier. Also if Eva can't find it in 24 hours, I win completely.

So you're saying the vandalism you're reporting to the researcher was added by you?

I didn't report the vandalism to the researcher. I asked the researcher to inform me of how he would have classified the article that contains the vandalism that I added.

Note: the researcher is on vacation in Oregon now, but he said he'd take a look at the article next week. It's quite clear that he's going to report back that the IKEA article would have fallen into the "quick revert" category of data points, which will prove my point that he's missing the "worst" forms of vandalism because their tracks are covered by the gentle snows of recent and obvious forms of vandalism.

I am convinced that the researcher's study proves nothing, other than what I said before -- that Wikipedia is being more frequently assaulted by vandalism, and how quickly the "most recent" vandalistic edit is reverted really means diddly squat.

I don't follow this at all. The 'vandalistic' edit was in May, so by what logic does it fall in the 'quick revert' form of vandalism?

I agree with you that his study proves nothing. Since the study has to be able to identify vandalism in order to have a 'statistical universe', it is only valid if it spots all vandalism. If it only spots the kind of vandalism that is easily spotted and therefore quickly reverted, it is entirely invalid. To spell it out, if the statistical universe is defined as 'vandalism that is quickly spotted and reverted', clearly it will be valid, but self-evident. If by contrast it is defined as 'all vandalism', the study genuinely has to pick out all vandalism.

My ongoing study clearly shows that bad but non-obvious vandalism is unlikely to be spotted at all, and so this guy's research is entirely worthless.

The edit doesn't, the article does. In this case the study would have picked up a June 1 blanking that was reverted instantly by the Cluebot bot, and not recognized any prior vandalism even if it yet remained in the article.

Exactly. If Dr. Cobb's punching of "Random article" returned the IKEA article, he would have traced back only as far as the first available vandalism, however Dr. Cobb happens to personally and subjectively define "vandalism". In this case, his data point sensor would have lit up on a recent, easy-to-spot vandalism. He wouldn't have even gotten to my gem of a vandalism, because he stopped on the most recent one he encountered.

I would further add, that if Dr. Cobb was spending only 3 or 4 seconds per edit to evaluate, he may have even still missed my vandalism, even if it were the first one that he'd encounter in his back-tracking steps. Because my vandalism doesn't look like vandalism, to the untrained eye of a diff-browsing mathematician.

Seriously, "outer pillow pocket with filling" was The Onion grade material. I think I missed my calling in satirical comedy.

Actually I thought this was very good, one of your very best,Maestro. My talents are rather weak in that direction, but I feel more at home in the 'fog of confusion' technique - see the storm (or rather the fog) building up around the article I mentioned by email.

Although a Wikipedian has finally got it:

http://en.wikipedia.org/w/index.php?title=...oldid=299010104

That was someone correcting himself on the Beethoven article.

No. Follow the article history for a few days, and look at Workpermits talk page.

I think one of the more interesting studies of vandalism was this one which focused exclusively on featured articles (Wikipedia's so-called "best work"). It found that vandalism other than the painfully obvious "replaced content with POOP" sort takes about half a day on average to revert. My personal favorite comes from the article on the Second Crusade. First, the content was changed to say that the Crusade was led by Bill Cosby, Gregory Peck and Harry Potter (diff). A day later, this information was still there, and another editor came around and vandalized the article more obviously (diff). Three hours later some sort of RC patroller came along and reverted back to the Gregory Peck, Harry Potter, Bill Cosby version (diff). Then another day later, someone finally realized and reverted the original vandalism (diff). All told, Wikipedia showed for 42 hours that Harry Potter was a leader in the Crusades.

No wonder my history essay got poor marks!

Thank you sir that example earns a place here

http://www.wikipediareview.com/Creative_Vandalism_on_Wikipedia

and let us not forget the study which created it here

http://en.wikipedia.org/wiki/User:Colonel_Chaos/study

by 'Colonel Chaos'.

Best wishes for Colonel Chaos being promoted to General Chaos someday soon.

Ja Ja

Well Professor Chaos already has a General Disarray.

Clearly that was the Children's Crucade.