This page contains Frequently Asked Questions about ClueBot NG. Please make sure your question is not listed here before asking.
ClueBot NG works by generating a probability that a given edit is vandalism. Edits with a probability above a certain (threshold) score are considered vandalism. It's often difficult to tell exactly what the origins of a score are. ClueBot NG examines statistics from the edit, and calculates relationships between the statistics and the output. These relationships are not always apparent to a human. A human usually looks primarily at the meaning of content, while ClueBot NG looks at a large set of numbers. It's usually very difficult or impossible for a human, just looking at that set of numbers, to figure out why ClueBot NG did what it did.
In many cases, this leads to correct classifications that could easily be missed by a human, or extremely rapid reverts where it could take a human some time to do research. On the flip side, it also means that it can be very difficult to determine why a false positive occurred. For more information, see False Positives.
ClueBot NG does not use weights. It uses an algorithm called an artificial neural network that not only determines optimal weights, but can also discover more complex patterns and relationships. There are no static weights that can be modified.
There is no bad words list. The bot uses Bayesian classification to automatically generate a list of words with precise statistical probabilities.
The bot handles quoted text differently from regular text. Simply quoting a range of text is not enough to "fool" the bot if the edit is vandalism, but for constructive edits, some sometimes-acceptable bad words inside of quotes may be ignored.
ClueBot NG is different from other anti-vandal bots in that it is not preprogrammed to recognize vandalism. Instead, it learns what is considered vandalism by reviewing a very large number of both vandalism and constructive edits. This very large set of edits is called the dataset.
We use some edits that are automatically generated, but this is far from ideal, for two reasons. First, generated edits are always biased to some degree, depending on whatever statistics are used to generate them. Second, generated edits are often inaccurate, and small inaccuracies in the dataset can cause large decreases in accuracy.
We need people to manually review edits for the dataset. If you'd like to help out, please see here.
Anti-vandal bot edits usually aren't tagged with the bot flag. This is intentional, and is not specific to ClueBot NG – it applies to all anti-vandal bots.
There are two reasons for this:
ClueBot NG works by generating a probability that a given edit is vandalism. Edits with a probability above a certain (threshold) score are considered vandalism. The higher the threshold is set, the fewer false positives there are, but also the fewer vandalism edits are caught. To catch a large amount of vandalism, the threshold must be set at a level where there are very few, but some, false positives. More information is available in the algorithms and false positives sections.
See Why did ClueBot NG classify this edit as vandalism or constructive?
ClueBot NG's false positive rate is set by the operators at runtime. The current setting can be found in the statistics section. It has jumped around since the start, at times being 0.5% (1 in 200), 0.25% (1 in 400), and 0.1% (1 in 1000).
Before complaining about the false positive rate, please read the section on false positives to make sure you fully understand their role in the bot's operation, and the implications of adjusting the rate. Also keep in mind that the set false positive percentage is a maximum, and actual rate will probably be less due to post-processing filters.
If you feel you have a compelling reason that the false positive rate is too high, and your reason has not already been addressed on the talk page (please look through the archives – this topic has already been discussed at length several times), post your concerns and suggested false positive rate on the talk page. If your concerns have already been addressed by past discussions, you suggest an unreasonable false positive rate, or you do not suggest a false positive rate at all, your post will likely not receive a thorough response.
The false positive rate is the percentage of not-vandalism edits that are incorrectly reverted as vandalism. It is not the percentage of bot reverts that are incorrect.
The false positive (FP) rate calculations do not involve the number of reported false positives. A series of edits, known as the trial dataset, are used. These edits are known to be correct (verified by humans) and random (this is important). They are run through the bot offline, and these results form the basis of the FP rate calculation. This ensures that the FP rate is an accurate maximum.
Go here.
We submit them to the review interface for verification, then add them to the dataset. This should improve bot operation as long as it does not introduce significant dataset bias.
In our spare time, we manually review false positives. If we come across a false positive that may be able to be prevented by some method other than dataset expansion, we look into changing the code. If we see a false positive that occurred due to some reason that has not already been addressed in previous discussions, we usually leave a comment to that effect.
We used to do this, and the result was chaotic. Without explicit directions, users would leave malformed responses, responses without the required data (such as the link to the edit in question), and unrelated comments. Even with templates and comments to guide users, the result was a mess that was very difficult and time-consuming to review by hand, and impossible to automatically scan to import into the dataset.
If you believe the instructions for using the false positive reporting interface are unclear, feel free to modify and improve them. If you dislike the entire concept of the false positive reporting interface in general, feel free to suggest an alternative method on the talk page. But first, please review the archives to make sure your idea has not already been discussed, and also keep in mind that we do not have the time to both manually sort through chaotic freeform reporting and improve the bot. If you do post on the talk page, be sure to actually suggest an alternative method – complaints without suggestions do not help us improve anything.
Due to ClueBot NG's algorithm using a dataset instead of set rules, there will always be room for improvement. As the bot runs, people contribute to the dataset, and it learns more and more. But, even as it learns and improves, the number of false positives won't necessarily decrease – see the above section on why ClueBot NG has false positives. As long as the false positive rate is below an acceptable level, the bot can legitimately operate as an asset to Wikipedia, and improvements to the bot will result in increased vandalism catch rate. Additionally, even humans make mistakes – bots may even make proportionally fewer mistakes than humans. Bot mistakes are often noticed more because there's a greater raw volume of them (because the bot makes many more edits in total).
We always appreciate hearing that we do a good job. Please add praise to the praise page. Barnstars and other awards go on the awards page.
Complaints are filed according to the following criteria:
We love helpful suggestions, and they can be left on the talk page. Before posting, please make sure your suggestion is realistic, practical, and makes sense. Also, please glance through the archives to make sure the suggestion has not already been submitted and/or implemented.
We try to respond to and potentially implement helpful suggestions as soon as possible, particularly if they have a real chance of improving the bot's operation.
There is very little relation, besides the name. The core and algorithms used are written by different people, and use entirely different concepts. The only code shared between ClueBot and ClueBot NG is the interface to Wikipedia, and even that was refactored for ClueBot NG.
Cobi wrote the original ClueBot in its entirety, and maintained it for three years. Crispy wrote the core (vandalism detection algorithm) of ClueBot NG. Crispy and Cobi were and are part of the same organization called ClueNet, which is the origin of both bot names. In light of the new core, and greatly increased edit rate, Cobi largely refactored the ClueBot interface code and wrote the currently-used dataset collection scripts, as well as the review interface, for ClueBot NG.
It stands for "Next Generation".
Setting up ClueBot NG can be a complex process. You'll need to talk to the development team for instructions and help with setting it up. To talk with the dev team, join the IRC Channel.
Before trying to set it up, you need to consider a few things. Most importantly, ClueBot NG learns what is vandalism based on a dataset, which often needs to be very large to be effective. To be effective on Wikipedia, it needs at least 10,000 constructive edits and 10,000 vandalism edits. This is impractical for most smaller wikis to generate. It may be possible to use the dataset generated from Wikipedia on your wiki, if the content and vandalism trends are sufficiently similar. To test this for your wiki, you will need to at least come up with a trial dataset - say, about a hundred constructive edits and fifty vandalism edits, at a minimum. More is better. With a trial dataset, you can evaluate whether or not the Wikipedia training set can be used on your wiki. If your wiki is very large and has a sufficient number of edits to generate a full training set, then this is preferable, as it can then be personalized to your wiki's content and trends. However, generating the necessary datasets is up to you. It's important that they be random, and a representative sampling (i.e., unbiased). In addition to this, the trial dataset should be random, to accurately calculate a threshold and gauge effectiveness. You will also need a Linux/UNIX system to run the bot on, and sufficient knowledge to compile and install the various dependencies.