The Wikimedia Foundation Anti-Harassment Tools team ran a survey to measure administrator confidence on English Wikipedia during September 2017. We had 117 participants fill out the survey. The survey is designed not to be a widely comprehensive survey but a specific and narrow one. The Anti-Harassment Tools Team is interested in measuring how admins feel about different kinds of conflict specific activities (wikihounding, vandalism, harassment, sockpuppetry), how confident they feel spotting, mitigating, and intervening in these case types, and if they feel supported with tools and other resources from the Wikimedia Foundation.

On Monday, October 2, 2017, the Anti-Harassment Tools team will share further results from the comments section of the survey and our preliminary analysis. The team wants to work with the community to identify significant findings and how they could influence our team's work on tool development.

Later in October, we will have a second discussion that will focus on the findings in the survey comments sections that are directed towards policy changes or different ways of reporting and managing cases.

• Survey information
  • The survey was open for any active English Wikipedia administrator from September 12 to 24, 2017.
  • This survey is a semi-annual survey for the Wikimedia Foundation's Anti-Harassment Tools Team to gauge how well materials, tools, training, and information exists for admins in recognizing and mitigating things like sockpuppetting, vandalism, and harassment. This will be integral in our team in helping determine what can be updated, what works, what doesn't, and how to better support the Wikipedia community.
  • Our privacy statement is listed here: https://wikimediafoundation.org/wiki/Semi-Annual_Admin_Survey_Privacy_Statement
  • These results were collated on September 25, 2017. Questions are presented here in the order they were asked on the survey.
  • The survey received 117 responses.
• Demographics
  • How old are you?
    • under 18 – 0.9%
    • 18–24 – 10.3%
    • 25–35 – 21.4%
    • 36–45 – 25.6%
    • 46–60 – 22.2%
    • above 60 – 14.5%
    • Prefer not to say – 5.1%
  • Where are you from?
    • North America – 56.4%
    • Europe – 27.4%
    • Asia – 1.7%
    • Middle East – 0.9%
    • Australia – 11.1%
    • Africa – 0.9%
    • South America – 0%
    • Prefer not to say – 1.7%
  • Sex and gender?
    • Female – 9.4%
    • Male – 86.3%
    • Non binary/Third Gender – 0%
    • Prefer to self describe – 1.7%
    • Prefer not to say – 2.6%
  • How long have you been editing Wikipedia?
    • under a year – 0%
    • 1–5 years – 8.5%
    • 6–10 years – 15.4%
    • 10+ years – 76.1%
  • Are you an administrator on Wikipedia?
    • Yes – 100%
    • No – 0%
• Recognizing misconduct
  • How confident are you in recognizing sockpuppetry?
    • 1 (Not confident) – 1.7%
    • 2 – 18.8%
    • 3 – 25.6%
    • 4 – 37.6%
    • 5 (Very confident) – 16.2%
  • How confident are you in recognizing vandalism?
    • 1 (Not confident) – 0%
    • 2 – 0%
    • 3 – 0.9%
    • 4 – 21.4%
    • 5 (Very confident) – 77.8%
  • How confident are you in recognizing wikihounding?
    • 1 (Not confident) – 3.4%
    • 2 – 15.4%
    • 3 – 29.1%
    • 4 – 39.3%
    • 5 (Very confident) – 12.8%
  • How confident are you in recognizing harassment?
    • 1 (Not confident) – 0.9%
    • 2 – 8.5%
    • 3 – 21.4%
    • 4 – 48.7%
    • 5 (Very confident) – 20.5%
• Skills and tools
  • Do you feel like you have the skills or tools to intervene or stop cases of sockpuppeting?
    • Rarely – 13.7%
    • Sometimes, though it depends – 32.5%
    • Usually – 35%
    • Almost always – 18.8%
  • Do you feel like you have the skills or tools to intervene or stop cases of vandalism?
    • Rarely – 0%
    • Sometimes, though it depends – 2.6%
    • Usually – 17.9%
    • Almost always – 79.5%
  • Do you feel like you have the skills or tools to intervene or stop cases of wikihounding?
    • Rarely – 14.8%
    • Sometimes, though it depends – 47.8%
    • Usually – 28.7%
    • Almost always – 8.7%
  • Do you feel like you have the skills or tools to intervene or stop cases of harassment?
    • Rarely – 28.2%
    • Sometimes, though it depends – 44.4%
    • Usually – 28.2%
    • Almost always – 11.1%
• Preparedness
  • Wikipedia has provided me enough resources to solve, mitigate, or intervene in cases of sockpuppetry.
    • 1 (Strongly disagree) – 6%
    • 2 – 17.2%
    • 3 – 30.2%
    • 4 – 31%
    • 5 (Strongly agree) – 15.5%
  • Wikipedia has provided me enough resources to solve, mitigate, or intervene in cases of vandalism.
    • 1 (Strongly disagree) – 0.9%
    • 2 – 0.9%
    • 3 – 5.2%
    • 4 – 24.1%
    • 5 (Strongly agree) – 69%
  • Wikipedia has provided me enough resources to solve, mitigate, or intervene in cases of wikihounding.
    • 1 (Strongly disagree) – 6%
    • 2 – 23.3%
    • 3 – 33.6%
    • 4 – 28.4%
    • 5 (Strongly agree) – 8.6%
  • Wikipedia has provided me enough resources to solve, mitigate, or intervene in cases of harassment.
    • 1 (Strongly disagree) – 10.3%
    • 2 – 23.1%
    • 3 – 30.8%
    • 4 – 26.5%
    • 5 (Strongly agree) – 9.4%

Numbers:

• Feedback comments by admins-
  • 45 by survey
  • 4 by email

Most common themes by admins about tech tools

• “Better tools to technically prevent sockpuppetry”
• “Checkuser tool is extremely outdated and kludgy with regard to the technical data and output it provides.”
• “Stronger bot-detection and reporting of these activities, for administrator confirmation, would be helpful.”
• Requests for improvement to blocking tools was mentioned, eg. rangeblocks, user page/topic blocks, smartblocks

Different types of tools discussed in admin confidence survey

• Checkuser
  • Reporting improvements
  • Type of data pulled
  • Better interface with blocking and reporting
• Vandalism patrol tools
  • Twinkle
  • Rollback
  • Recent changes
• Functional CAPTCHA
• Blacklists
• Suppression/deletion user history
• Notification of future edits of warned vandals
• Detection tools
  • ClueBot NG
  • ORES
• Improved reporting systems of bots used for abuse
• AbuseFilter
• Blocking tool
  • Rangeblocks
  • Smart blocking- the system would block accounts who used an IP address with the same OS and browser as the person who made the contentious edit instead of just same IP address
  • User page or topic blocks (instead of full wiki site blocks)
• Feed to flag formerly open proxies to be unblocked
• RangeContribs tool
• Interaction Timeline tool
• Improve Autoconfirmed criteria, aims to prevent abusive bot accounts
• Mute
  • Third party enforced mute

Summary of remarks in the comment section:

• Harassment policy as written interferes with enforcement
  • Too often the person perpetrating harassment is adhering to the letter of the law so admins are reluctant to sanction them because what they're is not over the bright line. Wikilawyering is a big problem.
  • It is common for harassers to make positive contributions to some degree, so deciding to sanction them is a value judgement. To address this dynamic there needs to be a policy that says that incivility, uncollegiality, hounding and harassment are not outweighed by positive contributions.
  • Harassment and wikihounding cases are often much more complex and ambiguous. Accusations of harassment or hounding can occur when someone is genuinely trying to solve an actual problem.
  • Clear cut harassment is comparatively easy to resolve (by blocking) but a lot of cases aren't as clear cut. The list of offenses at WP:HARASSMENT is pretty limited, and some harassers are careful not to stray over those lines. Hounding is related - in some cases it's hard to parse out legitimate hounding, and editors who are simply scrutinizing another editor's work.
• Social barriers to addressing harassment
  • The greatest issue facing Wikipedia (as has been for many years) is the refusal of the community to deal adequately with toxic personalities and harassment.
  • Hounding and harassment are difficult for individual administrators to deal with because of social issues and wikipolitics.
• Chilling effects of harassment
  • Bullying is rampant and increasing.
  • Some users are reluctant to applied for advanced permissions because of the potential for harassment.
  • Best to refrain from addressing content disputes or wikihounding/anti-harassment enforcement because with one mistake you can be outed; and it’s irreversible.
  • We don't have tools or enough awareness/support to prevent or stop off-wiki harassment.
  • Too often admins who step in to stop harassment end up being hounded by other editors.  Very experienced admins and editors are reluctant to take action towards prominent trolls because they could be harassed out of Wikipedia, and it isn't worth the problems.
  • If users see others are being abused they'll be discouraged and not take the risk of getting involved, or perhaps pick up the bad habits themselves.
• Victim blaming
  • If the target of harassment is not perfect it gets turned on them.
  • People who report being harassed have their own actions questioned whether or not their report is valid.
• More training and resources
  • Not enough done to prepare admins for handling harassment.
  • More information and access to resources that can help victims of wikihounding or harassment.
  • Specifically trained users to deal with harassment/hounding.
  • A review board of people trained to handle reports of harassment.
• Time intensive
  • To be involved with dispute resolution requires a commitment of time, energy, and emotional fortitude.
  • Because of the community desire for transparency, discussing the admin action with the community can take even more time than doing the action.
  • As unpaid volunteers admins incentivized not to involve themselves in dispute resolution.
  • Wikihounding is probably the hardest of these to deal with. Needs a lot of investigation and checking of diffs etc.
  • Volunteers generally are outgunned and outnumbered in their efforts to improve beyond the status quo in fighting abuse.
• Specialization
  • Some of us just don't get involved in some aspects of administration.
  • I have little direct experience with the other categories (besides vandalism.)
  • I prefer to leave that kind of job to somebody else.
• AN/I related
  • AN/I is useless. Cases are auto-archived without being closed.
  • Harassment  is usually debated forever at AN and nobody wins. Resolution attempts become another venue for the argument.
  • When dealing with less blatant harassment at ANI it is not uncommon that people will side with the harasser.
  • When cases go to noticeboards, if an admin takes action against established editors, the admins are more likely to come off worse in the long run.
  • ANI tends to be combative. An admins taking actions to address harassment will lose credibility as an administrator and possibly lose their advanced permissions.
  • Editors that belong to a group with shared interests, ANI is overwhelmed by the group’s interests. There is a tendency towards mob rule with different outcomes happening depending on who joins the discussion.
• Role of WMF
  • WMF needs to be more aggressive in pursuing obvious cases of abuse when there has been a violation of an ISP's ToS.
  • WMF can take a more assertive role when it comes to behaviour management.
  • WMF need to institute a 'bright line' rule which sets basic standards of civility and empowers admins.
  • WMF should give a clearer explanation of legal support available to administrators outside the US.
  • WMF should encourage more rapid resolution of sockpuppet cases needing checkuser support.
• Other
  • Sockpuppet system can be difficult to use if unaccustom to the process, and it can take quite a while for the issue to be resolved.
  • The community has failed to enforce the Terms of User  mandatory paid editing disclosure.  allowing Wikipedia to become over run with commercial content.
  • The worst part of Wikipedia - the most demotivating, time wasting, uninspiring, thankless and horrible - is dealing with humans who choose to abuse the platform, act in poor faith, or use an encyclopedic knowledge of an a myriad of subjective and debatable guidelines to push an agenda.
  • A lot of harassment/hounding issues are connected to some topical dispute (e.g a POV dispute, which are notionally not for the administrator noticeboards to handle) and they become hopelessly self-perpetuating.