| This is a DRAFT form of the RFC, not the RFC itself. Please discuss the proposals with a view to improving them, rather than supporting or opposing. |
A consensus was reached on 27 May 2011 to restrict the creation of new mainspace articles to contributors whose accounts have reached autoconfirmed status for the duration of a trial. It was agreed that some form of evidence-based evaluation would then occur, and the question of implementing the restriction on the longer term reexamined in light of the results. Some details of the implementation of this need to be worked out. Hence this RFC.
The most popular of the proposals will be implemented.
Within the framework of the recommendations made in the closing discussion summary:
- To operate the trial for a period of 6 (six) months during which the holders of registered accounts may not create pages in article mainspace until their account has reached autoconfirmed status
- After six months of trial, to stop the trial for a period of 30 (thirty) days for evaluation, before reinstalling the restriction as a long-term feature of en.Wikipedia.
Discussion[edit]
This seems, to me, to be the simplest way to go about this, and seems the way most likely to yield results. Waiting only 30 days will give us some vague sense, but we need to know what the longer-term impact will be. We also need to know what will happen once we shut it off, hence the 30 days afterwards. The Blade of the Northern Lights (話して下さい) 16:52, 2 June 2011 (UTC)[reply]
- Not acceptable as per closing admin summary "Most crucially, the trial should be for a strictly defined time period, with a firm understanding that the feature will be deactivated at the end of the trial and not reactivated (if ever) until the results are reviewed and discussed by the community.". Cenarium (talk) 22:43, 2 June 2011 (UTC)[reply]
- Frankly, it's immaterial whether or not it's still on when we discuss permanent implementation. I suggest it because it's easier to decide when we can actually see what effect it's having. The difference between this and PC is that this would confirm it'll still be running, which should resolve a lot of the issues the PC trial is plagued with. The Blade of the Northern Lights (話して下さい) 22:58, 2 June 2011 (UTC)[reply]
- It still isn't in accordance with the closing summary so cannot be considered here, since this RFC is made pursuant to the closing admin assessment. Cenarium (talk) 23:05, 2 June 2011 (UTC)[reply]
- Definitely not acceptable. You state "...stop the trial for a period of 30 (thirty) days for evaluation, before reinstalling the restriction as a long term feature of en.Wikipedia" - surely this assumes that the trial is going to be successful - you stop it for 30 days, then state that we'd reinstall it as a long term feature. Surely it would be - stop it for 30 days, evaluate it, discuss whether it should be continued, seek feedback from users affected by the change, THEN decide whether we're putting it back. FishBarking? 17:27, 12 June 2011 (UTC)[reply]
Trial for a period of 6 (six) months during which the holders of registered accounts may not create pages in article mainspace until their account has reached autoconfirmed status, with one exception: they retain the ability to create articles via the Article Wizard.
User:Rd232/creationdraft, which would replace the current MediaWiki:Nocreatetext, illustrates how options would be presented to the new user in this case.
Data should be gathered for 30 days prior to trial, and evaluation of the overall concept should begin after 4 months. If no consensus is reached by 6 months to continue the concept indefinitely, it is switched off. If consensus is reached, it is retained indefinitely. Evaluation of the Wizard immediate-creation option may begin separately at any time, if editors feel there are issues arising which require discussion. If there is a consensus to switch off that option, it may be done at any time.
Discussion[edit]
- Not acceptable per closing admin summary: "Most crucially, the trial should be for a strictly defined time period, with a firm understanding that the feature will be deactivated at the end of the trial and not reactivated (if ever) until the results are reviewed and discussed by the community.". Cenarium (talk) 22:44, 2 June 2011 (UTC)[reply]
- Comments can be made here. There is also some elaboration of the technical bits there as well.
The proposal above reflects the spirit of the RfC but is both too long and insufficient to gather the necessary data to evaluate the impact of restricting article creation. Instead I propose a shorter trial which requires some developer involvement and may require gathering and anonymizing information on not just contributions but potential contributions.
Technical details[edit]
- Format
The trial should proceed in three phases. An initial data gathering phase to last 30 days where no external changes are made to article creation, an implementation phase where article creation is restricted (see below for details) also to last 30 days and a followup phase where the article creation is returned to the status quo ante for another 30 days. Given the volume of articles created and deleted per week, 3-6 months is more than necessary to determine the immediate impact of restricting article creation. 30 days will provide more than enough data. The pre and post periods are there to gather data for a representative comparison should contemporaneous statistics be deemed too difficult to collect.
- Data
The trial should gather data not only on number of articles created but their disposition after 15 days, the history of the accounts which created the articles and the number of articles not created during the trial timeframe. Some of these data are available to editors and administrators but some will only be available to developers. Specifically:
- During the entire trial all pages where a non-autoconfirmed account attempts to create an article should be noted, as well as the account itself. Creating this notification may be burdensome for the developers or the servers or it may not, I do not know the exact implementation details.
- Each account which creates an article or attempts to create an article should be recorded and the number of follow-up edits (and status like blocks, etc) should be noted. This data may be sensitive, as it would require monitoring not only edits but attempted edits so I suggest that developers develop a system to log the information and present anonymized statistics to the community (simple summary statistics will do).
- If possible, the articles created during the pre and post period by non-autoconfirmed accounts should be recorded and the status of the article (deleted, tagged, nominated for deletion, etc.) for 15 days past the date of creation should be noted.
- Motivation
A primary concern among supporters of the RfC was disenchantment with wikipedia due to an aggressive (but reasonable) response by new page patrollers when faced with a grossly inappropriate article. Chief among the concerns of those critical of the RfC was the potential loss in contributors due to increased editing friction. A trial which simply changes the ability of non-autoconfirmed accounts to create articles without determining the impact on potential article creation will overestimate the effectiveness of the trial to reduce inappropriate articles and greatly underestimate the potential loss of contributors.
My version of the trial solves these problems in two ways. First, breaking up the trial into three parts where data is gathered in each allows us to see the switch-on and switch-off effects of the policy change. We can compare the overall article volume and quantity as well as editor retention in the trial to both the pre-trial condition and a post trial condition where some new editors may have learned of the change. Second, the inclusion of statistics on article attempts during the trial will catch most cases where a non-autoconfirmed account attempts to create an article and what their response is. If they still attempt to create the article via the AfC we will know the source of the change. Likewise if they go on to make regular edits (or request confirmation) before re-creating the article we can measure this as well. Of course should the account simply stop editing after a stymied attempt to create an article we will know this as well.
- Potential Improvement
Though I suspect that the developers will not want to engage in a trial which presents a non-uniform face to users, it would be most optimal to engage in a trial where editors are randomly selected to see one of three options when opening the "create a new page" link:
- A page describing the policy change and linking to AfC
- An immediate redirection to AfC with the article title preloaded
- A page simply noting the policy change and nothing more.
Using these three options we can track user response during the trial against each change. We would record the same data as I described above but would have the additional advantage of a randomized trial showing the effectiveness of each potential change. Since one of these three (or something like them) options will be made the default should the trial be deemed a success it behooves us to kill two birds with one stone and perform some A/B testing while we are conducting the trial.
Discussion[edit]
- Support as proposer. While the trial I propose above will be technically complicated and will require a great deal more attention from the WMF and developers I feel we have a responsibility to accurately gather user data before making a change of this scale. We cannot simply just turn the feature on, leave it there for a few months and then note that the sky has not fallen. This was a principal flaw with the PC trial and was as bad as the debacle over the trial end date. Without statistically meaningful and valid metrics any discussion post trial will devolve into a recapitulation of positions held before the trial. Thank you. Protonk (talk) 23:09, 29 May 2011 (UTC)[reply]
- Interesting. i) I think the "live" phase needs to be at least 3 months, though. A/B randomised testing is an interesting option; they've done that with the fundraising banners AFAIK so it may not be so unlikely a notion to get it here. ii) part of the reason I think the live phase needs to be longer is because the primary object is not articles (which are more or less being handled anyway), but editors, and it needs more time to evaluate how editors entering through the new process have different wikilives than those who didn't/don't. iii) I don't see the point of a switchoff per se. I would relegate switch-off to a failsafe down the line, if consensus can't be had by about 6 months, say, that it should be retained. Rd232 talk 23:39, 29 May 2011 (UTC)[reply]
- I included the switchoff phase because during the course of the trial we are asking the devs to gather and aggregate data they normally (I assume) would not. At the end of the 30 day post phase we can look at the response from editors for all three phases. So an article created on the first day of the "pre" phase will be tracked for 90 days (the editor as well), and then anything after that can be tracked for 90-t days. After the end of the post phase the devs can remove the tracking and data collection regardless of the outcome. Protonk (talk) 00:28, 30 May 2011 (UTC)[reply]
- Well I doubt there's much benefit to the devs to switching that sort of data collection off; it just seems like an extra task. The big deal is setting it up in the first place, I would think. Unless the devs express a preference, I would keep both the data tracking and trial in sync, and leave the trial running up to 6 months, and then switch off both if it hasn't been previously agreed to implement the idea indefinitely. If an agreement is reached to that effect, data collection can stop sooner. Rd232 talk 00:36, 30 May 2011 (UTC)[reply]
- The benefit is twofold. First, I'm sure there is some performance cost to gathering this data, however small. Second, the data I am suggesting we gather is nominally private. Specifically when we record that a non-autoconfirmed account visited a red-link then navigated away we are capturing data which I doubt we want to keep around in perpetuity. But because this information is vital for understanding what would otherwise be only the absence of an article I recommend we gather it during the trial. And I recommend this method of testing in order to avoid a 6 month trial. I have other reasons to want to avoid a 6 month trial (which I will articulate elsewhere), but 6 months is a lifetime in terms of a website. If we turn this feature on and it has a deleterious effect on new user growth and we persist for 6 months when we are done it will be too late to reverse the trend. 90 days with a 30 day switch on is more than sufficient. Protonk (talk) 00:46, 30 May 2011 (UTC)[reply]
- I agree that we don't need a long trial to collect good data. But I think that re-opening the flood gates will not provide useful data. There's almost certainly going to be a surge in new article creation once we go back to a free for all. Also, the trial can be short. But the long-term impact is still important. Are editors that sign up sticking around longer because they experience a better learning curve? That's the key question we're trying to answer. Shooterwalker (talk) 02:17, 30 May 2011 (UTC)[reply]
- "we don't need a long trial to collect good data." seems to be contradicted by "Are editors that sign up sticking around longer". I'd argue the most important stickiness is not on the timeframe of days, it's at least weeks (if not months or years...). That points to a longer trial. Rd232 talk 02:34, 30 May 2011 (UTC)[reply]
- I disagree with that statement. I'll post some data when I put a section in the talk page explaining my decision, but a rough estimate of the half life for a wikipedia account is on the order of days. And specifically we are talking about marginal (as in on the edge, no pejorative connotation) editors for whom the difference in making 5 edits and stopping and 20 edits and continuing is critical. If at the end of a 90 day trial an account is still editing whether or not they stay is pretty much out of the scope of their first article. Protonk (talk) 03:17, 30 May 2011 (UTC)[reply]
- The trial itself can be short. Gathering the data after the trial is over will definitely require months. But we can revert to the status quo while we wait for the data to come in. Shooterwalker (talk) 04:31, 30 May 2011 (UTC)[reply]
- "Gathering the data after the trial is over will definitely require months." ?? why? Data processing might take a little time, but not so much as to require a long wait after trial completion (I would think it could easily be done in a 2 month time period, from month 4 to 6 of a 6-month trial as per my proposal). Unless you know something I don't? Rd232 talk 04:39, 30 May 2011 (UTC)[reply]
- Some comments here. Protonk (talk) 05:03, 30 May 2011 (UTC)[reply]
- The most relevant question, in my opinion, is "who is still editing after 90 days?" If we have a 6-month trial, the soonest we could answer that question would be at the end of 9-months. I'd prefer something shorter than 6-months for that reason. Shooterwalker (talk) 04:44, 30 May 2011 (UTC)[reply]
- That's not true. New accounts and new articles are created all the time. The soonest we could say "who started editing at the tail end of the trial and is still editing 90 days later" is 9 months, but that isn't a very interesting question. Further you would have an uphill battle convincing me that editor retention past 90 is a matter that this proposal should at all be concerned with. The issue of auto-confirmed editors creating articles is one of a 0-30 day timeframe. If someone is still editing 30 days after their account creation then whatever gets them to stay or leave probably isn't within the scope of the autoconfirmed user group. Protonk (talk) 05:07, 30 May 2011 (UTC)[reply]
- I highly doubt that new editors are watching this trial page and just waiting for some post trial shutdown in order to create a flood of new articles. Much more likely is the vast majority of editors don't know this page exists and will never know. Their interaction with the trial will be when they go to create an article and either can or can't. As for long term impact, I feel that is totally beyond the interest of this trial or any prospective data we may collect on it. IF we want long term data on wikipedians then we can use data dumps. For our purposes a window of 60-30 days (from the start of the actual switch on to the actual switch off, measured in days until we stop collecting data) is fine. Any editor in the sample still editing at the end of the 90s can be considered a long term editor. Protonk (talk) 03:17, 30 May 2011 (UTC)[reply]
- Comment: 15 days is too short to measuring AFD deletions. Too accomodate for relists and late nominations the measured period should be at least a month. I also have serious doubts about the technical feasibility of measuring "attempted article creation". I think the only thing you can measure is the number of times the no-create text is viewed, but that does not necessarily mean the editor wanted to create an article (he might just have clicked a red link or wanted to see the deletion log). Yoenit (talk) 10:40, 30 May 2011 (UTC)[reply]
- 15 days was just thrown out there. For a 90 day trial we would be able to follow any article created for 90 - t days. As for the limitation, if we can only measure how many times the no-create text is viewed, that's something. If we can specifically measure how many times a non-confirmed account viewed the text that is better. Protonk (talk) 14:47, 30 May 2011 (UTC)[reply]
- As per the closing statement "Most crucially, the trial should be for a strictly defined time period, with a firm understanding that the feature will be deactivated at the end of the trial and not reactivated (if ever) until the results are reviewed and discussed by the community.", the restriction should be lifted at the end of the trial and after the review period, a consensus should develop for reinstating it. And honestly, what's the point of a trial if we keep on regardless of the results ? I'm not entirely certain on how this proposal 'ends' the trial. Cenarium (talk) 22:51, 2 June 2011 (UTC)[reply]
- What statistics do we want to collect?
- How many new accounts were created during the trial, compared to a similar non-trial period?
- Of the new accounts created under trial conditions, how many editors were still active after 90 days compared to editors from a similar non-trial period?
- How often were articles created by the trial cohort, compared to a similar non-trial period?
- Of the articles created by the trial cohort, how often were articles retained compared to a similar non-trial period?
- Of the articles created by the trial cohort, how often were articles deleted at AFD compared to a similar non-trial period?
- Of the articles created by the trial cohort, how often were articles speedily deleted compared to a similar non-trial period?
- How did activity at "Articles for Creation" change during the trial, compared to a similar non-trial period?
- For each of these questions, comparisons should be made in absolute quantity, and in relative percentages or per-capita comparisons wherever possible.
- Which of those are reasonably feasible?
- [Restating 1. from a different perspective] Let's imagine that the trial's finished, and we're trying to evaluate it. What statistics are we going to wish we'd collected in order to help decide whether the costs outweigh the benefits? Rd232 talk 02:40, 30 May 2011 (UTC)[reply]
- How does this effect the growth in the number of WP articles?
- Does the reduced flow of articles through NPP result in a slower pace there, with patrollers doing more improving of articles and less templating.
- We also need to measure the unintended consequences - if we make article creation more difficult will that increase the number of incidents where people create artiles on talkpages, turn redirects and dabpages into articles or even change the subject of an article. ϢereSpielChequers 05:41, 2 June 2011 (UTC)[reply]
- There are two more options for "how often were articles deleted by process X", PROD and BLPPROD, I'd include those separately. I would hope that a good article creation process would drastically reduce the number of deletions by BLPPROD.
- How many articles were retained but still marked "unsourced BLP" at the end of the trial and control tranches?
- (Extra difficulty) Does this have an effect on the OTRS volume? (Tricky to measure in many ways, this might not be doable.) --joe deckertalk to me 16:34, 6 June 2011 (UTC)[reply]
- How many accounts managed to gain autoconfirmed status by performing relatively tiny edits just to get the required 10 - such as editing their userpage in small chunks just to get to the needed total? FishBarking? 17:33, 12 June 2011 (UTC)[reply]
Requiring developer action[edit]
For implementation[edit]
Currently MediaWiki's settings would only allow us to prevent non-autoconfirmed users from creating pages in any namespace, including their own userspace. (This is the createpage userright - see m:Manual:User rights - which in MediaWiki all user groups have by default. On en.wp's MediaWiki configuration this is currently removed from anonymous users, with the result that everyone but anonymous users can create pages.) Possibly the desired result can be better achieved another way, but the most obvious solution is:
- A new user right,
createarticle is required, assigned to all groups by default, like createpage is now. That needs a modification to MediaWiki.
- For en.wp that new right needs removing from anonymous ('*') and non-autoconfirmed ('user') user groups. That needs a modification to the MediaWiki settings, once 1. is done.
- Create MediaWiki:Nocreatetext-anon as a variant of MediaWiki:Nocreatetext shown to anonymous users, if it exists.
For trial data[edit]
MediaWiki:Nocreatetext is the message shown to users who try to create an article and don't have the necessary permission.
- The current MediaWiki:Nocreatetext would be moved to MediaWiki:Nocreatetext-anon (which would be shown to anonymous users in that situation)
- A new version of it would be created. It would be much the same, but replacing the "log in /create" line with "click here to start an article". That link would go a page looking something like this mockup: User:Rd232/creationdraft.
I am very dismayed at the result that we have to have a "trial period". The pending changes trial could not have been more of a disaster. WP:PCRFC is still in limbo, no up or down decision has been made after literally years of discussion. We must not repeat the mistakes that led to that situation. We need to plan out what happens after the trial period now, or we will end up in the same indefinite quagmire, to the benefit of nobody. That being said, I really don't like the option to allow creation with the article wizard. The idea behind the wizard is a good one, and when it works it works brilliantly. Unfortunately it has also led to a lot of new articles that look more like proper Wikipedia articles but are still junk. On another note, we also need some safeguard here to prevent this from turning into another thousand headed hydra of a discussion with thirty or forty separate proposals duking it out, insuring that no one proposal will ever attain consensus. How to do that? Not sure. Beeblebrox (talk) 21:15, 2 June 2011 (UTC)[reply]
- I'd concur; the Wizard discussion is related, but should probably be separated from this, at least at this point. Once we get this implemented and see what happens, then maybe we can talk about it, but I want this to focus on the actual implementation. I've got my idea above (6 months/1 month/long-term), and to keep this from becoming a giant mess I'd like other proposals to be along those lines. The Wizard discussion, as I see it, is something that should happen separately. That would help keep the discussion under control, as it would streamline the discussion into the more straightforward question; how do we implement the consensus at the initial RfC? The Blade of the Northern Lights (話して下さい) 21:26, 2 June 2011 (UTC)[reply]
|