Wikipedia:Bots/Requests for approval/MusikBot II

The following discussion is an archived debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA. The result of the discussion was

Approved.

MusikBot II

Operator: MusikAnimal (talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)

Time filed: 03:47, Tuesday, November 1, 2016 (UTC)

Automatic, Supervised, or Manual: Automatic

Programming language(s): Ruby

Source code available: GitHub

Function overview: Maintains the AWB CheckPage and counts the number of users with access

Links to relevant discussions (where appropriate): Special:Permalink/747216495#New bot task for AWB pages, Wikipedia:Bureaucrats'_noticeboard/Archive_21#Request_for_bureaucrats_promoting_candidates, Wikipedia:Bots/Requests_for_approval/AWBCPBot

Edit period(s): Twice weekly

Estimated number of pages affected: 1 (CheckPage), 4 in the bot's userspace, and maybe 0-2 user talk pages per run (see #6 in the function details)

Exclusion compliant (Yes/No): No

Adminbot (Yes/No): Yes

Already has a bot flag (Yes/No): No

Function details: Most functionality takes over what User:ListManBot did (bot operator is aware). The only new functionality is to automatically revoke inactive and blocked users. The specifics are as follows:

Alphabetize the list and remove any duplicates
Make all formatting consistent (whitespace, newlines, etc)
- The prose within the CheckPage will remain untouched (text above and/or below the list of users)
Update to reflect any user renames
- The bot keeps track of users who were present on the CheckPage on the last run. It will only rename the account if it sees that one of the previous users has been renamed since the last run. This addresses the concern of where an account is WP:USURP'd and the new account should not have access to AWB
Remove admins from the list since they automatically have access
Remove users who have been indefinitely blocked for over a year
Remove inactive users
- Inactive is defined as one year of no editing activity (not any logged action, since AWB is for editing)
- The reason for removing inactive users is purely to keep the CheckPage tidy. It has grown to be quite large and hard to manage, so we want to remove users who evidently are no longer using the software.
- One week prior to revoking the bot will issue a talk page notice with the content:
  Hello [username]! This message is to inform you that due to editing inactivity, your access to AutoWikiBrowser may be revoked. If you do not resume editing within the next week your username will be removed from the CheckPage. This is purely for routine maintenance and is not indicative of wrongdoing on your part. You may regain access at any time by simply requesting it at WP:PERM/AWB.
- If they do not resume editing they will removed from the CheckPage and added to User:MusikBot II/AWBListMan/Report/User, which will serve as a reference for admins
- When new requests for access are made at WP:PERM/AWB, MusikBot will comment if they are on the auto-revoked page, indicating to admins it may be a speedy approval
The same processing will be done on the bot section of the CheckPage, but currently there is not consensus to remove inactive bots. Instead the same report will be generated for bots as if they had been removed, again serving only as a reference
The counts of users/bots with access are written to User:MusikBot II/AWBListMan/User count and User:MusikBot II/AWBListMan/Bot count. This will be transcluded in {{Wikipedia:AutoWikiBrowser/User count}}

Functionality can also be tweaked at User:MusikBot II/AWBListMan/config.

Since this will require the bot have admin rights, I'm getting a head start by opening the BRFA while I finish implementing some functionality (specifically renamed users, #3 above). All other functionality should be good to go, see the edits on the test CheckPage: [1][2] (editing of the bot section has since been disabled). The reports can be found at User:MusikBot II/AWBListMan/Report/Bot and User:MusikBot II/AWBListMan/Report/User.

If the bot is approved, I think I'll first send out a mass message to all 580 inactive users that their access is about to be removed (see #6 above), and then have the bot do it's first run a week later. Moving forward, the bot will generally only be notifying a few users at most per run, and hence mass message will not be needed.

— MusikAnimal ^talk 20:18, 1 November 2016 (UTC)[reply]

Notifications

Community notifications sent:

— xaosflux ^Talk 04:20, 1 November 2016 (UTC)[reply]

Discussion

MusikAnimal - do you plan on using OAUTH, BotPasswords or the legacy authentication system? If OAUTH or BotPasswords - please link to your consumer/botpasswords configuration. — xaosflux ^Talk 14:02, 1 November 2016 (UTC)[reply]
Currently using BotPasswords. The grants are listed at Special:BotPasswords/MusikBot II. So the only admin-y thing it can do is edit protected pages. What's also really cool is we can restrict usage to the IP range of Tool Labs (which I think is just one single IP). I have not done this yet as I'm currently doing testing from my local machine — MusikAnimal ^talk 18:10, 1 November 2016 (UTC)[reply]
Actually it looks like you can't link directly to a bot's grants page. I've poked some people on IRC about this. I guess take my word for it? :)
Also wanted to say I'll be on holiday this Friday through next Tuesday (8 November). If I don't write back here during that time that is why. Best — MusikAnimal ^talk 00:08, 3 November 2016 (UTC)[reply]
Hi MusikAnimal, just a little note but shouldnt the "Estimated number of pages affected:" include user talk pages for your one week removal notices? KylieTastic (talk) 19:51, 1 November 2016 (UTC)[reply]
Thank you! I have modified the estimate, though it honestly is a complete guess. The first go around I'm going to personally mass message all 580 inactive users, then run the bot a week later — MusikAnimal ^talk 20:18, 1 November 2016 (UTC)[reply]
Came here because of noticeboard announcements. Looks like a useful task; the BotOp is an experienced and trusted user. Don't see any problems going forward. -FASTILY 01:49, 4 November 2016 (UTC)[reply]

On hold until 09NOV2016 to allow for community comments. — xaosflux ^Talk 02:15, 5 November 2016 (UTC)[reply]

MusikAnimal Go ahead and send out your MassMessage - as that is really not a function of the bot - but may reveal if there are unknown community issues. Please link to this BRFA in your MassMessage. — xaosflux ^Talk 02:01, 6 November 2016 (UTC)[reply]
Done Mass message has been sent (example) — MusikAnimal ^talk 23:38, 8 November 2016 (UTC)[reply]
This sounds a reasonable thing to do. Even just one run, or once per year should declutter. Graeme Bartlett (talk) 00:44, 7 November 2016 (UTC)[reply]
A one-time or once a year run will indeed do a lot of cleanup with a single edit, but the hope is to fully automate the process. Things like renamed users are more time-sensitive, and there's little concern of impact on site performance, so I think a higher frequency is desirable. ListManBot, who did all of the same tasks except removing inactive/blocked users, ran once a week — MusikAnimal ^talk 00:03, 9 November 2016 (UTC)[reply]
The message is a worded a little oddly given that it appears to have been sent to users whom are not inactive. :D Otherwise, carry on with the task. --Izno (talk) 23:43, 8 November 2016 (UTC)[reply]
@Izno: Sorry about that! The mass message system followed a redirect of your alternate account IznoRepeat, which indeed has not edited in over a year. This is an excellent scenario that I'm glad came up... What's different is that the bot (as opposed to mass message) will include the username in the message, that way you would know it was referring to your alt account and not your primary. The other issue that surfaced just now was that the message was sent users who were renamed but are still active. The logic to detect renames has not been implemented yet, but either way it wouldn't have been caught because there wasn't a previous run to compare to (see #3 above). I have manually gone through and rollbacked the other 7 improperly sent messages. Apologies for the noise :) — MusikAnimal ^talk 00:02, 9 November 2016 (UTC)[reply]
Ah. I will go make an edit over there. --Izno (talk) 00:09, 9 November 2016 (UTC)[reply]

Approved for trial (1000 edits). Please provide a link to the relevant contributions and/or diffs when the trial is complete. . For trial, please make a complete copy of WP:AWB/CP in userspace, and run the job. After running, if you are comfortable with the results use your exiting admin account to sync this in to the check page. — xaosflux ^Talk 14:30, 11 November 2016 (UTC)[reply]

@Xaosflux: Thanks! I will re-run what I have very soon. This will make only 5 edits (userspace version of the CheckPage, 2 edits for the reports, and 2 for the counts). After that, the bot will update the counts every time someone is added to the CheckPage. I estimate that would involve 5-10 edits per month. Updating the CheckPage itself is the only complicated task, and the only one requiring the admin bit. Cleanup there would only happen maybe a few times per month at most (see ListManBot's edits to the CheckPage). 1,000 edits will take many years :)

Instead I recommend emulating all the scenarios in the bot's userspace (User:MusikBot II/AWBListMan/CheckPage). E.g. I can add one of my test accounts, then rename it (which I'm able to do), and see if the bot correctly updates the test CheckPage. Similarly I could put the users in the wrong order, add extra whitespace, duplicates, add myself (an admin) and see if the correct adjustments are made. I think this is the only feasible way to run a trial as edits to the actual CheckPage will be infrequent, and I can't assure all scenarios will be covered within a reasonable amount of time — MusikAnimal ^talk 23:02, 11 November 2016 (UTC)[reply]

That is fine, I left the edit count high in case you wanted to send out the editor notifications. — xaosflux ^Talk 23:12, 11 November 2016 (UTC)[reply]

Those I already sent out via mass message. On that note, I owe those folks 4 more days before removing them from the CheckPage (one week's notice). In the meantime I'll finish what little coding is left to handle renames, then next Tuesday I'll do a full run in the userspace, and assuming all is well I'll manually sync with the actual CheckPage. From there I'll run the userspace tests described above, and report back here :) — MusikAnimal ^talk 23:37, 11 November 2016 (UTC)[reply]

@MusikAnimal: How is this going? I looked over the bot's edits, and they seemed odd to say the least. There's obviously more than 8 editors who are inactive on the CheckPage. Am I missing soemthing? ~ Rob₁₃^Talk 11:43, 16 November 2016 (UTC)[reply]

It was going off of a version of the CheckPage after a previous run. So just a few days went by, and 8 more users became "inactive"! Right now I'm struggling with some connection issues on my local machine. I might have to run the bot on Tool Labs for the trial. Hopefully I'll have the actual CheckPage updated today, and the other aspects of the trial (testing that renames are properly done) will soon follow — MusikAnimal ^talk 19:30, 16 November 2016 (UTC)[reply]

Trial complete. This was done in several steps, I will try to carefully explain. All tests were done against User:MusikBot II/CheckPage:

First, the bot was ran to remove admins and indefinitely blocked users (blocked over a year ago): [3]. This edit also illustrates how it sorted the list. I have synced these changes to the actual CheckPage [4]. Then the edit to update the report: [5]
The bot was ran again, this time looking at inactive users (this was done as a separate edit so that the trail is easier to evaluate): [6]. Here I restored alternate accounts to the report and the bot's cache, so that they can be properly notified (see above, the original mass message sent out did not indicate it was the user's alternate account). Again I synced the actual CheckPage with these changes [7]. And the updated report: Special:PermaLink/751455529
On the next run, no changes to the CheckPage were made, only the notifications of inactivity were sent out to the alt accounts (e.g. [8]). The bot's local cache has been updated, so when it is ran a week from now, any of those alt accounts that still have not edited would be removed from the CheckPage.
For bots, editing of the CheckPage has been disabled in the config, so only a report was generated [9]

Some more examples, using a smaller example list of users for easier analysis:

Removing admins, indefinitely blocked users and inactive users CheckPage Report
Handling whitespace, sorting, duplicates, and respecting text before/after the list of users CheckPage (no change to report)

For renaming users, I chose not to do an actual rename and pollute the rename log (and have to undo a rename of my test account). Instead, following the logic in #6 of the function details, I've changed the "last run" datestamp (stored on the filesystem) to a time in the past, before Ks5stm was renamed to Ks0alt. After running the bot again we have [10]. If we'd like to do a real rename test, we can, but hopefully this diff asserts the bot is capable of properly handling renames.

There were some points where I messed up the bot's local cache during testing, which resulted in an inaccurate report. All of that has been sorted out, and I'm happy to do more example tests using a smaller pool of users so that evaluating the bot's actions is easier, should the above be insufficient. I can however say with confidence that the bot is stable and in my opinion ready for production. Thank you for your time! — MusikAnimal ^talk 20:58, 25 November 2016 (UTC)[reply]

Approved. Will need +sysop, +bot. — xaosflux ^Talk 01:36, 4 December 2016 (UTC)[reply]

The above discussion is preserved as an archive of the debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA.