Commons:Bots/Requests/MDanielsBot 3

MDanielsBot (talk · contribs) 3

Operator: Mdaniels5757 (talk · contributions · Statistics · Recent activity · block log · User rights log · uploads · Global account information) (assign permissions)

Bot's tasks for which permission is being sought: Assisting with the License Migration by marking files as redundant, not eligible, or eligible.

Details (as of now, may add additional checks in the future):

1. Check if migration is redundant:
   1. If the migration would be redundant (i.e. if the image is licensed under CC-BY-(SA-)3.0 already), mark it as such.
2. Check if the image is ineligible for migration:
   1. If the EXIF data shows that the image was taken after the deadline for migration eligibility (Aug. 1, 2009), mark it as ineligible for migration.
   2. If there is a {{Original upload date}} template showing it is was uploaded after the deadline, mark it as ineligible.
   3. If there is a wikitable with an original upload log showing it was originally uploaded after the deadline, mark it as ineligible.
3. Check if the image is eligible for migration:
   1. If there is a {{Original upload date}} template showing it was uploaded before the deadline, mark it as eligible.
   2. If there is a wikitable with an original upload log showing it was originally uploaded before the deadline, mark it as eligible.

Automatic or manually assisted: Automatic

Edit type (e.g. Continuous, daily, one time run): One time run, then as needed

Maximum edit rate (e.g. edits per minute): Worst-case, 1x/5sec. (uses maxlag).

Bot flag requested: (Y/N): Already flagged

Programming language(s): Python (source available at github) (Updated link: here)

--Mdaniels5757 (talk) 20:37, 22 May 2020 (UTC)

Discussion

Thanks for making the request. How do you plan to exclude users that have previously opted out of license migration back in the days when the campaign was active? --Schlurcher (talk) 15:55, 23 May 2020 (UTC)

@Schlurcher: That's a good question. I'm not aware of any central list, and the only person I was aware of of who opted out was Infrogmation (I note that User:Schlurcher/Lizenz is ineligible for migration because it is v1.2 only, and, even if it was v1.2+, migration would be redundant). There are a couple of things I could do.

1) Create some sort of central list, and have the bot make sure no uploader or person linked to on the page is on it. Requires some effort, but ultimately would work.

2) Do nothing. The opt-out provision "exists at the discretion of the local community which may impose restrictions upon it (such as time limits on when the request can be made), or even prohibit the opt-out option." ([1]), and "copyright holders wishing to exercise this option should act in an expeditious manner" ([2]). They've had a decade, so at this point I think that it could be fair to say that they have not acted in an "expeditious manner" if they haven't opted their photos out yet.

Any other ideas, or thoughts as to what I should do? --Mdaniels5757 (talk) 16:37, 23 May 2020 (UTC)

Thanks for diving into this discussion. It's a long time ago, but I it has been clear to me that Commons has allowed opting out (the option is still a parameter on Commons:License Migration Task Force/Migration, so it is fair to say that the possibility was granted). The problem was that there was no central coordination of this. There are a couple of people that opted out of the process. You might have guessed, that I was one of them. Also Ralf Roletschek (talk · contribs) comes in mind. People have dealt with this differently. I have first started to revert all these changes, but the updates kept coming, so I subsequently re-licensed all my pictures form v1.3 or later to v1.2 only to circumvent the re-licensing. As I understand, Ralf, has stopped uploading full resolution pictures to commons due to this. Others have used the opt-out parameter as described on the task force page. I'm currently most concerned that we re-visit this re-licensing without respecting prior decisions. Especially users might have reverted the edits 10 years ago and have left since then. And we now start all over again, not giving them the option to opt-out again.

I'll try not to be super complicated, but some kind of safeguard seems needed. Do you think there is a cutoff-date by when we can consider that the first round of re-licensing was considered complete? We could then modify your proposal:

1. Check if the image is eligible for migration:
   1. If file was uploaded to commons after the cutoff and there is a {{Original upload date}} template showing it was uploaded before the deadline, mark it as eligible.
   2. If file was uploaded to commons after the cutoff and there is a wikitable with an original upload log showing it was originally uploaded before the deadline, mark it as eligible.

Seems feasible to implement, and you would still be able to check once all files that have been recently transferred from other wikis (the intention of the request) as well as not change, potentially again, files that existed during the first re-licensing activity. I think you should also keep a log of all files re-licensed and only touch them once. --Schlurcher (talk) 18:12, 25 May 2020 (UTC)

Sorry, I'm en-0. Da du mich genannt hast... Ich habe mal ein schlechtes, winziges und unwichtiges Foto als Beispiel. Ich habe das 2004 klipp und klar unter GFDL hochgeladen, nicht mehr und nicht weniger. Es ist ein Unding, wenn das dann Jahre später mit einer Lizenz versehen wird, die es zum Zeitpunkt der Veröffentlichung überhaupt noch nicht gab. Nach kontinentaleuropäischem Recht ist das nicht zulässig, man kann keinem Vertrag zustimmen, deren Inhalt man nicht kennt - und eine Lizenz ist ein Vertragsangebot. Das Ganze ist vollkommen in die Hose gegangen. Bei 25% Ablehnung der Umlizenzierung hätte es ein Opt-In geben müssen. --Ralf Roletschek 18:36, 25 May 2020 (UTC)

Danke, für deinen Beitrag. Sobald ich etwas mehr Zeit habe werde ich sowohl die Diskussion hier als auch deinen Beitrag übersetzten, sodass wir gemeinsam einen Weg finden können und du beitragen kannst, falls du möchtest. --Schlurcher (talk) 21:20, 25 May 2020 (UTC)

Ralf has highlighted a picture that he uploaded Example that was subsequently migrated and he switched it to opt-out. Ralf has also highlighted that he rejects the complete license migration philosophy as a whole and that it may violate local law.

Could you please confirm that you will ignore all picture that have the opt-out parameter? I think you do. --Schlurcher (talk) 19:10, 30 May 2020 (UTC)

Belated, but the answer is "of course, but I (mostly) don't need to". The bot will only run on files in Category:License migration candidates or Category:License migration needs review, but if there is an opt-out parameter, the file shouldn't be in those categories (it'd be in Category:License migration opt-out, which the bot doesn't touch). I just thought of one case where that could be a problem (if there are two templates, one with the opt-out parameter and one without) and I'll add a check for that. --Mdaniels5757 (talk) 18:44, 1 July 2020 (UTC)

Section break

OK, I have a plan for opt-outs:

I will make a list of those who wish to opt out at User:MDanielsBot/LROptOut. I will also put onto it all users who (a) uploaded more than 10 files in Category:License migration opt-out or (b) were linked to in pages in Category:License migration opt-out more than 10 times. I made 10 a minimum to make the list smaller, and keep it to people who are likely to have files pending migration. I will reach out to active users on the list below to make sure they want to opt out. If a user adds their name to the "manual" section of User:MDanielsBot/LROptOut, I will have the bot mark their file as "opt-out". If I add a user's name to the "automatic" section (which I will get from the table below, once I clean it up), the bot will skip the file. If you're curious, here is the table (ignore the id column):

Extended content

+-----+--------------------------------------+-------+ | id | link | count | +-----+--------------------------------------+-------+ | 10 | User:Infrogmation | 32669 | | 13 | User:Fir0002 | 2215 | | 47 | User:FloraUploadR | 1577 | | 42 | User:Gone Postal | 1084 | | 29 | User:Slowking4 | 1044 | | 43 | User:Kogo | 977 | | 63 | User:MGA73bot2 | 926 | | 8 | User:EvaK | 830 | | 52 | User:Smial | 304 | | 22 | User:Erin Silversmith | 279 | | 48 | User:Olei | 276 | | 46 | User:Erkan Yilmaz | 245 | | 18 | User:Rotatebot | 241 | | 26 | User:Rillke | 187 | | 5 | User:Ralf Roletschek | 175 | | 15 | User:File Upload Bot (Magnus Manske) | 162 | | 37 | User:Btr | 153 | | 2 | User:Muhammad Mahdi Karim | 118 | | 32 | User:Toutíorîx | 97 | | 49 | User:Rillke/Spice | 91 | | 50 | User:Lensovet | 88 | | 99 | User:NickK | 82 | | 60 | User:Asbestos~commonswiki | 58 | | 119 | User:Kristo~commonswiki | 50 | | 118 | User:Kristo | 49 | | 112 | User:Mark | 45 | | 67 | User:Marcela | 41 | | 11 | User:Kabelleger | 40 | | 156 | User:MER-C | 31 | | 66 | User:Stefdn | 31 | | 205 | User:Dodo von den Bergen | 26 | | 40 | User:Lezo | 25 | | 434 | User:Skyfi | 23 | | 94 | User:Leyo | 22 | | 58 | User:BetacommandBot | 17 | | 81 | User:Andreas Plank | 15 | | 362 | User:Nmedard | 14 | | 1 | User:FastilyClone | 14 | | 31 | User:Charles48 | 14 | | 74 | User:THOR07 | 14 | | 12 | User:Petar Marjanovic | 13 | | 132 | User:Chamaeleon~commonswiki | 12 | | 635 | User:Rfc1394 | 12 | | 70 | User:Magdalens | 12 | | 95 | User:Yellowcard | 11 | +-----+--------------------------------------+-------+ 45 rows in set (0.00 sec)

Best, --Mdaniels5757 (talk) 18:23, 26 May 2020 (UTC)

Seems a reasonable approach. Could you please make the categories of User:MDanielsBot/LROptOut self explanatory and explain there meaning on the page as well. Thank you. For users like User:Kristo~commonswiki, you should exclude both User:Kristo as well as User:Kristo~commonswiki. --Schlurcher (talk) 19:10, 30 May 2020 (UTC)

@Ralf Roletschek: : Mdaniels5757 schlägt nun vor, dass alle weiteren Dateien von Nutzern, die bereits mehr als 10 Mal eine Datei als opt-out generell von seinem neuen Durchlauf der Migration ausschließen wird. Damit würde der Bot keine deiner Bilder mehr migrieren. Wäre dies für dich ein akzeptabler Kompromiss? Das adressiert natürlich in keiner Weise deine Bedenken bezüglich der Migration im Allgemeinen, aber die Migration wurde ja auch schon vorher so durchgeführt. Mein Hauptaugenmerk ist nun daran, dass Personen, die schon einmal widersprochen haben jetzt nicht nachträglich Zwangsmigriert werden, nur weil Sie nicht mehr aktiv am Projekt beteiligt sind. --Schlurcher (talk) 19:10, 30 May 2020 (UTC)

Für mich incl. meinen Vorgängeraccount Marcela wäre das ok. Ohne Rücksprache glaube ich, das auch für Fir0002 und EvaK sagen zu können, sie sind beide schon lange inaktiv. Beide haben ihre Mitarbeit unter anderem wegen der Umlizenzierung eingestellt. Was ist mit den Leuten, die nie widersprechen konnten, weil sie vor der Umlizenzierung aufgehört haben? Von einer stillschweigenden Zustimmung kann man ja nur bei Leuten ausgehen, die das mitbekommen haben und nichts dagegen unternehmen. --Ralf Roletschek 19:29, 30 May 2020 (UTC)

Hi Ralf, bei dem Vorschlag hier geht es um Dateien, die vor der Umlizenzierung in ein anderes Wikipedia Projekt hochgeladen wurden, aber erst jetzt nach Commons übertragen wurden. Insofern ist dies quasi nur eine Nachholung der großen Migration. Dein Einwand ist natürlich gleichermaßen berechtigt. Traf allerdings auch bei der vorherigen großen Migration zu und wurde auch damals als Risiko akzeptiert. --Schlurcher (talk) 08:10, 31 May 2020 (UTC)

"Als Risiko akzeptiert" - nur nicht von den Betroffenen. Man darf doch keine Lizenz ändern, wenn der Urheber dem nicht zustimmt. Und es gibt keine Notwendigkeit. --Ralf Roletschek 08:43, 31 May 2020 (UTC)

I think the aspect whether this shall or shall not be done at all should be discussed at the Village pump or a similar venue with larger community feedback. The bot job itself appears as nobrainer to me, and can be approved if there is community support. Different opinions welcome. --Krd 06:55, 7 June 2020 (UTC)

@Krd: OK, thanks. Is there anything in particular you think should be discussed? (e.g. whether to clear the migration backlog at all, opt-outs in general, the bot's particular method of determining opt-outs, etc.) Best, --Mdaniels5757 (talk) 19:59, 9 June 2020 (UTC)

•  Comment I'm one of the users that have been working a lot with license migration. I fixed hundreds of thousands of files on Commons, en.wiki and other wikis.

It is impossible to have a bot fix everything because someone will make crappy moves from xx.wiki to Commons and someone will edit files on Commons and ruin the license or file history.

For example A upload a photo on en.wiki in 2008 and license it GFDL. My bot fixed the license migration on en.wiki. In 2017 B downloaded the photo and upload it manually on Commons and added GFDL but did not add an original upload log. Another example A uploaded the photo on en.wiki as PD-self. B then moved the file to Commons and added GFDL. In both cases the bot can't make the right choise.

I think that what is suggest above is a good solution. The bot will only fix the files if there are very good indicators of what the correct answer is.

I  Support the approval of the bot or at least that there is a test run. --MGA73 (talk) 18:19, 12 June 2020 (UTC)

@Mdaniels5757: I think is should be discussed if the intended job shall be generally done, i.e. changing file license information. --Krd 10:23, 14 June 2020 (UTC)

@Krd: I do not understand what needs to be discussed? Per m:Licensing_update it was decided to do the license migration. The only reasons it is not done yet is 1) because the files were spread out over more than hundred wikis and not all wikis have finished the project 2) because transfers to Commons are not always done as good as they should and 3) because it is still possible to upload files that include a GFDL. I almost finished the proces on Commons 10 years ago and when I returned after a break there were again thousands of files to fix.

The only way to end the project is to stop uploading files to wikis that include a GFDL and move all files to Commons. The bot is meant to make our lives easier. --MGA73 (talk) 14:07, 14 June 2020 (UTC)

Adding on that bots have been approved to change licensing information for the migration in the past: Commons:Bots/Requests/AbiBot and Commons:Bots/Requests/DrilBot. --Mdaniels5757 (talk) 18:05, 14 June 2020 (UTC)

@Krd and Mdaniels5757: Can we get a test run so we can see how the bot works? Atm. the process is manual and the script I used earlier do not work (see Commons_talk:License_Migration_Task_Force#Restart_2020). --MGA73 (talk) 20:51, 20 June 2020 (UTC)

@MGA73: The test run was here. --Mdaniels5757 (talk) 21:41, 20 June 2020 (UTC)

• @Mdaniels5757: Thank you! I checked all the edits and they are correct. Some of the file pages are messy but we can't blame that on the bot. I think bot is ready to be approved. --MGA73 (talk) 22:00, 20 June 2020 (UTC)
• @Mdaniels5757 and Krd: What is missing before the bot can be approved? --MGA73 (talk) 19:11, 30 June 2020 (UTC)
  • @MGA73: I'm not quite sure. User:Krd wanted some more discussion on whether the migration should happen at all and/or whether a bot should help (not quite sure of the scope), but hasn't yet responded to my mention of previous approvals for similar tasks or your request for clarification. From a technical standpoint, the bot is ready (with some minor improvements mentioned above that I may add in the future). Best, --Mdaniels5757 (talk) 18:43, 1 July 2020 (UTC)

  I'm not sure if there will be anything to discuss, but I think it's reasonable to start such discussion at a more popular venue, as bot request are no broadly watched. If there arises that nothing else is required, I think that can be approved shortly. --Krd 11:17, 6 July 2020 (UTC)

Thank you for reply Krd. The license migration was announced on meta at m:Licensing update and it was a global Wikimedia project made by m:Wikimedia Foundation Board of Trustees. Also see wmf:Resolution:Licensing update approval. I can't find the actual vote but the result can be seen at m:Licensing update/Result. There was a total of 17462 votes and 13242 or 75.8% said yes. So I think we had enough users to say yes. I doubt we will have 17 k users comment even if we make a post at VP.

Hundred of thousands of files have been migrated already (see Category:License migration completed 608 k files and Category:License migration not eligible 84 k files). What we have left is files that have been moved to Commons after the big license migration in 2009 and 2010 or files that was never finished because they were too hard to migrate without manual review. Now we have a new bot that seems to be able to fix some more files that we would otherwise have to fix manually.

If it was a new project I would totally agree that it should be discussed first. But it is not a new project it is trying to end a 11 year old project approved by WMF and 13k users. --MGA73 (talk) 12:18, 6 July 2020 (UTC)

I will then go ahead and call this approved now. --Krd 13:24, 6 July 2020 (UTC)