Commons:Bots/Requests/RussaviaBot
RussaviaBot (talk · contribs)
Operator: Russavia (talk · contributions · Statistics · Recent activity · block log · User rights log · uploads · Global account information)
Bot's tasks for which permission is being sought: Flickr uploads
Automatic or manually assisted: Both automatic and manually assisted, dependant on the stream being uploaded
Edit type (e.g. Continuous, daily, one time run): As needed
Maximum edit rate (eg edits per minute): 10 edits per minute (unlikely given time required between uploads)
Bot flag requested: (Y/N): Y
Programming language(s): flickrripper.py
russavia (talk) 19:21, 22 December 2012 (UTC)
Discussion
I will also require the reviewer flag for the bot. russavia (talk) 19:21, 22 December 2012 (UTC)
Info I just granted the reviewer flag to the bot; I'll leave it up to the other bureaucrats to decide on giving (or not) the bot flag to Russavia's bot. odder (talk) 19:43, 22 December 2012 (UTC)- Why such a high upload rate? The guidelines recommend a maximum of 10/min. Also, how does this fit with all the other Flickr uploaders? Didn't they just give admins a way of directly uploading a set from Flickr without downloading it first? --99of9 (talk) 21:02, 22 December 2012 (UTC)
- Sorry my mistake. 10 edits per minute (unlikely given....). We do have a new way of directly uploading sets (up to a maximum of 50 files), but it's broken. It's also no good when one has a stream of say 23,000 photos which we can upload having to rely on a broken feature of upload wizard. At least with the bot, one is able to do what they need to do, and know it will work first time :) russavia (talk) 21:10, 22 December 2012 (UTC)
Example set has been uploaded to Category:Les Misérables premiere in Sydney (this was a manually reviewed set) russavia (talk) 08:12, 23 December 2012 (UTC)
- Among the set/streams that will be uploaded from are http://www.flickr.com/photos/kkendall/2733754814/, http://www.flickr.com/photos/lesotho/,
http://www.flickr.com/photos/sgerner/with/7849481018/#photo_7849481018, http://www.flickr.com/photos/babasteve/sets/72157617989418894/with/4615112282/, http://www.flickr.com/photos/kdixon/sets/72157629450822445/with/6951742723/, http://www.flickr.com/photos/56880002@N04/sets/72157625567546258/with/5247473187/, http://www.flickr.com/photos/emmanueldyan, http://www.flickr.com/photos/evarinaldiphotography plus many, many more. Just to give examples of what needs to be uploaded. russavia (talk) 12:31, 23 December 2012 (UTC)
Example set uploaded to Category:Nicki Minaj Pink Friday perfume launch in Sydney (this was an automated set). russavia (talk) 14:43, 23 December 2012 (UTC)
- Please use language templates for Description/Author/Source fields. Description looks very excessive and more like spam then information about image. --EugeneZelenko (talk) 15:34, 23 December 2012 (UTC)
- This is an example of an automated upload; use of language templates for the description is not possible (I believe). Language templates for the author and source fields isn't just not possible, but would be outright silly if it were available. The description can be modified, as I will also be reviewing all uploads once completed; recategorising, fixing descriptions, etc. russavia (talk) 16:20, 23 December 2012 (UTC)
- I agree it's not really useful for the Author/Source, but for so many uploads, it's important to automate the language tags around the description. We may need to customize flickrripper.py a bit, but it won't be too hard. --99of9 (talk) 00:27, 24 December 2012 (UTC)
- This is not currently required for this tool which can also upload entire sets, and the like. I could always use that tool as plenty of others do, and be left with same problem of language templates. I think we need to weigh up whether we want thousands of relevant photos on the project at the 'expense' of a single template. The community basically says yes, given its previous approvals of the use of flickrripper.py at these very request forums. One could look at Commons:Bots/Requests/タチコマ_robot, as an example, which also allows for the use of flickrripper.py and which has been put to use for myself with the uploading of Category:Photos of Turkmenistan by Kerri-Jo -- all of which were cleaned up by myself after uploading. That is just one bot which is authorised to use flickrripper.py. So I fail to see why this request would be treated any differently to all the other past requests for the use of flickripper.py. russavia (talk) 07:27, 24 December 2012 (UTC)
- It's not either/or. We can have both the language template and the uploads, we just need to improve the script in advance of the upload. (It will only be about a one line change.) I'm just aiming for best-practice, not rejecting your offer. Going forward I will ask for the same of other applicants (especially if we've already improved the script). --99of9 (talk) 09:29, 24 December 2012 (UTC)
- Right, we should be aiming for best-practice, but in some instances it is extremely difficult. Take, for example, a stream which has some descriptions in English and some in Russian. The language templates, if we code it into flickrripper.py, it would make it impossible to run -autonomous uploads of a largish stream, without a heap of cleanup work afterwards. I'm not against requiring editors to cleanup uploads, in fact, I'm all for it. But let's see what we can work out in the coming days. If you want to do the code for flickrripper.py let me know, and I will pastebin the code I am using (in fact, check your email, I'll send you the link to private pastebin). Cheers, russavia (talk) 01:51, 25 December 2012 (UTC)
- Requests for Bot flag is neither the place or median to make enhancement requests for code that is in regular use. Bots should not be required to do anything beyond what is required of people. People aren't required to annotate the language that image description template field displays. Furthermore what is requested here is an impossibility. While multi-million/billion dollar Artificial Intelligence implementations such as Google translate are able to distinguish a few languages, they are generally unreliable for most other languages. Mind that flickrripper.py relies on various toolserver tools that basically determine what the description template the script spits supposed to contain. You are welcome to consult those people to improve their code which would improve flickrripper.py as well but as I mentioned, this is a very difficult endeavor. -- とある白い猫 ちぃ？ 13:11, 25 December 2012 (UTC)
- Right, we should be aiming for best-practice, but in some instances it is extremely difficult. Take, for example, a stream which has some descriptions in English and some in Russian. The language templates, if we code it into flickrripper.py, it would make it impossible to run -autonomous uploads of a largish stream, without a heap of cleanup work afterwards. I'm not against requiring editors to cleanup uploads, in fact, I'm all for it. But let's see what we can work out in the coming days. If you want to do the code for flickrripper.py let me know, and I will pastebin the code I am using (in fact, check your email, I'll send you the link to private pastebin). Cheers, russavia (talk) 01:51, 25 December 2012 (UTC)
- It's not either/or. We can have both the language template and the uploads, we just need to improve the script in advance of the upload. (It will only be about a one line change.) I'm just aiming for best-practice, not rejecting your offer. Going forward I will ask for the same of other applicants (especially if we've already improved the script). --99of9 (talk) 09:29, 24 December 2012 (UTC)
- This is not currently required for this tool which can also upload entire sets, and the like. I could always use that tool as plenty of others do, and be left with same problem of language templates. I think we need to weigh up whether we want thousands of relevant photos on the project at the 'expense' of a single template. The community basically says yes, given its previous approvals of the use of flickrripper.py at these very request forums. One could look at Commons:Bots/Requests/タチコマ_robot, as an example, which also allows for the use of flickrripper.py and which has been put to use for myself with the uploading of Category:Photos of Turkmenistan by Kerri-Jo -- all of which were cleaned up by myself after uploading. That is just one bot which is authorised to use flickrripper.py. So I fail to see why this request would be treated any differently to all the other past requests for the use of flickripper.py. russavia (talk) 07:27, 24 December 2012 (UTC)
- I agree it's not really useful for the Author/Source, but for so many uploads, it's important to automate the language tags around the description. We may need to customize flickrripper.py a bit, but it won't be too hard. --99of9 (talk) 00:27, 24 December 2012 (UTC)
- This is an example of an automated upload; use of language templates for the description is not possible (I believe). Language templates for the author and source fields isn't just not possible, but would be outright silly if it were available. The description can be modified, as I will also be reviewing all uploads once completed; recategorising, fixing descriptions, etc. russavia (talk) 16:20, 23 December 2012 (UTC)
- Bots are tools to do mass actions. It is not good idea to not to discuss how to do this action in best way apriori. Why other bots or people should clean-up after this one? I hope image selection on Flickr is not random and bot operator could do necessarily adjustments as needed. --EugeneZelenko (talk) 16:00, 25 December 2012 (UTC)
- Other bots or people are not going to be required to clean up after the operation of my bot; for I will be doing so myself, and have already done so myself. Plenty of people have been approved to use bots incorporating flickrripper.py, so really, my request should not be treated any differently than any other request. Otherwise, we really should be revoking the bot flag of all other bots using flickrripper.py. There is absolutely no reason not to grant this flag. russavia (talk) 16:21, 25 December 2012 (UTC)
- Sorry, just to elaborate. If it is technically impossible to add language tags to the -autonomous uploads (which is how I will mainly be doing the uploads), then we should not be holding up this request for any longer than is necessary. 99of9 will take a look at the flickrripper.py code I've pastebin'ed for him; so we'll see what happens with that and whether it is an easy process. russavia (talk) 16:28, 25 December 2012 (UTC)
- Bots are tools to do mass actions. It is not good idea to not to discuss how to do this action in best way apriori. Why other bots or people should clean-up after this one? I hope image selection on Flickr is not random and bot operator could do necessarily adjustments as needed. --EugeneZelenko (talk) 16:00, 25 December 2012 (UTC)
- Looks like you select big sets of photos made by particular photographer. I doubt that there will be too much variations in language for such conditions. You could specify language for such sets manually and bot will do the rest. Sure, it may be annoying for individual images. --EugeneZelenko (talk) 16:01, 26 December 2012 (UTC)
- Sure, that shouldn't be a problem. I have uploaded via autonomous a sizeable set of images of people of Bhutan to give this a proper test, before this gets approved and I am let loose on Flickr ;) All appears to be good with the set (150 images uploaded), just the normal categorisation and description cleanup is required (nothing major). You can see the images in Category:Steve_Evans_photos_of_Bhutan -- they are the files with title of "xxx - Flickr -- babasteve". All looks in order, and I think we should be good to go :) russavia (talk) 17:15, 26 December 2012 (UTC)
- Looks like you select big sets of photos made by particular photographer. I doubt that there will be too much variations in language for such conditions. You could specify language for such sets manually and bot will do the rest. Sure, it may be annoying for individual images. --EugeneZelenko (talk) 16:01, 26 December 2012 (UTC)
- Category:Photographs by Emmanuel Dyan -- the synagogue photos in this category are a better example of automated uploads using flickrripper.py russavia (talk) 17:45, 23 December 2012 (UTC)
Code is now done, and one can see File:Mel. B - Flickr - Eva Rinaldi Celebrity and Live Music Photographer.jpg, File:Mel. B - Flickr - Eva Rinaldi Celebrity and Live Music Photographer (3).jpg and File:Mel. B - Flickr - Eva Rinaldi Celebrity and Live Music Photographer (1).jpg for the results. russavia (talk) 07:02, 26 December 2012 (UTC)
.jpg){kind=link}
.jpg){kind=link}
- Russavia has a wealth of experience as a bureaucrat as to what counts as best practice around here, and plenty of folks will be keenly looking over his shoulder. If he has a bucket load of goodies to bring to Commons by bot, there seems little point in spending a long time in discussion over the bot flag. It would be good practice to get some independent views on a significantly sized test set before uploading tens of thousands of new files, though this might be usefully done under the bot flag at Russavia's discretion. --Fæ (talk) 23:56, 23 December 2012 (UTC)
- Recent batch uploads from Flickr by MaybeMaybeMaybe have resulted in a substantial amount of community effort being directed at cleaning up the files that were in violation of COM:SCOPE and/or COM:IDENT, as well as weeding out duplicates. In addition, there are policy discussions in progress about bulk uploads from Flickr and consent requirements of various countries. This seems like poor timing. Delicious carbuncle (talk) 13:45, 24 December 2012 (UTC)
- Default categorization is not perfect on my point of view. So will be good idea to operate bot without flag to give images more chances to be processed by humans or at least use {{Check categories}}. Is it possible to suppress photographer category (like Category:Steve Evans) at the time of upload? --EugeneZelenko (talk) 15:42, 29 December 2012 (UTC)
- The bot pulls categories from the tags on the stream, so it isn't possible to suppress categories. I am personally going thru all uploads and doing the categorisation, and fixing them all. This is being done by placing into Category:Files from Kentaro Iemoto Flickr stream (check) (for example), and then once categorisation is done files will be moved to Category:Files from Kentaro Iemoto Flickr stream. But if it is cool for me to operate it without the bot flag, then that's great, as I have tens of thousands of photos to upload -- I just got one stream of 4,000 Japanese aviation photos (the category just above) relicenced to CC-BY-SA, and I have requests from other editors to upload their streams (Yannf, for example). It just means that they will appear on recent changes -- and if the community is happy for that, then great. russavia (talk) 16:34, 29 December 2012 (UTC)
- Also, Eugene, if we are going allow flickrripper.py to be used by myself without the bot flag, we need to discuss whether this is also going to allow other editors to do the same thing. And if this is, then we need to discuss what requirements we should expect from editors -- i.e. admins only? long-term editors? Will they be required to request being able to use it? (I say yes). Otherwise we are opening the door to all and sundry. russavia (talk) 16:54, 29 December 2012 (UTC)
- I think it's good idea to improve bot code to avoid unnecessary tags actions. Probably tags to category mapping will be best solution, when empty category means tag removal.
- What concern policies, I think bot operation discussion is must be, but bot flag is not necessarily outcome of such discussions.
- EugeneZelenko (talk) 15:16, 30 December 2012 (UTC)
Unless there are further comments I propose to close this as approved but without applying the bot flag, per the discussion above. This does not mean anyone can just use flickrripper: according to the bot guidelines submitting a request for approval is required irrespective of whether the bot flag is to be applied. Btw, russavia, please check your process re:categorization, in a small random sample I found File:Karima Delli - Flickr - Yann Forget.jpg without any subject categories. Also, russavia, as you've said somewhere, you are on the inclusionist wing of commons contributors, so please be careful to only upload sets which are clearly in COM:SCOPE (as you have so far). We really don't want to have to debate mass uploads of marginal educational value. --99of9 (talk) 11:19, 7 March 2013 (UTC)
- Approved as above. --99of9 (talk) 05:11, 19 March 2013 (UTC)