Commons:Requests for comment/Technical needs survey/Taking on certain upload tools

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search
Overview page Next proposal

Taking on certain upload tools[edit]

Description of the Problem[edit]

  • Problem description: Certain tools, many of which are not part of the mediawiki itself, are nonetheless very basic for people who upload files to Commons. Many of these are currently each maintained by a single individual. We need a plan for more robust maintenance of these over time.
  • Proposal type: process request
  • Proposed solution: a program manager at WMF should be responsible for a plan for maintenance (or replacement) of these tools going forward. I (Jmabel) am not trying to dictate a particular technical solution here, just to have some entity that is not "the community" take primary responsibility. If this is best done by a paid team at WMF, great. If this is best done by a better-organized and "deeper" pool of volunteers, great. And some might best be left to exactly whoever is doing them now, but if that is a single individual we need at least a plan as to what should happen if that individual becomes unavailable. If it's some mix of the above, or even third parties like the Flickr Foundation, great. And if individuals want to contribute on their own, and the community can adopt their tools or not, that's also great. But I think we need program management from within WMF so that someone has the job of making overall status visible and making sure the ball doesn't get dropped.

Initially, we need to identify what tools would have this status. People are welcome to add to this initial list (and/or clarify situations), but please stick to existing (or previously existing and now broken) tools used by contributors who upload content.

  1. Special:UploadWizard: as I understand it, this is part of mediawiki, and is already maintained by WMF staff
  2. Special:Upload: as I understand it, this is part of mediawiki, and is already maintained by WMF staff
  3. Uploading apps for mobile devices (I know nothing here, I never use them, can someone please fill this in?)
  4. Flickr2Commons: the Flickr Foundation has already taken on the task of replacing this with a more robust tool, which I think means this is well covered
  5. Batch uploader(s) (programs running on a PC): there have been several of these over the years, notably Commonist, which I believe is dead. I have no idea of the current status here
    1. Pattypan: for batch upload via spreadsheets, some issues but working, developed by Yarl and maintained by Abbe98
    2. Vicuna Uploader
  6. tool(s) for mass uploads from GLAMs or other databases of file content: I have no idea of the status of these
  7. Video2Commons: especially important because of its ability to convert file formats. This is often broken in one or another degree. See phab:T353659
  8. CropTool: (rotating and cropping, either for overwrite or for a new file). Currently in danger of breaking because the Grid Engine is about to go away and no one has dealt with this.
  9. Url2Commons: for direct upload from the given URL: written by Magnus Manske but not actively maintained (many unresolved issues)
  10. Commons:derivativeFX, tool at https://iw.toolforge.org/derivative: to easily upload derivative works
  11. IA-upload: used to upload PD works on the Internet Archive to Commons as DJVU files. Some commons issues like: phab:T300761.
  12. The API itself using pywikibot or custom scripts
  • Phabricator ticket:
  • Further remarks: I'm very open to "sympathetic edits" to the above proposal, but reserve the right to revert edits that I think hijack my proposal to be something else. - Jmabel ! talk 22:31, 6 December 2023 (UTC)[reply]
Jmabel ! talk 19:13, 17 December 2023 (UTC)[reply]
@Jmabel: With the ideal tool, everything entered by the user should be sharable in the upload session: all or part of the description, source, author, templates, cats, freeform stuff after the description, freeform stuff before the cats... This could follow the model of the granularity of global preferences vs. local preferences.   — 🇺🇦Jeff G. ツ please ping or talk to me🇺🇦 22:25, 17 December 2023 (UTC)[reply]

Discussion[edit]

@Jmabel: (or anyone else). Is there some reason why these things are done through third party solutions instead of just being integrated into the website to begin with? Like is there a reason it's better to have the WMF maintain the CropTool instead of them just making cropping an actual feature of mediawiki? --Adamant1 (talk) 11:15, 7 December 2023 (UTC)[reply]

If this tools will also be available via the API then there is no reason to not make them a feature of mediawiki. But batch uploads via a GUI only tool is no fun. C.Suthorn (@Life_is@no-pony.farm - p7.ee/p) (talk) 16:33, 7 December 2023 (UTC)[reply]
As I say, I'm not prejudging the technical solution here. Obviously, if something can be brought into mediawiki and provide essentially the existing capability, that's great, and also benefits other sites using mediawiki. What I am saying is that for Commons, all of the above constitute part of the core functionality that we provide to uploaders, and that this deserves the same level of program management and, ultimately, robustness as the content editing that is core functionality across the sister projects. - 18:46, 7 December 2023 (UTC)
Thanks for the clarification. I'm certainly not against the proposal. I was just wondering about the trade offs between having them manage the applications in house versus just building similar features into mediawiki. I guess they aren't mutually exclusive though. --Adamant1 (talk) 13:27, 9 December 2023 (UTC)[reply]
I think that is pretty unavoidable. Different problems require different dedicated tooling to emphasise different aspects that help with the problem. —TheDJ (talk • contribs) 12:49, 18 December 2023 (UTC)[reply]
@Adamant1 Working tools the WMF deems useful for all MediaWiki installations are in Core. Working tools the WMF deems useful for some MediaWiki installations are in Extensions. Working tools the WMF deems useful for all WMF MediaWiki installations are in WMF Builds. Working tools developed by others who saw a need and filled it could be upgraded to any of the above. As far as I know.   — 🇺🇦Jeff G. ツ please ping or talk to me🇺🇦 00:49, 10 December 2023 (UTC)[reply]
  •  Comment If there is an interest on going deeper into this topic we could make a voting where the users order the tools on how much they need them. GPSLeo (talk) 07:50, 22 December 2023 (UTC)[reply]
    @Jmabel Do you agree to make a vote on the most needed and used upload tools out of this list? GPSLeo (talk) 09:59, 5 January 2024 (UTC)[reply]
    As I mentioned at another thread, there are three distinct types of users who do uploads: About 2 million users who uploaded less then 20 files each since Commons was started (and one million of these each uploaded only one file ever). 315 users who uploaded 58% of all files that have been uploaded to Commons. And the third group, that uploaded between 20 and 40000 files. All three groups have very different needs for an upload tool and all three groups are probably very different likely to take part in any vote or even know about a vote. A vote should be planned with this difference in view. IMHO only members of the 315 group are actually participating in the discussions. The group of 2 million are the one's whose opinions count outside of the Wikipedia bubble. And the third group (who uploaded 40% of all files) are the people who would benefit the most from better tools. C.Suthorn (@Life_is@no-pony.farm - p7.ee/p) (talk) 11:52, 5 January 2024 (UTC)[reply]
    We should ask for what they see as most relevant tool and not for the tool they are using. I do not use the UploadWizard but as a Wikiloves contest organizer this is the most important tool for me despite I do not use it to upload files myself. GPSLeo (talk) 13:35, 5 January 2024 (UTC)[reply]
  • This is one area where I would like to see more diversity in the movement. I think WMF should take steps to make sure the tool ecosystem can thrive (e.g. support APIs), but I don't know that it should be the one responsible for individual tools, especially the ones not integrated into MW. I think this is the perfect area for chapters to get involved, or individuals via grants. That said, I do think it would be cool if WMF added an s3-compatible upload API, so that people could use more off the shelf tools instead of having to develop custom ones. Bawolff (talk) 19:16, 11 February 2024 (UTC)[reply]

Votes[edit]

Retrieved from "https://commons.wikimedia.org/w/index.php?title=Commons:Requests_for_comment/Technical_needs_survey/Taking_on_certain_upload_tools&oldid=851421833"