Commons:Village pump/Technical/Archive/2020/01

Substituting "#invoke"

See also: Commons:Administrators' noticeboard#Spam?.

I wanted to substitute {{subst:#invoke:String|replace|{{BASEPAGENAME}}|%([0-9]+%)%.jpg||plain=false}}, but it seems to be ineffective. Using it will only result in a {{BASEPAGENAME}}. Is there any way to substitute module invokes, specifically Module:String, properly? Thanks. Ahmad^talk 08:27, 1 January 2020 (UTC)

Fixed - I got it. The code didn't work because I forgot to set a subst: for {{BASEPAGENAME}} as well. Resolved now. Ahmad^talk 18:38, 1 January 2020 (UTC)

This section was archived on a request by: Ahmad^talk 18:38, 1 January 2020 (UTC)

POTD/MOTD access

As per this thread, POTD and MOTD creation or editing of descriptions is restricted to users that have autopatrol (autopatroller, patroller, image reviewer, admin etc.) due to security concerns with it's transclusion to the main page. This is a short term measure, long term measure needs to be discussed. A healthy balance between trust/AGF and prevention of abuse is needed to ensure all users can contribute without damaging. Traditionally, only administrators have the ability to impact the main page, however, this situation is different. Non-administrator contributions are needed to ensure descriptions are translated. ~riley (talk) 19:18, 1 January 2020 (UTC)

@~riley: They all have the same structure, right? If yes, maybe we can create an abuse filter for it. The entier text (for POTD) must be in this format, I think: \{\{[PM]otd description\|1\=(.*?)\|2\=[a-zA-Z]+\|3\=[2-9][0-9]+\|4\=[01][0-9]\|5\=[0-3][0-9]\}\}. The filter will disallow edits if they are not in this format. Another alternative (which I personally prefer), in case the first one can return false positives, is to limit the number of { characters used in these pages to 4, require the new wikitext to contain \{\{[PM]otd description\|, and limit the page size as well. This way, users can only add one template to the page, and that one template must be Potd description (or Motd, so we'll kill two birds with one stone). Also, the size limitation will stop the vandal from adding nonsense content in order to bring the main page down. Ahmad^talk 20:23, 1 January 2020 (UTC)

@Ahmad252: I considered this, however, there is nothing in place to stop a user from adding inappropriate text. Sure, another abuse filter will catch vulgar words but there is nothing in place to stop a user from creating a description like "You have no life if you edit Wikipedia" and having it displayed on the main page. ~riley (talk) 21:54, 1 January 2020 (UTC)

@~riley: In that case, I can only think of two solutions: 1. To setup a bot to translate the English description into the target language using, say, Google Translate and then compare the result with the text on the P/MOTD template. If they don't match at all, then it will revert (or delete) the edit (or page) and will log all its actions on a page. 2. To require all the edits on P/MOTD templates (including page creation) to be reviewed first by a patroller before going on main page. I know it's currently possible for existing pages, but for restricting page creations, I think we need a new functionality. We also need something to attract people's attention to the pending changes on these templates before the end of the corresponding day. Ahmad^talk 09:38, 2 January 2020 (UTC)

Tech News: 2020-02

Latest tech news from the Wikimedia technical community. Please tell other users about these changes. Not all changes will affect you. Translations are available.

Changes later this week

• When trying to move a page, if the target title already exists then a warning message is shown. The warning message will now include a link to the target title. [1]
• The new version of MediaWiki will be on test wikis and MediaWiki.org from 7 January. It will be on non-Wikipedia wikis and some Wikipedias from 8 January. It will be on all wikis from 9 January (calendar).

Tech news prepared by Tech News writers and posted by bot • Contribute • Translate • Get help • Give feedback • Subscribe or unsubscribe.

21:18, 6 January 2020 (UTC)

Translation to zh-hans and zh-hant

Translation to zh-hans and zh-hant were disabled, but some pages or templates have zh-hans or zh-hant versions with 0% translation. When the display language set to zh's variant version like zh-cn, zh-hans or zh-tw, they would shown the "zh-hans" or "zh-hant" page by default, not "zh" (like Template:Rename/i18n/zh-hans). --Tim Wu (talk) 08:11, 9 January 2020 (UTC)

First row of drop down menu is never selectable

Annotation 2020-01-10 174633

The first row (p.ex. Araujo & Sobrinhos) of the drop down menu is never selectable when there are more than 4 rows. Ex. File:Porto - façades avec faïences 63 (32957921463).jpg

• Commons Preferences / Gadgets / (checked) Place categories above all other content.

NOTE: When using categories on the bottom of the page (default settings) the problem does not occur.

• OS - Windows 10 Home - Version 1809
• Browser - Google Chrome - Version 79.0.3945.117 (Official Build) (64-bit)

--JotaCartas (talk) 17:59, 10 January 2020 (UTC)

• This problem doesn’t happen with Monobook. -- Tuválkin ✉ ✇ 19:02, 10 January 2020 (UTC)

@Tuvalkin: , obrigado, já experimentei e funciona bem em Monobook --JotaCartas (talk) 12:36, 13 January 2020 (UTC)

Tech News: 2020-03

Latest tech news from the Wikimedia technical community. Please tell other users about these changes. Not all changes will affect you. Translations are available.

Recent changes

• You can no longer read Wikimedia wikis if your browser use very old TLS. This is because it is a security problem for everyone. It can lead to downgrade attacks. Since 9 December you just see a warning. Soon the browser will not connect to the wikis at all. Most are users on Android systems older than 4.4. You can read the browser recommendations. [2]
• Special:LinkSearch has been moved from the "Redirecting special pages" section on Special:SpecialPages to the "Lists of pages" section. [3]

Changes later this week

• Wikis can protect pages so that only some users can edit them. The standard protection levels are Allow only autoconfirmed users and Allow only administrators. If your wiki use more protection levels the technical name might be renamed for standardisation. This doesn't affect what users see. [4]
• The new version of MediaWiki will be on test wikis and MediaWiki.org from 14 January. It will be on non-Wikipedia wikis and some Wikipedias from 15 January. It will be on all wikis from 16 January (calendar).

Future changes

• Deepcat and Catgraph will stop working. This will happen at the end of January. This is because you can now use the normal search function instead. [5]
• You can use <ref follow="…"> to merge footnotes that follow each other. It is meant to be used for digitised books on Wikisource. If the order of the footnotes is wrong no error was shown but the bad <ref> was shown outside the <references /> list. This will change and you will see an error message instead. [6]

Tech news prepared by Tech News writers and posted by bot • Contribute • Translate • Get help • Give feedback • Subscribe or unsubscribe.

MediaWiki message delivery (talk) 18:39, 13 January 2020 (UTC)

by= in {{Derived from}}

There is only one by= parameter in {{Derived from}} eventhough the listed files can have separate users. Can someone change so there is a by=, by2=, by3= ... parameter for each listed file?Jonteemil (talk) 10:00, 14 January 2020 (UTC)

Also noticed now that {{Own based}} has the "and" between the first and the second listed files instead of the last and second last as it should be. This would also be good to fix.Jonteemil (talk) 10:08, 14 January 2020 (UTC)

Flickr-to-Commons not working

Spinning Dancer

attn. @Tm: . Does someone knows if there is a problem with Flickr-to-Commons? In my case it has a litle icon of a dancer after "Make sure you authorise first! ⟳" , nothing else works and the "run " button is not showing. --JotaCartas (talk) 21:13, 14 January 2020 (UTC)

Note: also reported here --JotaCartas (talk) 01:33, 15 January 2020 (UTC)

It looks like the problem solved--JotaCartas (talk) 14:29, 15 January 2020 (UTC)

No thumbnails for File:Tissot behrmann.png

Thumbnail generation for File:Tissot behrmann.png generates mostly black images, with only a few lines at the top resembling the original image.

The image is used in multiple articles. The image is far from the limits I found documented, and the file type matches the extension. Who can look deeper into this issue? --Cat's paw (talk) 21:05, 15 January 2020 (UTC)

I get the same mostly black image if I load the file in Gimp, and "Error loading PNG file: IDAT: invalid distance too far back". I think there's something in the file which isn't handled as expected by certain PNG software. --ghouston (talk) 03:35, 16 January 2020 (UTC)

I ran "pngcrush -fix" on it and it seems to have helped. --ghouston (talk) 03:41, 16 January 2020 (UTC)

That file was uploaded 15 years ago! --ghouston (talk) 06:59, 16 January 2020 (UTC)

Thanks for solving this. My Gimp loaded the image without even the tiniest grumbling. -- Cat's paw (talk) 11:38, 16 January 2020 (UTC)

It probably depends on the version, or more likely the version of the PNG library that it's linked with. --ghouston (talk) 13:34, 16 January 2020 (UTC)

Tech News: 2020-04

Latest tech news from the Wikimedia technical community. Please tell other users about these changes. Not all changes will affect you. Translations are available.

Changes later this week

• The new version of MediaWiki will be on test wikis and MediaWiki.org from 21 January. It will be on non-Wikipedia wikis and some Wikipedias from 22 January. It will be on all wikis from 23 January (calendar).

Future changes

• There is a new suggestion for what to show when someone edits without registering an account. This is to give unregistered editors better privacy and make some anti-vandalism work go faster. You can give feedback.
• Pywikibot is a Python library to automate work on wikis. It will no longer support Python 2. Use the python2 tag if you need to continue running Python 2 scripts. The Pywikibot team strongly recommends to migrate to Python 3. You can get help to do so. [7]
• The weekly MediaWiki branch cut will soon become automated. The timing for this cut may change. You can discuss in Phabricator if this affects you. [8]
• You can read about coming technical events and mentoring interns.

Tech news prepared by Tech News writers and posted by bot • Contribute • Translate • Get help • Give feedback • Subscribe or unsubscribe.

19:40, 20 January 2020 (UTC)

WebM compatibility

I read that only some sub-formats of WebP are supported. Could anyone tell me if the format of (photos.app.goo.gl/HvL1iTbYjLNWbB8h6) this image is supported on Commons, and if not, what I need to do to change the image so it is supported? Thanks. datumizer ☎ 20:37, 20 January 2020 (UTC)

Sorry, the server would not take the URL of the link. You will need to copy and paste into your browser. datumizer ☎ 20:37, 20 January 2020 (UTC)

Help about Template:IMOcat

I added <includeonly>[[Category:IMO {{{1}}}]]</includeonly> to automatically categorizing ship categories, but then it appears in Category:IMO 7027411 because this ship is used as an example. Then I want something like "not categorizing in" template in Template:IMOcat/doc, but I can't find one effective (or just because the time lag). I need Template:IMOcat not to be in Category:IMO 7027411, please help.--そらみみ (talk) 11:54, 9 January 2020 (UTC)

• I think there's no problem now, because it has been reverted.--そらみみ (talk) 08:06, 11 January 2020 (UTC)
• I'm hoping that this can be deprecated in category space soon anyway, I'm working on expanding the display of ship info in {{Wikidata Infobox}} at the moment. Thanks. Mike Peel (talk) 08:48, 11 January 2020 (UTC)

• Thanks for your contribution. There are a lot of ship categories, so it can be a hard work. Good luck.--そらみみ (talk) 02:18, 23 January 2020 (UTC)

• @そらみみ: I think you might be able to do something with checking the namespace of the including page. Template:Geograph does this: {{#ifeq:{{NAMESPACENUMBER}}|6|...}} for instance. --bjh21 (talk) 19:19, 22 January 2020 (UTC)

• That's a good idea. But I dare it may influence so many pages.--そらみみ (talk) 02:18, 23 January 2020 (UTC)

Loop of error reports

I wanted to leave a message on a User page with a link to COM:FOP.
I wasn't able to save the userpage because "An automated filter has identified this edit as potentially unconstructive, and it has been disallowed. If this edit is constructive, please report this error."
I wanted to report the error and on https://commons.wikimedia.org/w/index.php?title=Commons:Abuse_filter/Error_reporting&withJS=MediaWiki:ABFeasySubmit.js and when I clicked on the button to submit the report I got the message "Error while crating the report: This action has been automatically identified as harmful, and therefore disallowed. If you believe your action was constructive, please inform an administrator of what you were trying to do." --193.171.152.104 13:14, 21 January 2020 (UTC)

It is filename that triggered the filter. Ankry (talk) 22:09, 22 January 2020 (UTC)

I don't think so. I was able to post the filename, but the link to Commons:Freedom of panorama was not allwed. I get the same warning here as well. not even <nowiki> helps. OK, I got it "Summary table" is a bad word combination. So I can not add a like to #Summary table to the link to FOP --193.171.152.104 14:41, 23 January 2020 (UTC)

Public database lag

Tracked in Phabricator
Task T232446

I believe the intent of the p_commonswiki database was to enable volunteers to report and analyse problems within as brief a time as was reasonable from the live database. I have been attempting to examine some characteristics of a deleted file from yesterday (24 January) and have realized that I can only see the filearchive table up to deletions on 22 January, a 3 day lag. I previously had thought that anything over a 1 day lag was a bug, but do not recall seeing this committed in a policy.

Could someone let me know if a 3 day lag is normal for database operations, or if there is a known performance issue? Thanks --Fæ (talk) 15:24, 25 January 2020 (UTC)

@Fæ: There was a big data compression job running, see phab:T232446 and phab:T243488. The lag seems to be decreasing now, but slowly. Thanks. Mike Peel (talk) 15:49, 25 January 2020 (UTC)

Great info, I only vaguely follow these issues. Do you happen to know if there is a normal/target maximum lag time that ops aim for? --Fæ (talk) 15:52, 25 January 2020 (UTC)

No idea, sorry. If you look at [9], things seem to turn red when there's a bit of a lag, but I don't know if that's a target or a pretty colour scheme. Thanks. Mike Peel (talk) 16:11, 25 January 2020 (UTC)

3 days would indeed be excessive. Currently lag for s4 (commons is part of s4) is like a couple of minutes. I dont know if there are any sla type things on replica lag, but it is very unusual when things are going normally for it to exceed an hour. Usually its a couple minutes at most. Bawolff (talk) 02:57, 28 January 2020 (UTC)

p.s. https://grafana.wikimedia.org/d/000000303/mysql-replication-lag?orgId=1&fullscreen&panelId=4&from=now-7d&to=now suggests there was an issue but it was fixed. It only seemed to effect one of the public replicas (edit: i guess it affected all of them, but one of the servers took a much longer time to recover), so it was probably possible to use a different replica to avoid the issue. Bawolff (talk) 03:03, 28 January 2020 (UTC)

Tech News: 2020-05

Latest tech news from the Wikimedia technical community. Please tell other users about these changes. Not all changes will affect you. Translations are available.

Problems

• Some mobile diffs have problems. A couple of buttons are not shown. Structured data diffs on Commons are confusing. The developers are working on fixing it. [10][11]
• Administrators on wikis that use Structured Discussions can't move discussion pages. This is a bug. The developers are working on fixing it. [12]

Changes later this week

• There is no new MediaWiki version this week.

Future changes

• There is JavaScript code on Special:Undelete for administrators that makes it possible to automatically select multiple checkboxes by holding the "Shift" key and clicking. This code is also loaded by accident on other special pages and on articles. This makes pages slower to load. This will be fixed. If you know of other special pages where this is useful please tell the developers at phab:T232688.

Tech news prepared by Tech News writers and posted by bot • Contribute • Translate • Get help • Give feedback • Subscribe or unsubscribe.

18:52, 27 January 2020 (UTC)

European Union

Why does that page say "The time allocated for running scripts has expired" a hundred times over??

Jcwf (talk) 20:05, 27 January 2020 (UTC)

@Mike Peel: European Union.--Roy17 (talk) 00:36, 28 January 2020 (UTC)

It's kinda weird. At 00:36, 28 January 2020 (UTC) I saw lots of lines as described by Jcwf. I tried removing the infobox and previewing, the lines disappeared, so it should be the infobox's problem, but some time later the lines disappeared. Last night they appeared again but numbered only a dozen. Now they disappear again.--Roy17 (talk) 03:28, 30 January 2020 (UTC)

Template:Retouched picture

{{Retouched picture}} taking a video as its value will be rendered weirdly, see special:permalink/390959947.--Roy17 (talk) 03:28, 30 January 2020 (UTC)

Actor tables

I have no idea where to give feedback on the WMF decision to add multiple "actor" tables for everything important to query about in our underpinning wiki database. My queries have been made to run two or four times longer than they used to. Writing basic queries is now a pain in the arse, as *everything* I used to have old scripts for must be re-written making the SQL harder to read and awkwardly JOINed to multiple actor tables when we need to report account names or comments, or log entries, all of which are now hidden behind uniquely different actor identities for every type of usage.

From an actual casual volunteer user perspective, it's offputting, it's hard to understand, and it makes me want to avoid writing new reports or even bothering to fix the broken ones. As an example my popular report User:Fæ/Userlist which five years ago took literally a few seconds to run, took 2 hours to run after the WMF decided to drop a few inconvenient tables, and today takes six hours and frequently fails to complete because of database time-outs, just because of the extra JOINs needed with actor tables.

Imagine me, leaning out of my window shouting Oh.My.God. It's just such a waste of time. --Fæ (talk) 14:05, 29 January 2020 (UTC)

So WMF removed the user_daily_contribs table because it stopped being updated in 2015. That seems reasonable enough, better to get rid of it then to have people assume its accurate. As for the actor schema refactor. I mean, if you're honestly curious about the rationale instead of just ranting, you can read about it at phab:T161671 (and links as info is very spread out). But very basically, it was determined that the size of the revision table was hindering scalablity of the sites, so the decision was made to split up some of the info into other tables, to make the big tables more compact. Actor was introduced so we could normalize the storage of the names of anonymous users, a major source of unnecessarily repetitive data in the revision table. Having a bit more complex schema, and mildly inconveniencing toolforge users, is in my opinion a reasonable trade-off to have a scalable site that is fast and responsive as we continue to grow and get bigger. Toolforge does have _compat tables that can be used to make old queries continue to work, but for best performance your queries need to be rewritten to work best with the new schema. Bawolff (talk) 05:55, 30 January 2020 (UTC)

I wanted to try to prove a point and write a query and say, see its not so bad, but after trying I must conclude that the redaction views on toolforge really do make writing certain types of queries cumbersome. The fastest query I could come up with was select user_name, user_editcount,(SELECT revactor_timestamp from revision_actor_temp inner join actor_revision on revactor_actor = actor_id WHERE actor_name = user_name order by revactor_timestamp desc LIMIT 1) 'last edit', groups, reg from (select user_name, user_editcount, GROUP_CONCAT( ug1.ug_group SEPARATOR ", " ) "groups", substr(user_registration, 1, 4) "reg" from user left join user_groups ug1 on ug1.ug_user = user_id left join user_groups ug2 on ug2.ug_user = user_id and ug2.ug_group = 'bot' where user_editcount>= 10000 and ug2.ug_user is null group by 1,2,4) t order by user_editcount desc;. I'd be curious if that was faster than yours. I suspect its slow, but not 6 hours slow. Bawolff (talk) 08:27, 30 January 2020 (UTC)

It seems largely a problem with the mariadb optimizer. Running the query without the (SELECT revactor_timestamp from revision_actor_temp inner join actor on revactor_actor = actor_id WHERE actor_name = user_name order by revactor_timestamp desc LIMIT 1) subquery is fast, and running that subquery separately for each result is also fast. Putting the two together is slow. So if you want to do this fast, you can first do the general query without the last edit part. Then in a loop, do a separate query for the last edit of each individual user returned. Bawolff (talk) 09:13, 30 January 2020 (UTC)

So part of the issue seems to be how redaction is done on the actor table. However I don't think the current method is very good, and I'm going to try and talk some folks into using a different method which should be more efficient [Edit: I misunderstood something. Still going to try and see what can be done better, but my initial idea was a bust]. Bawolff (talk) 03:30, 31 January 2020 (UTC)

btw, this is my current most optimized version - https://quarry.wmflabs.org/query/41786 (probably still too slow to complete in quary though). Seems like the big part of the problem, is mysql optimizer will look at all a user's edit to determine what their latest edit is, instead of just their most recent edit (Unless you are looking up a single user, then everything works sanely). So having the query look for only the most recent edits in recent edits, and if that turns up nothing, then look at all edits, tends to be a bit faster. Bawolff (talk) 20:15, 31 January 2020 (UTC)

Just FYI, my revised query ran in 2 hours 4 min 31.38 sec (2932 rows in set). Although i didn't realize your list was only people active in last 30 days, so not directly comparable, although probably limiting it to such people would make it run faster not slower. Bawolff (talk) 21:25, 31 January 2020 (UTC)

@Fæ: Ok, so final result - https://quarry.wmflabs.org/query/41803 runs in 77.5 seconds. It still kind of proves your point though, as it took me 2 days to get the query this optimized. Bawolff (talk) 04:41, 1 February 2020 (UTC)

Thanks for the analysis, I was not expecting it. The result is impressively fast. Don't delete that quarry report! My report is not currently broken, so pulling in your solution is something that I'll look at in slow time, for me that might be in a couple of months. If you have more inspiration later, it would be good to add pointers to the quarry report so I don't lose track of them. I'm not shouting out of the study window today, so that's an improvement. --Fæ (talk) 12:14, 1 February 2020 (UTC)

Wrong labels translation

The File:Antonino Cannavacciuolo-signature.svg have some translation errors in the labels.

In the Details section, “SVG sviluppo” should be “Sviluppo SVG”, and “Questa firma è stata creata con un ignoto SVG programma.” should be “Questa firma è stata creata con un programma SVG ignoto.”. How can I fix? --2001:B07:6442:8903:A57E:C924:88B7:B2BA 11:38, 24 January 2020 (UTC)

The second one seems to be generated by Template:I18n/othertool (found with Special:Search/template: insource:"ignoto SVG programma"), so you can (with care) edit that template to correct it. I think (but am far from sure) that Template:Image generation always insists on putting the file type before the word for "development": {{uc:{{#invoke:File|extension|file={{PAGENAME}}}}}}{{I18n/COA|development}}, so correcting “SVG sviluppo” would involve more complicated template editing. --bjh21 (talk) 16:30, 5 February 2020 (UTC)

Can’t salt

For some reason, admins (or me) are currently unable to salt anything in mainspace, aka the Gallery namespace. I’m not sure if it’s done intentionally or an error. Previously it was able to be done. 1989 (talk) 17:22, 31 January 2020 (UTC)

What’s even weirder is the logs are saying I deleted Special:Categories when I actually deleted Categories. [13] 1989 (talk) 17:33, 31 January 2020 (UTC)

There's a patch for the second issue phab:T196669; it's not yet merged. For the first issue, I think there's a phab task, but I could not find it. – Ammarpad (talk) 07:15, 8 February 2020 (UTC)