Bot's tasks for which permission is being sought:
Upload of public domain images of artwork and related metadata, using the Template:Artwork of artwork provided through partnerships with Danish GLAMS.
For the SMK contribution, the images are downloaded and metadata is mapped to the Artwork template, utilising the Statens Museum for Kunst, SMK API: entry KMS1 and the wikilabskultur Artwork template preprocessor.
Automatic or manually assisted:
The bot is manually assisted and will most likely be running from a stand alone computer
Edit type(e.g. Continuous, daily, one time run): One time run (several batches)
When:
The bot operates at specified intervals (batches). Activity depends on the availability of new batches from GLAMs, it can be started on demand by the local operator, on a stand alone pc
Maximum edit rate (e.g. edits per minute):
Most likely 10-50 edits per minute, but only in short bursts, and it can be set to a much lower rate. The first proposed upload will be in the form of approx. 500 files of 10-30MB in size, time to finish is not of the essence, the total size of the contribution from SMK currently is approx. 70.000 images, the collection is expected to be quite static.
Bot flag requested:(Y/N):
Y
Programming language(s):
The bot is written in Python using the pywikibot library
Could medium be internationalized with template or Wikidata item? If not, please use language tag. Same for object type like Blyant. Could bot add painting by or more detailed category? Please also make batch categories hidden. --EugeneZelenko (talk) 16:04, 20 December 2021 (UTC)[reply]
This file should have a default sortkey (Defaultsort) "Købke, Christen, 1838" as we generally sort items by the creator's last name.
It should have a category of "Paintings by Christen Købke in Statens Museum for Kunst" with a local sortkey of "|1838" (after the category name). I did that manually.
It should have a category of "Paintings by Christen Købke" with a local sortkey of "|1838" (after the category name). Is this not redundant? No, because those paintings will eventually be divided up as landscapes, marine paintings, portraits and so on.
It might have a category of "19th-century paintings in the Statens Museum for Kunst". Note: Here it is the Statens Museum for Kunst. Don't ask.
It might have a category of "Landscape paintings in the Statens Museum for Kunst".
It might have a category of "1838 paintings from Denmark".
Thank you for your comment, and the detailed example, it's very helpful, I can definitely improve on the categorisation along the lines you suggested WLKBot (talk) 21:00, 22 December 2021 (UTC)[reply]
@Kim Bach: please don't use your bot account to make manual edits like the previous comment.
Please wrap the medium field in {{Da}} to make clear that this is in Danish. Makes it easier to later replace in internationalization.
I don't care that much about the categorization as long a you add some object type categories it should be fine.
Including the copyright and license template makes {{Artwork}} really big. Please create two sections example and put the permission part in the bottom section.
Please only use {{Creator}} if the template exists (or you're creating it) because otherwise we'll end up with broken links for years
Not sure if you plan to upload some 3D works, but in that case you can use {{Licensed-PD}}
@Krd I assume that the "?" means that you'd like an update :-). We're busy implementing the changes suggested by the community, and we will not run another test until we're confident in the changes we're making, this should only be weeks away. Kim Bach (talk) 13:59, 6 February 2022 (UTC)[reply]
Wraps the medium field in {{Da}}, will move to Wikidata items in test and production runs
We'll be adding structured data using QuickStatements after we've uploaded the images, we will add the Wikidata-item to the Artwork template when we have created the Wikidata-items or identified already existing Wikidata-items
We've created two sections for the copyright and permissions
We're no longer using Creator template, we've switched to using Wikidata items, and will not upload images that has no Wikidata item for creators. The idea is to add the ones that are missing to Wikidata as we go along, using QuickStatements
We've added ...by... categories, but we think that most of the might have suggestions by Rsteen are better handled through structured data, which we'll be adding using QuickStatements
Below is an example of a file that was uploaded manually, where the Wikitext was generated by our updated code, and used to improve the code, we'll perform a few more of these before doing a test run, but we feel like we're ready
Giovanni Battista Piranesi, Det indre af Pantheon, 1768, KKSgb9860-86, Statens Museum for Kunst
The intention is to resume testing, firstly by doing some more manual uploads, in the context of Kim Bach, in preparation of new test runs.
Pending approval we'll move slowly to production runs, the plan is to do that one creator at a time.
@Krd We've made a test run of 20 new media files. We're of course referencing a number of new categories, we should probably create those as well. Maybe as we go along, maybe at a later point in time.
We're referencing a number of object types that aren't created yet. It looks like this is done in LUA code. I do have all the Q item numbers of the object types we're using on hand, but we're also planning on adding structure data using QuickStatemenst, we're currently testing that.
Thank you Multichill, I've incorporated your suggestions and I'll be omitting the "other fields" in the future, they were meant for internal bookkeeping (created and updated timestamps of metadata from SMK)
I've updated the wikitext for this image, trying to take your and Eugene's suggestions into account Tobias Stimmer, Romerne indtager Satricum, 1574, KKSgb22345, Statens Museum for KunstKim Bach (talk) 22:22, 6 December 2022 (UTC)[reply]
Thank you EugeneZelenko, I can address your comments in this way:
Internationalisation of object type
I suppose you mean that I should use this:
object type={{en|Woodcut print}}
Excessive white space
I can beautify the wikitext along these guidelines:
Thanks, I’ll go for that, is at ends with the suggestion by Eugene (space after pipe, I guess that’s not to importerne.
Another possible issue is the new lines. They flush left, and it doesn”t look good. Is there a way to add indentation that doesn’t show up in the rendered wiki page, I suppose the List templates could be used for that purpose. Kim Bach (talk) 14:40, 8 December 2022 (UTC)[reply]
I've made changes that tries to accommodate your standard usage suggestion - this is a "bit" confusing because the Usage sample and the Multilingual sample differs :-/ - and the suggestion with spaces trailing pipes by EugeneZelenko. I've updated the wikitext for the Tobias Stimmer image to reflect the suggested changes :-) Kim Bach (talk) 11:09, 9 December 2022 (UTC)[reply]
The reason we have two titles for this particular item, is that the museum DB can have several titles for an item.
The best thing to do, might be to limit it to one - since one of the titles is considered official by the museum. We could use that, and then, at a later point in time, add more titles to wikidata, we could also change it to one line, like you suggested.
Using wikidata
Wikidata, works fine for the medium tag wrapped in the Technique template, we can do that, since we've already mapped the Q-numbers.
Is it possible to use Q-numbers for the object type too?
Thank you, yes, that was a general error in the code from last year, everything was a factor 10 off, as you noticed, we've fixed that since. I forgot to fix this manually in the batch from last year, so I'll do that. Kim Bach (talk) 15:57, 21 December 2022 (UTC)[reply]
An updated status. We're currently refactoring the category generating code, this will most likely not be ready until january 2023. We still welcome comments to the last test run. Kim Bach (talk) 16:10, 21 December 2022 (UTC)[reply]
We've refactored the code, and are now ready to perform a new test. The main change is that we now try to create new category pages, if they don't exist Kim Bach (talk) 21:31, 11 January 2023 (UTC)[reply]
Indeed, as you might have noticed from the discussion above, what is missing is a (correct) mapping of the "object type" field.
We've been in discussion with the maintainers of the artwork template, unfortunately the field is not supporting Wikidata Q-items directly, so we'll have to add these to the mapping table, all are P31-tems.
Once we've done that - a few hundered - we're ready for at new test run.
I've drafted a version of the object type data table, where I've added the object types I was missing, you can find it here User:Kim_Bach/sandbox/object_type_data. I've asked the maintainers of the Artwork template to review it.
Thank you, I had some good suggestions from the community and I will work along those lines. I’ll post an update when the changes are made. Kim Bach (talk) 10:00, 7 April 2023 (UTC)[reply]
Thank you, and that is understandable, it has been a long process, and I have been very cautious. I just had help from MGA73 and I'm compiling a new list, that isn't such a mess when trying to apply a diff. Kim Bach (talk) 12:44, 10 April 2023 (UTC)[reply]
Test of 2023-04-27
There was an issue with the "object type" property, I've corrected the error, and updated the new images, so they should reflect the changes in the code--Kim Bach (talk) 07:21, 28 April 2023 (UTC)[reply]
If you assign categories automatically, you need to follow the general naming convention for categories more closely. Take File:Tobias Stimmer, Gaius Duillius sejrer i et søslag over kartaginienserne, 1574, KKSgb22331, Statens Museum for Kunst.jpg. It is categorized as Woodcut prints by Tobias Stimmer in the Statens Museum for Kunst. The normal category would be Woodcuts by Tobias Stimmer in the Statens Museum for Kunst. It is also categorized as Graphics by Tobias Stimmer in the Statens Museum for Kunst. That category is not necessary and represents overcategorization. The category Woodcuts by Tobias Stimmer in the Statens Museum for Kunst should lead to Woodcuts in the Statens Museum for Kunst, which again should lead to Prints in the Statens Museum for Kunst. Try and take a look at how it is done at the Rijksmuseum, which is more or less the gold standard. Take Category:Prints in the Rijksmuseum Amsterdam. Here all the different types of prints are gathered and you can go to subcategories by artist, by century and so on. We should attempt something similar. Cheers Rsteen (talk) 11:51, 1 May 2023 (UTC)[reply]
Thank you. @Rsteen: What we need to do, is to look much closer at the category mapping, as you suggested, it might not be that diffocult. Do you understand JSON format? Kim Bach (talk) 04:01, 2 May 2023 (UTC)[reply]
Absolutely. And thank you for your work. There is a goldmine of artworks at Statens Museum for Kunst. Cheers Rsteen (talk) 05:05, 2 May 2023 (UTC)[reply]
So...The strategy would be to create categories similar to the ones the Rijksmuseum Amsterdam has created, and then create a mapping table between those categories and the data returned by SMKAPI. Taking into consideration that some existing categories are not using the naming convention. I'll propose something.
Statens Museum for Kunst uses a total of 128 unique categories, Rijksmuseum Amsterdam has 38 main categories in their navigation box. I will work on some kind of consolidation, like the 38 categories in the navigation box. I'm sure the art historians at SMK can be helpful in grouping the categories. Kim Bach (talk) 11:50, 3 May 2023 (UTC)[reply]
Thank you for your kind assistance, I can see that I'm creating a lot of work for you. I'll be more careful before updating in production in the future. Kim Bach (talk) 21:36, 5 May 2023 (UTC)[reply]
I've been working with User:Rsteen on the categorisation, and we've come to an agreement on how to proceed.
So I hereby request permission to perform a new test upload. Note that the new version of the bot will create artist and object related categories on the fly, if they're missing. --Kim Bach (talk) 14:21, 2 June 2023 (UTC)[reply]
@Rsteen: I've noticed that you're changing "Collections of the Statens Museum for Kunst" to "in the Statens Museum for Kunst by artist", I guess I should update the code to reflect that, I must admit that I'm not that good at LUA that I understand this short hand Kim Bach (talk) 07:17, 18 August 2023 (UTC)[reply]
Hi Kim. Yeah, but that is not a big problem. I think you should make the full run. After that, categories can be streamlined if needed. Cheers Rsteen (talk) 16:43, 18 August 2023 (UTC)[reply]
Thanks, pending further feedback, I'll start running batches of say 100 items, since there are a few calls to wikidata and commons APIs, stepping it up if no major issues are detected Kim Bach (talk) 08:33, 19 August 2023 (UTC)[reply]
I've noticed that WLKBot has received "Bot status", is that to be considered an approval, or is it still pending an official approval process? Thank you! Kim Bach (talk) 11:26, 25 August 2023 (UTC)[reply]
Bot status has been applied to keep the extended testing out of recent changes and watchlists. I think approval is still pending, and can be applied if all relevant questions have been addressed. I have to admit that I don't understand if that is the case. Please feel free to summarise. Krd12:58, 1 September 2023 (UTC)[reply]
I'm of the opinion that all questions has been addressed, as well as can be expected,
_-_Charles_Hodder_(1835%E2%80%931926),_Artist,_Self_Portrait_-_PG_1441_-_National_Galleries_of_Scotland.jpg&action=edit){kind=link}
