Commons talk:Batch uploading/TheNounProject

Checking on a bot request

Hi

I was just checking on a bot request I made earlier in the year for icons from The Noun Project and I found Commons:Batch uploading/TheNounProject. Can I help you in any way to make the bot upload easier? If you're not able to do it then do you have any suggestions of other users I could ask?

Thanks

John Cummings (talk) 11:17, 30 November 2017 (UTC)

Hello @John Cummings: , thanks for reminding me. I'm really sorry for forgetting about this task. I just run few test with icons from this collection. These icons are under cc-zero licence. I haven't figured out yet how to use icons under CC-by-sa license as the API seems to have some limits about it learned from Thenounproject, while reporting the impossibility to import icons under cc-by-sa license, that a paid API key is needed to access all icons. For now I suggest we take a look at the small tests run by the bot and adjust things such as categories, templates to use etc. African Hope (talk) 04:15, 4 December 2017 (UTC)

Thanks very much African Hope, no need to apologise :). Looking at this page it seems like their is a monthly limit to 5000 items accessed through the API for the free API key]. Yes a small test run sounds great, please let me know where I can help. Thanks again. 12:34, 7 December 2017 (UTC)John Cummings (talk)

Indeed John, the free API allows to access 5000 items monthly, but as mentioned explained by Thenounproject twitter account, does't grant access to cc-by-sa icons, only icons under cc-zero. You can help by taking a look at Special:Contributions/CivBot and check if the icons imported by the bot have the proper fields and tags (category, description, license, template, etc). African Hope (talk) 12:49, 7 December 2017 (UTC)

African Hope, very glad to know you got at least some of it working. The only issue I can see is there is a broken link in the documentation for the page, meaning it gets added to Category:Files with broken file links. Maybe you could add them to a subcategory called something like 'SVG icons from The Noun Project needing additional categorisation'? It would be really nice to be able to categorise by subject. Do you know how much it would cost to get the API to where we could just get everything?

Thanks very much

John Cummings (talk) 08:56, 14 December 2017 (UTC)

Hello @John Cummings: , I edited the Template:The Noun Project to include automatically the following categories: SVG icons from The Noun Project needing additional categorisation and SVG icons from The Noun Project. Regarding the API cost, Thenounproject offers plans starting at 150 USD/ month to get all icons (see their plans page. African Hope (talk) 18:08, 14 December 2017 (UTC)

African Hope, super, do you think that it would be possible to grab all the images needed in one month? If so I could ask around the chapters. John Cummings (talk) 19:06, 14 December 2017 (UTC)

John Cummings, just how many icons are needed? African Hope (talk) 19:35, 14 December 2017 (UTC)

African Hope, could we take all of them? The problem is filtering similar designs would require going through and making choices, there are around 1 million files so that is quite a lot... Do you know how many CC0 images there are? John Cummings (talk) 19:53, 14 December 2017 (UTC)

John Cummings, with the free API limits, it's quite hard to guess the number of CC-0 icons available on thenounproject. However there are ways to filter icons either by term of by license. I can't tell exactlyhow many images we can import with the paid license but that's a minimum of 5000 icons provided. For paid licenses, thenounproject asks to contact them directly via infothenounproject.com. It might be a good occasion to ask about the number of icons we will be allowed to import and so on. African Hope (talk) 21:29, 14 December 2017 (UTC)

Thanks African Hope, looking at this page there seem to be both CC0 and PD icons, I guess both of these can be downloaded using the free API key. I've just found this page which gives guidance on how they would like the items to be credited. Maybe do the first 5000 and then we review? I'll do some more research in the mean time. Looking at the tags it appears that it wouldn't map exactly for Commons since it would mean that icons were in both a broad category and then a specific subcategory e.g potato and food, or would that be OK? John Cummings (talk) 21:50, 14 December 2017 (UTC)

John Cummings I am okay with the idea to start with the 5000. What do you mean by it wouldn't map exactly for Commons[...], I can't get it. African Hope (talk) 12:12, 15 December 2017 (UTC)

African Hope, I'm sorry not to be clear. If you take the example of this image of a potato and click on the little tag icon you can see the tags for it are:

• Potato
• Food
• Potatoes
• Vegetable
• Crude Potato

Whilst some of these would be useful for categorisation on Commons I guess that Food is a bit vague and Crude Potato probably isn't a category. What do you think?

Thanks

John Cummings (talk) 13:04, 15 December 2017 (UTC)

Ah I see, John Cummings. Indeed, we can't rely on the original tags of icons to map them with Commons categories since, how you correctly mentioned, these tags are sometimes too vague. However, the category you suggested and that I created SVG icons from The Noun Project needing additional categorization, might serve as a temporary category until humans can manually add the appropriate categories. African Hope (talk) 15:38, 15 December 2017 (UTC)

Sounds good :), John Cummings (talk) 16:03, 15 December 2017 (UTC)

Agreed then. I'll try import much more items this weekend. Any additional feedback from the community will be welcomed. I asked some opinion here but received no feedback so far. African Hope (talk) 16:08, 15 December 2017 (UTC)

👍 John Cummings (talk) 08:29, 16 December 2017 (UTC)

Imported 500 icons

Hi John Cummings, I just imported 500 icons. Will take a break and wait for any remark or warning from the community before proceeding with more imports. However, the free license API is such a pain: you can't import more than 50 icons from a term or from a collection so I try terms and collections randomly. Some of the terms or collections don't have public license icons. I guess thenounproject did that on purpose in order to push people towards purchasing a paid license as this is their business model. African Hope (talk) 00:53, 19 December 2017 (UTC)

African Hope, it looks great, thanks very much, I think everything looks good. My only other thought is perhaps we could use the name of the file as the category? The only issue I can see is where the icon name is singular but the category name is plural, e.g icon is called Potato, but the category name is Potatoes, but there will be a category redirect in that case. I will see about getting some money for a month of paid API access. John Cummings (talk) 10:50, 19 December 2017 (UTC)

I agree over the necessity of categorizing these icons. The issue with titles is that they are so unique, it will rarely map more than one icon. For example if you look at the latest 500 imported pics (Category:SVG icons from The Noun Project needing additional categorisation), titles are unique. Maybe terms instead of titles would be a better fit as they are relatively broad. African Hope (talk) 14:19, 19 December 2017 (UTC)

@African Hope: , yes this sounds like a sensible approach, let me know if I can help. Thanks, John Cummings (talk) 09:19, 10 January 2018 (UTC)

Hey @John Cummings: , it's been a busy period. I hope to start bulk imports again very soon. I ran into few issues while uploading. I noticed that sometimes icons at thenounproject.com have duplicate names so the bot gets confused and ends up uploading many icons as new versions of a same icon: example here. I will work on that and let you know. Best, African Hope (talk) 12:30, 6 February 2018 (UTC)

Any work is greatly appreciated, no rush :) Let me know if I can be of help doing manual labour. John Cummings (talk) 18:53, 12 February 2018 (UTC)

Issue with duplicate names

Hello @John Cummings: ,

A recent message from @Ham II: regarding a set of icons to upload made me consider a solution regarding the issue I am facing with duplicate names of icons. On thenounproject, several icons share the same name. For example you will find dozens of icons called "ceiling" (eg: ceiling. This poses a problem. The solution I am thinking about is to import icons while appending to their original title their unique id number. For example the icon above would be imported to Commons as "Ceiling (1404505) - The Noun Project.svg". But I am wondering whether the title would be weird or not. Do you think that this workaround could be a good tradeoff @John Cummings and Ham II: ? African Hope (talk) 20:31, 6 March 2018 (UTC)

Hi @African Hope:  :), I think this will work great, I know that a similar solution is used for Flickr photos because of the duplicate names. Thanks very much, John Cummings (talk) 20:59, 6 March 2018 (UTC)

I agree with John; see, e.g., this example from Flickr. Ham II (talk) 05:37, 7 March 2018 (UTC)

@African Hope: I've manually uploaded a few CC-BY Noun Project icons to Commons using the following, simplified naming style: File:Ceiling (NP1283257).png, File:Moai (NP56677).png, File:Moai (NP1581836).png. (Unfortunately they're PNGs as I don't know how to crop SVGs.) The simpler the style the easier it would be to harmonise the titles of manually uploaded and batch-uploaded files, I think. Ham II (talk) 09:36, 18 March 2018 (UTC)

@Ham II: , Yes it would be good to harmonise the titles. A tool is currently being requested from volunteer developers to ease the upload process and add relevant details to icons. However you did great so far with the icons you just uploaded. African Hope (talk) 09:44, 18 March 2018 (UTC)

Ok @John Cummings: , thank you for the explanation. African Hope (talk) 00:44, 7 March 2018 (UTC)

1000 more icons imported

@John Cummings: , following your advice, I adjusted the bot task and could upload 1000 more icons under cc-0 license. Will try more in the upcoming days. African Hope (talk) 03:49, 7 March 2018 (UTC)

@African Hope: 👍👍👍👍👍👍👍👍👍 John Cummings (talk) 06:28, 7 March 2018 (UTC)

undeleted category:SVG icons from The Noun Project needing additional categorisation and added info

It had been deleted so I explained the situation there. Also, may I request https://thenounproject.com/edward/uploads please? Thank you. Eventually it would be nice to have it set up like Flickr Upload Bot where there is a WMFlabs page to just auth with one's account and send the bot an identifier and have it do the work quasi-automatically. Arlo James Barnes 19:55, 30 September 2019 (UTC)