User talk:Bdcousineau/Archives 1

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search

Unfortunately, the licence is not useful -- the Gerald Ford presidential library has copyright on the photo, but does not have copyright on the design on the bottle. In any case, it is not so clear that the Gerald Ford presidential library is the same as the U.S. federal government. AnonMoos (talk) 13:41, 7 August 2012 (UTC)[reply]

dansk  italiano  sicilianu  Deutsch  català  magyar  čeština  português do Brasil  Esperanto  español  português  English  hrvatski  français  Nederlands  Deutsch (Sie-Form)‎  norsk nynorsk  polski  galego  íslenska  slovenščina  suomi  svenska  Türkçe  Ελληνικά  беларуская (тарашкевіца)‎  български  македонски  русский  українська  മലയാളം  日本語  中文(简体)‎  中文(繁體)‎  فارسی  +/−


There seems to be a problem regarding the description and/or licensing of this particular file. It has been found that you've added in the image's description only a Template that's not a license and although it provides useful information about the image, it's not a valid license. Could you please resolve this problem, adding the license in the image linked above? You can edit the description page and change the text. Uploading a new version of the file does not change the description of the file. This page may give you more hints on which license to choose. Thank you.

This message was added automatically by Nikbot, if you need some help about it please read the text above again and follow the links in it, if you still need help ask at the ? Commons:Help desk in any language you like to use. --Nikbot 13:57, 7 August 2012 (UTC)

Only thing missing

[edit]

Is a "whip inflation now" button... AnonMoos (talk) 06:15, 21 August 2012 (UTC)[reply]

I'll be adding that soon! we'll have a whole section on that.....Thanks for paying attention Bdcousineau (talk) 13:40, 21 August 2012 (UTC)[reply]

GLAM

[edit]

I saw your listing at Commons:Batch_uploading#Gerald_R._Ford_Presidential_Library_and_Museum. Have you gotten in touch with anyone from GLAM yet?Smallman12q (talk) 20:29, 15 September 2012 (UTC)[reply]

Oh! great! are there other people I should connect with? I finally manged to get in touch with user:Dominic who is the NARA-wikipedian (The Ford Museum is a branch of NARA) so I'm think I'm all set re: Commons:Batch_uploading#Gerald_R._Ford_Presidential_Library_and_Museum. I've also been in touch sporadically with Lori P via email regarding GLAM. And user:Jameslwoodward has agreed to be my admin helper on most other tech issues. You can see on his user page the looooong convo we are in the middle of. Feel free to jump in and help steer however! I'm building our presence at Commons:Gerald R. Ford Presidential Library and Museum Bdcousineau (talk) 12:56, 17 September 2012 (UTC)[reply]
Not really, those are the main people to get in touch with for now. I saw your discussion at User talk:Jameslwoodward#Ford Museum. Are you planning on doing a supervised batch upload (or is someone going to write a bot to do uploads)? I've looked at the site and there doesn't seem to be a centralized gallery/database. While you're file-names are non-specific, a lot of them do have have specific anchor tags which could be parsed to give a viable file name (I've done several batch uploads like this). Galleries, such as the Artifact Collection which have a consistent layout can be automatically uploaded. You're relatively new here, so if you have any questions, feel free to ask on my talk page. Cheers.Smallman12q (talk) 23:25, 17 September 2012 (UTC)[reply]
yes, working thru many issues, the main one being my newbie-ness. There is a bot for the batch uploads I'll have access to through NARA. I've already uploaded Artifact Collectionas I created the webpages. I'm working on sorting/categorizing the image files from NARA that can go into the Commons:Gerald R. Ford Presidential Library and Museum presence. Thanks for your input.... Bdcousineau (talk) 18:21, 18 September 2012 (UTC)[reply]
Newbies are always welcome=D. Only part of the [artifact collection pages has been uploaded. Whip Inflation Now, for example has 24 images, while Category:"Whip Inflation Now" collections at the Gerald R. Ford Presidential Museum has 16. Compared to the NARA upload (such as File:Bill_of_Rights.jpg) it seems like your uploads lack templates and details such as {{NARA-cooperation}} and {{NARA-image}}. Do you need such templates (if so, I can help create them based on NARAs)? For the bot upload, who is doing the uploads (or this hasn't been decided)? Which materials do you plan on uploading...just images/video or the pdfs too? Also, does the Gerald R. Ford Presidential Library and Museum have an official acronym? Smallman12q (talk) 00:05, 19 September 2012 (UTC)[reply]

Sounds like you could really help me out!

yes, the templates are missing. I need them yes! PLEASE! In my convo with user:Jameslwoodward he mentioned that they are complex to build and that scared me, so I haven't blocked out the time to try to figure it out. I would love a template like {{NARA-cooperation}} that uses the Ford Library/Museum logo - (there is a logo located on Category:Gerald R. Ford Presidential Library and Museum)and also uses the langauge of the NARA one, but uses our full name in place of theirs. also, I'd like {{NARA-image}}and an instituion-template - again haven't had the time yet, still hashing out category names.

I have the coding from NARA for the bot-upload, again wanted to figure out cats first. But can supply bot string if you feel like helping with that too!!

I want to upload EVERYTHING, that's the point isn't it?

Most of our artifact collection is not yet photographed, I am uploading as fast as they are photgraphing (which means slooowllyyy). I think the discrepancy in the WIN category has to do with trademarked material, I didn;t want to put things on Commons that might get flagged for deletion. It's a sticky question about public domain DONATIONS (as opposed to federally created documents and photographs). We are discussing more precise liscensing language with NARA right now, which means in 5yrs we'll get some where!

acronym for the templates, etc: Ford Library Museum? I'm a big fan of Comm:Walters and modeling our presence on theirs as much as possible. and they use Template:Walters Art Museum as a name.

one last thing: can you tell me how to fix the left justification on Commons:Gerald R. Ford Presidential Library and Museum - it seems to be center-justified and I can't locate the code in the Comm:Walters or Comm:NARA that will let me fix that. It looks dorky right now.

happy to add you as protagonist for tech help too if you like. whew! Thanks Bdcousineau (talk) 13:36, 19 September 2012 (UTC)[reply]


I've fixed the tab header at Commons:Gerald R. Ford Presidential Library and Museum (some links to NARA should be fixed). For the acronym, I used COM:GFPLM...if you have a more official one, please let me know. I also created {{Gerald R. Ford Presidential Library and Museum-cooperation}}. The template is based on the NARA one (creating a template from scratch without templated-code requires more time). I'll create a {{NARA-image}} one also. You should get the licensing ironed out first (for legal reasons). I understand you have the code for the NARA-bot, but I'm not sure how applicable it is to your upload unless your databases match. Do you have a centralized database with all the images/pdfs and metadata...or is it in different directories on the website? You mention you're a web designer (css/graphics/html/layout/fonts)...are you also a web developer(code php/javascript/asp/cgi)?

You mention most of the artifact collection is still being photographed....generally, large batch uploads are done once with all the media. How is are the photographs being cataloged...if there is a central db to which they are added, then they can be automatically uploaded on addition. You shouldn't have to upload anything manually, unless it has an unusual format that isn't consistent or can't be parsed....which shouldn't happen. This isn't too long...on the wiki, things are done through lengthy discussions=P (see w:WP:VPP for example).If you have any questions, feel free to ask.Smallman12q (talk) 22:17, 19 September 2012 (UTC)[reply]

this so rocks! exaclty what I was thinking - giving NARA credit on the template but making it clear we are the physical repository. Ps love the COM:GFPLM
I'm actually the Museum's exhibit designer with rudimentary self-taught web design skills, NO tech developer skills, so dumb it down for me, please! I know enough html to do what I've done so far... I have a team-member in this project who is terrified so far; she's done some blogging. My bosses are totally on board with this so I can spend major time on it - in between other job-duties. So far they are pretty impressed.
There are some messy cats on the COM:GFPLM right now, I will organize them on Friday. some empty one will at that point need deletion.
I'm on my way to an wikipedia workshop all day tomorrow/Thursday so will be traveling/offline. Will resume Friday and answer your above comments more thoroughly then. again thanks Bdcousineau (talk) 00:46, 20 September 2012 (UTC)[reply]

So a few comments...it seems like you guys aren't using a centralized database, but rather a bunch of directories since your search is based on dtSearch (which is an awesome spider). What this means is that uploading will be a bit more difficult as there isn't a standard, central place to get all the info. You haven't stated who is going to code the upload?

There is a {{NARA-image-full}}...do you want something like that but "re-branded"? See File:Gun crews of a Navy cruiser.jpg for sample use.

You don't seem to be using any gadgets for your categorizing. Gadgets are like widgets, they're software built into the web page that help automate tasks/make them easier. You should enable HotCat and "Cat-a-lot" under the "Tools for categories" in Special:Preferences#mw-prefsection-gadgets. Cat is short for categories.

If you have any questions, feel free to ask.00:04, 21 September 2012 (UTC)

centralized database - much of our digitized material is also stored on ARC(NARA's centralized database)Arc catalog you can do a people search, Gerald R. Ford and get all the relevant materials in ARC. Here is the coding string used by user:Dominic to upload NARA's holding to commons. I have no idea how or if this works NARA upload script or User:US National Archives bot/script have fun? with it
{{NARA-image-full}} yes please, re-branded, too!
how will your lovely templates and etc. get auto-magically inserted into the image files I've already uploaded?
I'm cat-ing by hand and it's slow and boring. and messy. I added Cat-a-lot and HotCat. Thanks. This will help, once I figure them out.
thanks for adjusting the left justification.
this really is like chewing on an iceberg. Bdcousineau (talk) 15:38, 21 September 2012 (UTC)[reply]
oh ps. There are 2500+ images made by the Military District of Washington (MDW)of Ford's State funeral that I'd like to get up onto Commons.. we have them on disk, I will look for the digital source. Bdcousineau (talk) 16:40, 21 September 2012 (UTC)[reply]

Arbitrary break 1

[edit]

I looked at the Arc catalog and it seems the images you uploaded don't show up. Can you give an example link for something that does show up? (You should be able to search by id for it to be properly linked from the wiki). Also, for some of the images you uploaded such as File:President Ford examines a wood portrait.jpg, a higher-resolution version is available on your site.

I'll create the rebranded {{NARA-image-full}} this weekend hopefully.

I see you used HotCat...it adds a +/- next to the category names at the bottom. Cat-a-lot gives you a yellow tab at the bottom right hand of your screen when you are at a category page such as Category:Gerald R. Ford Presidential Library and Museum. To use cat-a-lot, go to a category page, first click on the yellow tab, and click on the white-space next to a file's naae or size (such as 1.5MB) to select the file and will give a green box. You can select several files. After selecting the files, type the category you want to add and press enter. Next to the category name will be two options: move (remove the current category and add a new one) or copy (add a new category). Pick one, and the files selected will be copied/moved.

Don't worry too much about categorizing specifics until you've determined what is to be uploaded. It's much easier to re-categorize en masse.

You mention you have images on a disk...and you need to look for a digital source (aren't they digitized if they are on cd)? You don't need to have an online source. Rather, if the images are properly labeled on the cd, a program can read the images off the cd and upload them. In this age, you could also upload the cd online, and just do the upload from there.

I can't do much with the source-code until I'm sure where exactly it is I'm uploading from. I've done several batch uploads before with my bot: User:Smallbot including Oregon Historical County Records Guide, Commons:Bible Illustrations, Commons:Chris's Acorns, and currently the Beaverton, Oregon Historic Photo Gallery.Smallman12q (talk) 22:59, 21 September 2012 (UTC)[reply]

ok. I'm off for a few days so won't be at work to do any of this til Wednesday. I figured out the HotCat but thanks for the quick lesson for Cat-a-lot. I only looked at it briefly. You are right, I'm getting too bogged down into the categories, I guess it's the only way to see progress since I haven't learned to do a whole lot else. And in my head I don;t have a plan, other than the main page, which I made/you finessed, and uploading images.
as far as Arc catalog, NONE of the artifacts are in ARC (I got them from my local source files and our website), and I pulled all of the photos that I uploaded thru the upload wizard from local source files and our website too. Anything that has the NARA template was uploaded by them, hence their file information is deeper than mine. Is that what you are driving at? when you saY :don't show up, do you mean the image is not in the Arc catalog? Nothing I've uploaded will link to ARC because the ARC id number is not in my summary. and vice versa, but you know that.
as far as higher res images go, from my local files my choices were: zip file, the file I uploaded and a thmbnl size. Crap! Am I totally making a mess of this?!
really, I love Commons and GLAM, and it's a great idea, but the process is not clearly outlined in terms of what steps need to be taken and in what order so that newbie museum/archives people like myself can partcipate and not totally make a mess. And frankly NARA has been no help at all.
ARGHH!!
I'll locate the cd with the MDW State Funeral images on it. have a great weekend. Thanks for your patience. Bdcousineau (talk) 00:22, 22 September 2012 (UTC)[reply]

Wikipedia and the Commons are a learning experience. It can be a little frustrating at first because not everything is well-defined and because its different compared to the real world, but its neat once you the hang of it. GLAM is still fairly new, so not everything has been ironed out. There is a checklist at Commons:GLAM/Getting Started. You need a plan. You don't need to worry about making a mess. Everything on the wiki is saved forever and can be reverted/restored. It's strange to see NARA not helping out....bureaucracy I guess.

You don't seem to have a central catalog/database. So, could you list the urls to the main pages/indexes of the various collections you want uploaded? I could parse/upload them with a program then. When you find the cd, you can upload to your site or another file-hosting site.

I'll leave a note to several GLAM other volunteers to reach out to you. If you have any other questions, feel free to ask. It's always a positive to see you guys willing to participate.Smallman12q (talk) 11:32, 22 September 2012 (UTC)[reply]

I've looked for the url's and this is what I've discovered about our digitized documents - for example:
http://www.fordlibrarymuseum.gov/library/guides/findingaid/nscmeetings.asp#FTL is a list of linked pdf's. Clicck on a link and you get to the multi-page pdf (like this one:
http://www.fordlibrarymuseum.gov/library/document/0312/1552373.pdf). I can't imagine there is a way to batch upload this material. This is true for every category of documents that we have.
Here is a short list to start with of documents to upload (you'll have to scroll down to find the embedded link that says: View digital copies of the documents to get to the list of individual pdf's)
http://www.fordlibrarymuseum.gov/library/guides/findingaid/presidentsspeeches.asp#FTL
http://www.fordlibrarymuseum.gov/library/guides/findingaid/nscmeetings.asp#FTL
http://www.fordlibrarymuseum.gov/library/guides/findingaid/presidentsspeeches.asp#FTL
I think we've made this discovery before. Am I providing you with useful information? I could upload each pdf file via an ftp site if that might be a better way.
I could also do deeper research into the ARC catalog to see what I can find there.
I may need to ask one of the digitizing librarians if this material live in another location we could get to.
Let me know if this helps, please! Bdcousineau (talk) 19:40, 26 September 2012 (UTC)[reply]
This is what I'm looking for. This can be batch uploaded. Unfortunately, it doesn't seem like all the material in the boxes is digitized. Also, please list the urls at Commons:Gerald R. Ford Presidential Library and Museum/Sources
Pseudo-code
Load Web page
Locate CONTAINER LIST
For each box
    For each bullet
        Parse anchor text for date and filename
        Download file
        Location = The original documents are located in box <#>, folder <anchor text> of the President’s Speeches and Statements: Reading Copies at the Gerald R. Ford Presidential Library.
        Upload file
    Next
Next
Done

I've also responded below. Cheers.Smallman12q (talk) 21:31, 26 September 2012 (UTC)[reply]

I'm here and happy to help!

[edit]

I'm here and happy to help in whatever way I can. I am currently working on templates for the Ford Library and Museum in my sandbox. Just let me know how I can help. Take care! Michael Barera (talk) 03:41, 25 September 2012 (UTC)[reply]

thanks see below Bdcousineau (talk) 18:18, 26 September 2012 (UTC)[reply]

things I need and tasks I need to do

[edit]

What I need is a list of what I need.

images to donate
templates Can one of you advise me what kinds of templates would be useful? I've heard {{Gerald R. Ford Presidential Library and Museum-cooperation}} and {{Gerald R. Ford Presidential Library and Museum-image}}. I think User:Smallman12q is handling those. Love the ones in the my sandbox - where will these go? they are institution tags, right?
urls for image collections I will start collecting these for User:Smallman12q

I have a question about cats: I am putting Category:Images of Gerald R. Ford - xyz category - at the Ford Library and Museum on each of our images. Am I on the right track? so each File:Whatever.jpg will have that lowest/bottom designator, and then the group will belong to Category:Photographs at the Gerald R. Ford Presidential Library and Museum which is in Category:Gerald R. Ford Presidential Library and Museum. I think this is right.

Is there a way to batch categorize into various sub cats Category:US National Archives series: Gerald and Betty Ford Historical Photographs, compiled 08/09/1974 - 01/20/1977, documenting the period 1884 - 1974? or will I have to go one by one?

Can we replace

with

into the image files I have gathered into Category:Photographs at the Gerald R. Ford Presidential Library and Museum?

I'd like to make some changes in the summary area of the image files uploaded by NARA - is that possible? Specifically replace the Current Location link from NARA to ours. Also, in Record ID replace the NARA link with one of ours but keep the ARC Record id number.

and add {{Gerald R. Ford Presidential Library and Museum-cooperation}} to the files that don't yet have it....

I am happy to do these tasks if someone tells me how. ok, so all this and an end to animal and child abuse too, please. Bdcousineau (talk) 18:19, 26 September 2012 (UTC)[reply]

You can change the "summary" area of by clicking the "edit" button to the right of the summary header. Your categorization is fine. It's possible to replace portions of text to all files in a category with a simple bot. For batch categorizing, you would use cat-a-lot. Don't worry about categorizing the images yet...we can do that (we sorted through NARA's=P). The presentation of the data on the images will follow NARA's style, but re-branded for Gerald R. Ford Presidential Library and Museum. Right now, the main focus is the batch upload of the images. Once we have the images, it is much easier to determine how to sort them. For now, please get the index urls of the collections to be uploaded, as you've done above, and if possible, that cd.

User:Michael Barera (above) has offered to help. There is a w:Wikipedia:Michigan Wikipedians at UMich at Ann Arbor, of which Michael Barera is part of, so you could meet have a meetup.Smallman12q (talk) 21:31, 26 September 2012 (UTC)[reply]

Break

[edit]

I've connected with User:Michael Barera and plan on meeting with him in October. I've populated Commons:Gerald R. Ford Presidential Library and Museum/Sources with a ton of url's ~ please let me know if this is what you were expecting. Some are at the pdf level, others at the .asp/index level. These are things being systematically digitized, thus some collections are incomplete. I can add a few more sources if this is what you are looking for. These are all DOCUMENTS no photographs. If there is anything else I can be doing please assign me a task. Thank you! Bdcousineau (talk) 17:56, 27 September 2012 (UTC)[reply]

A lot of these links come from http://www.fordlibrarymuseum.gov/library/docs.asp which is fine. I'll have to go through the expanded list and make sure all the relevant urls are listed. I primarily need at the index level. Your list is a great start! I'll finish up the list and make another list for the photographs. The lists will ensure the bot reads/uploads all the relevant stuff in the indexes. If you have any collections on cd/dvd that are in public domain, you can upload the cds to the cloud, and I'll parse it from there. I'm going to be busy this weekend and next week in rl...so I'll start coding the upload after next week. I'll also finish up the templates then (or Michael can). If you have any further questions feel free to ask.
Your site says you have 25 million pages of documents and 450,000 photographs. I don't know whose responsible for your digitization efforts, but you really really ought to have a centralized database/catalog. Organizing that amount of data without a db (database) will give new meaning to inefficiency.Smallman12q (talk) 00:03, 28 September 2012 (UTC)[reply]

That sounds great - I can use the time to do the cloud upload etc. I was just given a box with 75 discs of state funeral material. It's unclear yet if it's video or stills.

I've tracked down the person at the Ford Library who knows the most about the document databases. She is out til Monday. I will get with her on Monday/Tuesday and report back. Apparently while there is no centralized database, there are databases for sub-collections. So one level easier, hopefully.

photo database:

http://www.fordlibrarymuseum.gov/library/whphotos/whpo1974.asp http://www.fordlibrarymuseum.gov/library/whphotos/whpo1975.asp http://www.fordlibrarymuseum.gov/library/whphotos/whpo1976.asp

each of these links to a month calendar, each day on the calendar links to a pdf with multiple pages of photo-contqct sheets. I will check with the photo archivist to see if there is easier access to this. This is similar to how the Daily Diary is arranged.

on docs.asp please don't include anything from the column titled Seleted External Sources - this is material housed somewhere else so I can't at this point verify the copyright status.

have a great weekend, and thanks. Bdcousineau (talk) 13:44, 28 September 2012 (UTC)[reply]

I've looked around the site and a lot of the links are similar, though some are bit...so it's a bit more work. For now, I'll start with the .pdfs, then do audiovisual. The documents all seem to be stored under "http://www.fordlibrarymuseum.gov/library/document/<number>/<file.pdf>" Would it be possible for you to grab all the .pdf in http://www.fordlibrarymuseum.gov/library/document/* , put them in a .zip (or other container), and upload them? This will make uploading much easier (a local copy of all the .pdfs so don't have to bother your site...don't want to hog all the bandwidth/resources, and the local copy to make sure I've got all the .pdfs and none are missed.) Please do get in touch with whoever is responsible for the digitization, and ask if there is some sort of central database/repository...there may be one that's simply not connected online for security/safety. Smallman12q (talk) 00:21, 1 October 2012 (UTC)[reply]

Break

[edit]

I've connected with User:Michael Barera and plan on meeting with him in October. I've populated Commons:Gerald R. Ford Presidential Library and Museum/Sources with a ton of url's ~ please let me know if this is what you were expecting. Some are at the pdf level, others at the .asp/index level. These are things being systematically digitized, thus some collections are incomplete. I can add a few more sources if this is what you are looking for. These are all DOCUMENTS no photographs. If there is anything else I can be doing please assign me a task. Thank you! Bdcousineau (talk) 17:56, 27 September 2012 (UTC)[reply]

A lot of these links come from http://www.fordlibrarymuseum.gov/library/docs.asp which is fine. I'll have to go through the expanded list and make sure all the relevant urls are listed. I primarily need at the index level. Your list is a great start! I'll finish up the list and make another list for the photographs. The lists will ensure the bot reads/uploads all the relevant stuff in the indexes. If you have any collections on cd/dvd that are in public domain, you can upload the cds to the cloud, and I'll parse it from there. I'm going to be busy this weekend and next week in rl...so I'll start coding the upload after next week. I'll also finish up the templates then (or Michael can). If you have any further questions feel free to ask.
Your site says you have 25 million pages of documents and 450,000 photographs. I don't know whose responsible for your digitization efforts, but you really really ought to have a centralized database/catalog. Organizing that amount of data without a db (database) will give new meaning to inefficiency.Smallman12q (talk) 00:03, 28 September 2012 (UTC)[reply]

That sounds great - I can use the time to do the cloud upload etc. I was just given a box with 75 discs of state funeral material. It's unclear yet if it's video or stills.

I've tracked down the person at the Ford Library who knows the most about the document databases. She is out til Monday. I will get with her on Monday/Tuesday and report back. Apparently while there is no centralized database, there are databases for sub-collections. So one level easier, hopefully.

photo database:

http://www.fordlibrarymuseum.gov/library/whphotos/whpo1974.asp http://www.fordlibrarymuseum.gov/library/whphotos/whpo1975.asp http://www.fordlibrarymuseum.gov/library/whphotos/whpo1976.asp

each of these links to a month calendar, each day on the calendar links to a pdf with multiple pages of photo-contqct sheets. I will check with the photo archivist to see if there is easier access to this. This is similar to how the Daily Diary is arranged.

on docs.asp please don't include anything from the column titled Seleted External Sources - this is material housed somewhere else so I can't at this point verify the copyright status.

have a great weekend, and thanks. Bdcousineau (talk) 13:44, 28 September 2012 (UTC)[reply]

I've looked around the site and a lot of the links are similar, though some are bit...so it's a bit more work. For now, I'll start with the .pdfs, then do audiovisual. The documents all seem to be stored under "http://www.fordlibrarymuseum.gov/library/document/<number>/<file.pdf>" Would it be possible for you to grab all the .pdf in http://www.fordlibrarymuseum.gov/library/document/* , put them in a .zip (or other container), and upload them? This will make uploading much easier (a local copy of all the .pdfs so don't have to bother your site...don't want to hog all the bandwidth/resources, and the local copy to make sure I've got all the .pdfs and none are missed.) Please do get in touch with whoever is responsible for the digitization, and ask if there is some sort of central database/repository...there may be one that's simply not connected online for security/safety. Smallman12q (talk) 00:21, 1 October 2012 (UTC)[reply]

Upload

[edit]

Ok, finally, I've located the super secret stand-alone computer with (hopefully) all (?) the files we'll need for a batch upload. I will travel to it on Thursday and begin to zip up the .pdf files for you. I will group them by topic or collection. Where do you want me to put them? I will do as many as I can - at this point I do not know how many/how far I'll get. I will check in with you on this page on Thursday when I get started. I can devote Thursday and Friday to this. On Oct. 17th I have to present this project to other staff, so at that point I may get more help - i.e. from someone who works in same building as the super-computer, so I won't have to drive 260 miles to do more zips. Also the metrics are proving that this project is already a success so that's fantastic!! Bdcousineau (talk) 19:58, 1 October 2012 (UTC)[reply]

I think its ridiculous in this modern day you have to physically travel somewhere to get the digital files. Anyhow, to review:
  1. You guys don't have a central database (really quite sad), so the site will be scraped (tech jargon for read/parsed) with a bot
  2. The files will have to be downloaded...however, the site isn't totally consistent and the files may be big.
  3. All the documents (.pdf) appear to lie in "http://www.fordlibrarymuseum.gov/library/document/<number>/<file.pdf>"
  4. I need all the .pdfs in http://www.fordlibrarymuseum.gov/library/document/* in their relative folder by number and with original file name in a zip file. So the folder tree structure/listing should be like this. You don't have to include empty folders:
Zip
+---010
|       blah x1.pdf
|       blah x2.pdf
|       blah.pdf
|
+---050 (empty)
\---356
        hlab.pdf
        hlab y2.pdf
        halb.pdf
  1. You can upload the zip (or whatever archive format .rar/.7z (better compression)/etc) file to the internet. If you have an ftp server, you can put it there (or on your server). Alternatively, you could upload to a free file-hosting service Hotfile, Google drive, skydrive (Microsoft), dropbox, or Archive.org. You should break the file into 300MB-500MB chunks. I don't expect the total file to be too large (a few gigs at most).
  2. This is an elementary-level task for a system admin/tech guy. You really shouldn't have to go anywhere.
If you have questions, feel free to ask.

Smallman12q (talk) 01:38, 2 October 2012 (UTC)[reply]

yes, well, I'm going to Ann Arbor anyway to meet with the local Wikipeidans about content-creation, so the trip dovetails nicely.

and so in fact I do not have access to any downloadable file-hosting services (of course considered a security risk) so I'll have to put the .zips onto our ftp server and send you the link. I'm hoping this work-around will be serviceable and quick, and encourage you to not jump ship in helping us! I'm getting our tech guy to help me so ostensibly you will get something useful.

The original file name may be numerical and I'm assuming we need something more descriptive, ie. titles or names, right? so I"ll do folder # and then a descriptive name.

This whole process is so intensely interesting and I surely did not even guess what I was getting into. Pandora's box, I think!! Bdcousineau (talk) 17:34, 2 October 2012 (UTC)[reply]

The pdfs should be an easy task...a one liner. With w:robocopy, it'd be "robocopy sourcedir targetdir *.pdf /s" and then zip and upload the file. I don't need anything more descriptive. The description will be extracted from the link text of the w:anchor tag and on a per-page basis. For example the PRESIDENT'S SPEECHES AND STATEMENTS: READING COPIES, 1974-77 page has that general description (for categorizing) and this file has the text "8/13/74 - Presidential Appointees", so the description will read "Presidential Appointees", and the file date will be set to "8/13/74". The file name would be "Presidential Appointees - 8/13/74 (PRESIDENT'S SPEECHES AND STATEMENTS, 1974-77] (Gerald Ford Library).pdf." Unfortunately, that's all the description provided online, so there isn't a lot of information. Some of the information can be modified manually (such as using more descriptive general categories). Tech projects are always a new experience. If you have any questions/suggestions feel free to ask.Smallman12q (talk) 01:15, 3 October 2012 (UTC)[reply]

Thanks for the quick response

[edit]

Thanks for the quick response! We, the members of Michgian Wikipedians, are very excited to help you in this process. You are more than welcome to come join us for one of our weekly club meetings, held at 8pm in room 4041 of the Shapiro Library. There are a lot of different things we can do for this project, such as working on English Wikipedia articles that relate to these images. Also, to try to answer your question about templates and formats, I've created a mock-up of a description page for one of the images you have uploaded: here is the mock-up for this image. Let me know what you think! Michael Barera (talk) 01:07, 28 September 2012 (UTC)[reply]

here is the mock-up is fantastic, and exactly what I was going to ask you to do! thank you!! Is it easy to insert into all the material that's already uploaded? Can we replace the {{NARA-cooperation}} with the one Smallman12q made {{Gerald R. Ford Presidential Library and Museum-cooperation}} ?
one thing, though, for the artifacts, we'll need a different Institution:Gerald R. Ford Presidential Library - specifically for the Museum, since the artifacts live in Grand Rapids. Paper/photo/video materials are in Ann Arbor, three dimensional things in Grand Rapids.
when we meet, I'm hoping you can teach me about metrics - I'd like to start tracking how/when our materials are being used.
I'm excited to meet with you and look forward to hearing what other ways we can maximize the Ford presence in the wiki universe.
if there is something else I can be working on/thinking about please don't hesitate to let me know.
this is SO exciting!! Thank you. Bdcousineau (talk) 14:17, 28 September 2012 (UTC)[reply]
To answer your questions, yes it would be quite easy to add the mock-up information to the images: two of the three changes are just adding templates (my institution template and Smallman12q's cooperation template, the same one you mention in your second paragraph). The third change is the trickiest but probably the most useful: adding a link to the image source page on your website that displays the photograph number right on the Commons page. Also, to answer your question about an institution template for images from the Museum, I've already taken care of that: check this out. And one final question: I noticed on the image source pages on the Ford Library and Museum website that there are much higher resolution versions of the images you have uploaded available as zip files. Is there any reason why we can't unpack the zip files and upload those higher resolution images to Commons? The quality is quite frankly exceptional, and if there is no specific reason for not doing it, I think everyone will benefit from having those higher quality images on Commons and Wikipedia. And one final thing regarding your question about materials usage: if you go to your user page, there is a small drop-down menu (indicated by an arrow just to the right of the heart and star buttons) that opens up a menu containing a button named "usage of all files": if you click that button (here is the link for it directly), it will show you the number of files that you've uploaded and the number that are in use on Wikimedia projects, sorted by which project they are used on. If you check the "show details" option and hit the "do it!" button, it will show you exactly which images are used on which articles. This is super-useful, although it may take you time to get familiar with the interface. I hope that this helps. Just let me know if there is anything else I can do to help: I'll try to keep checking your talk page, but if I'm unresponsive just shoot me a line at my talk page. Take care! Michael Barera (talk) 14:47, 29 September 2012 (UTC)[reply]
love check this out - that's great thanks!
yes, to .zips I think Smallman12q will be doing another batch upload of photos once we get the directory issues resolved, so we'll get better resolution in the future. Should I got thru and replace the ones I've done with zip version? Is there a quick way?, or am I back to one-by-one...
thanks for the metrics info - extremely useful - will most likely get LOTS of Library staff support now.
right now I'm grappling with 75 discs of Ford State funeral material, trying to figure out how to get it to a place to be uploaded- and hoping youtube is not my only option? any ideas helpful
and looking to make Smallman12q's batch upload easier by providing zipped pdf's. This may take a day or two.
see you Thursday Bdcousineau (talk) 17:14, 1 October 2012 (UTC)[reply]
I don't know if there is a quick way to upload new versions of images: I know batch-uploaders exist, but I've only seen them used for the initial image uploads (but again, I don't know and I'm not an expert on this topic). Wikimedia Commons does accept video, although it has to be in the open-standard .ogv format (but some of the video content here is spectacular: check out this video from NASA). YouTube can be used well for video uploads, and I know there are cultural institutions that have made good use of it, but I think there is plenty of room for video here on Commons as well. Michael Barera (talk) 00:00, 2 October 2012 (UTC)[reply]
You could batch upload new versions by overriding the older ones...but don't worry about that now. Metrics are available in the form of page-views on the wiki, file usage across the wikis, and as increased traffic to your site. You mention you have 75 discs (thats possibly more than >30 Gigabytes) of material. What's on the discs...and how is it organized...if at all? Videos can be uploaded after being converted to .ogv (it's a format conversion like a word document to .pdf). We do use video increasingly on the wiki, but they'd get more views on YouTube. There's no reason you can't upload to both=P.Smallman12q (talk) 01:46, 2 October 2012 (UTC)[reply]

Since I'm stuck right now in figuring out how to conveniently get regular .pdf files to be uploaded, I think the 75 discs of Ford's State Funeral footage may wait til a much later date. Transferring to .ogv involves a download - see above. I've made contact with the military about retrieving still images from Ford's State Funeral; can get to that later.

Thanks for the metrics info; have done some of that and passed around evidence of this project's success. I think more resources will be thrown my way.

yes, I want to move forward on these damned pdfs before I do more on the previously loaded images. Thanks Michael Barera for your suggestions. Bdcousineau (talk) 17:43, 2 October 2012 (UTC)[reply]

It's best to get the pdfs done first, then images...etc. Converting to .ogv is realtively simple...its just a different format (like .doc to .pdf but 100x more computational work). w:Theora#Encoding lists some of the different software for transcoding. It'll just take a little time for 30 gigs of video...could be a few days on a desktop computer.Smallman12q (talk) 01:19, 3 October 2012 (UTC)[reply]

getting closer - found the drive where thousands of the scanned images reside. I'll be able to start zipping those up tomorrow as well. They do not have narrative names, so I assume I'll be spending an eternity deriving names for them. Fun for me!! Bdcousineau (talk) 19:44, 3 October 2012 (UTC)[reply]

I could batch upload with just a descriptive name, but its always better to have a thorough description. When you do your do your presentation on the 17th, you should suggest the creation of a central database for your digital data. This stuff needs to be stored in a structured format so that you can properly use it in the future. I've looked at several of the other presidential libraries online, and they also don't seem to have a structured format for their images/documents...this makes it difficult to work with digital data. Digital stuff needs cataloging and curating too=P.Smallman12q (talk) 22:13, 3 October 2012 (UTC)[reply]

Ann Arbor trip to upload files

[edit]

um, yes, the other Presidential Libraries are disorganized too, which will slow down my plan to help them all do a project like this one. I will recommend a central database to the staff here, and will not get far, I imagine. More on that as it happens.

I will attempt to upload zipped files today. I will post the ftp link as soon as I know it. Bdcousineau (talk) 16:58, 4 October 2012 (UTC)[reply]

some success I think!!

http://www.fordlibrarymuseum.gov/Download/Smallman12q1/0023.zip
/Smallman12q1/0055.zip
/Smallman12q1/0069.zip
/Smallman12q1/0068.zip
/Smallman12q1/0303.zip
/Smallman12q1/0312.zip

these are ready now. These are being uploaded- may take a coupla hours

/Smallman12q2/0394.zip
/Smallman12q2/0331.zip
/Smallman12q2/0358.zip

/Smallman12q3/presspeechesbox12345.zip
/Smallman12q3/presspeechesbox678910.zip
/Smallman12q3/presspeechesbox1112131415.zip

/Smallman12q4/0010.zip

/Smallman12q5/0314.zip    this one is a monster

you'll be able to see quickly I did not follow your request for structure, sorry. Please let me know if this has been useful. Can do more tomorrow now I know the sequence - each person only revealed a piece of the process, so it was a long day putting it together. ugh. on the bright side, NARa is really impressed with this project and is pushing it out on mnay social media channels now. so YAY us! Bdcousineau (talk) 20:58, 4 October 2012 (UTC)[reply]

Thanks for putting them up...ye its useful. The format is fine. I've gotten: 0023, 0055, 0069, 0303. The rest of the links are 404 (dead or not available yet). As I noted above, this really isn't meant to be a challenge...hopefully you can get some tech person on board. NARA has ARC (Archival Research Catalog) which supports digital files...for the future, you should integrate your digital files there. The massive NARA upload was made possible by an organized database. A social media initiative would definitely be a plus. I'll file a bot request tonight/tomorrow along with a sample upload. Hopefully your meeting the UMich Wikipedians goes well. Cheers.Smallman12q (talk) 23:37, 4 October 2012 (UTC)[reply]

Darn it! This is supposed to be easier. My coworkers were mostly stingy with their help. Not yr prob!. I will double check the uploads tomorrow as well as add some new ones. I only checked the first few, and they were fine. Yes NARA has the nice database. Our stuff is somewhat inARC too, Ijust haven't been able to point you to it. I tried to zip some image files, and it took a long time because they were 600 dpi- does commons accept 200 dpi?

The wikimedians were great - they will start article creation, the ones with tech skills will help creating hidden cats, and generally make this a better project. I am immensely grateful, as their enthusiastic participation will help grow this. Once we have some content created I will do more pr and get this project out there as much as possible in a variety of places.

I heard form the wikimedians I'll be having to name each of the PDFs after the fact, that'll keep me out of trouble for while. Bdcousineau (talk) 02:24, 5 October 2012 (UTC)[reply]

Ann Arbor part deux

[edit]

Thank you Michael for putting the content divider in, much easier.

I reloaded and rechecked the following and got NO 404's:

http://www.fordlibrarymuseum.gov/Download/Smallman12q1/0023.zip /Smallman12q1/0055.zip /Smallman12q1/0069.zip /Smallman12q1/0268.zip not /0068.zip /Smallman12q1/0303.zip /Smallman12q1/0312.zip

/Smallman12q2/0394.zip /Smallman12q2/0331.zip /Smallman12q2/0358.zip

/Smallman12q4/0010.zip

/Smallman12q5/0314.zip

/Smallman12q3/blahblah.zip did not make it over. Yet.

I have also test uploaded http://www.fordlibrarymuseum.gov/Download/Smallman12q6/WhiteHouseStateDinnerMenus.zip tonight from my Grand Rapids computer - please let me know if it worked so I can do the other 8 folders I have prepared.

What happens when you batch upload these? will they have that lovely template and institution tag (the LIBRARY institution tag please)?

Excited to learn the next step! just in time for my show-and-tell to staff and others. Bdcousineau (talk) 00:53, 6 October 2012 (UTC)[reply]

Sorry for the wait. I've filed the bot request at Commons:Bots/Requests/Smallbot 5. I've parsed one of the pages and placed the derived xml at Commons:Gerald R. Ford Presidential Library and Museum/Sources1 (this is the information I have to work with for the upload). Each link represents a file and its attributes. I'm not fully sure how the naming should go or what to put in the description as there isn't much to go on. I'm also not sure what exactly to put in the template. I've uploaded a sample at File:Vice President and Romanian President Ceausescu (Background material only) (Gerald Ford Library) (002301000) .pdf...it's rather lousy as I'm not sure what to put/exclude. You're welcome to suggest how you guys want it. Once we get the general format down, I'll be able to upload the files. I'll download the rest of the files tonight (they all work now). You can see limits at Commons:File types and Commons:Maximum file size. Typically, the maximum file size is 100MB, the filename length is less than 255 characters, and the highest resolution which will have a thumbnail is 12.5 megapixels (12.5 million pixels for the whole image)...if its higher, a thumbnail won't show, but a bot can make a lower resolution version. Smallman12q (talk) 22:28, 6 October 2012 (UTC)[reply]
I have the following files:
0010.zip
0023.zip
0055.zip
0069.zip
0268.zip
0303.zip
0312.zip
0314.zip
0331.zip
0358.zip
0394.zip
WhiteHouseStateDinnerMenus.zip
and will download the 1gb /Smallman12q5/0314.zip tmrw(got it). The next thing to do is to determine how to format the information for the files (what to display)...and then upload. Commons accepts 600dpi and 200dpi and lower and higher. Generally, higher dpi is preferred (better quality).Smallman12q (talk) 02:44, 7 October 2012 (UTC)[reply]

first of all, this is SO COOL! The parsed page is amazing and the sample File is really incredible. Does it make sense to grab all the zipped files I can get to you first and then determine the naming and description and format? I still have 12 zipped files. this will give me a day or so to figure out format. this might be a conversation I have to include someone else in, which will slow it down. also I can look around on wikipedia and see what others have done. I'm back At my desk on Tuesday so will make those 12 available and get back to you about display formatting. Sadly, each of the zipped files is a different collection of documents, which may make this harder for me. Bdcousineau (talk) 19:03, 8 October 2012 (UTC)[reply]

No rush with the zip files...and sorry it's difficult, not meant to be. Once the format is figured out, that's when I'll need 'em. Smallman12q (talk) 22:12, 8 October 2012 (UTC)[reply]

About hidden categories and archiving your talk page

[edit]

I've already asked Smallman12q about hidden categories after I noticed that the new banner template he has created is designed to automatically file the Ford images into an appropriate hidden category (although at this point it is only one, not like the nested structure that we were talking about on Thursday). I don't want to step on his toes on this, so I've asked him about it on his talk page and am now waiting for a response. Also, I know your talk page is getting a bit long, so I wanted to share with you the two principle options you have for archiving it (if you want to read more about it, check out this page on English Wikipedia):

Option 1: archive by year (which is what I use) - put the following code at the very top of your talk page: {{User:MiszaBot/config |minthreadsleft = 1 |minthreadstoarchive = 1 |algo = old(31d) |archive = User talk:Bdcousineau/Archives/%(year)d }}

Option 1: archive by size - put the following code at the very top of your talk page: {{User:MiszaBot/config |archiveheader = {{talk archive navigation}} |maxarchivesize = 70K |counter = 1 |minthreadsleft = 4 |minthreadstoarchive = 1 |algo = old(31d) |archive = User talk:Bdcousineau/Archive %(counter)d }}

You can then set the variables to whatever values you would like: "algo" controls how often the page is archived, and I've found that 31 days/one month is quite good, but you can make it more or less if you would like to. "Minthreadsleft" controls the minimum number of separate conversations that are left on your talk page, regardless of how old they are (I keep it at 1, but some people like it higher), and "minthreadstoarchive" is the minimum number of conversations required to older than your "algo" for a move to take place (this really should be kept at 1). The only other thing to note is you'll have to actually create a subpage to put your archive: if your take the first (by year) method, you'll need to name it "User talk:Bdcousineau/Archives/2012" (it is also useful to have an intermediate "User talk:Bdcousineau/Archives" as well, in my opinion). If you take the second (by size) option, you'll need to name it "User talk:Bdcousineau/Archive 1". I hope that this helps! Michael Barera (talk) 17:27, 6 October 2012 (UTC)[reply]

Alright, I've begun working on converting the categories to hidden ones and adding "mainspace" categories. To see an early example, check out this image: the "documents" category is now hidden and "mainspace" categories like "food menus in English" have been added. This is what I was talking about on Thursday: hope it makes sense! Michael Barera (talk) 00:36, 7 October 2012 (UTC)[reply]
I've been working on this for a few hours now, and I've done all of the "documents" (just one image there) and "collections" categories, as well as the first 13 of the 15 "photographs" categories (but not the two big NARA-uploaded ones). I have not touched the "artifacts" category yet, but for one afternoon I think this is a good amount of progress. I've converted the categories that I've "done" to hidden categories and made sure that each affected image is properly categorized with "mainspace" categories (some were already, but many I had to add categories to). I hope that this makes sense: I'll continue working on those other three categories that aren't done yet tomorrow, if all goes well. Take care! Michael Barera (talk) 04:04, 7 October 2012 (UTC)[reply]

Ok, I'm lost. let me read these last entries tonight a few times and I'll resurface when they make more sense. You two are way ahead of me, quickly! thanks for the incredible amount of work. Bdcousineau (talk) 22:28, 7 October 2012 (UTC)[reply]


Ok, got it , understood. Will choose an archive option tomorrow. Thanks for explaining. Also about the hidden categories: for the dinner menus which are being uploaded, could we add "state dinners" or "White House state dinners" or "błack tie affairs" or something like that, there will be a huge bunch of these menus.
Would it be easy to add other cats as we go? I can think of of a few others like "handmade crafts" we have a lot of handmade stuff that people sent the president. Let me think of a few more broad categories. Bdcousineau (talk) 18:42, 8 October 2012 (UTC)[reply]
The general categories should be designed beforehand. Afterwards, it's easy to mass-add categories with cat-a-lot.Smallman12q (talk) 22:10, 8 October 2012 (UTC)[reply]

I've got my thinking cap on about the format and general categories, and am looking for other examples. As for the above comment, what's so great about this project is that every step is unknown and mysterious to me, not necessarily difficult . This is prolly the most interesting thing I've worked in since I got to handle Lewis and Clark's compass. I had no idea what I was getting into, DOH! Bdcousineau (talk) 23:39, 8 October 2012 (UTC)[reply]

Authority Control Numbers

[edit]

This is a minor point, but do you know the w:authority control numbers for the Library and Museum? The NARA template lists some:

National Archives and Records Administration   wikidata:Q518155
National Archives and Records Administration
Native name National Archives and Records Administration
Parent institution Federal Government of the United States Edit this at Wikidata
Location
Washington, D.C. (headquarters), and many regional facilities and presidential libraries nationwide in the USA
Coordinates 38° 53′ 34″ N, 77° 01′ 23″ W Link to OpenStreetMap Link to Google Maps Edit this at Wikidata
Established 19 June 1934 Edit this at Wikidata
Website www.archives.gov
Authority file
institution QS:P195,Q518155

.Smallman12q (talk) 02:51, 7 October 2012 (UTC)[reply]

did a quick search no results. will get back to you on this. Bdcousineau (talk) 18:23, 8 October 2012 (UTC)[reply]

Hidden categories all done

[edit]

Alright, I've just finished with the last of the hidden categories: they have now all been marked as hidden and I've made sure that each and every last image in them has been categorized with appropriate "mainspace" categories. Going forward, it would be easiest to make sure that the hidden categories are all added to the images during the upload process itself: if you're creating new hidden categories, just remember to nest them inside the structure and to tag them with the "{{hiddencat}}" template. We (or other users) can add the "mainspace" categories later, although it would probably be easier to do this sooner rather than later. I hope this all makes sense. If you have any questions, feel free to ask, and I'm always willing to help with this project. Thanks again for making the drive from Grand Rapids last Thursday and take care! Michael Barera (talk) 03:50, 9 October 2012 (UTC)[reply]

thanks! I've been looking into the PEP internship to see if we can get you set up at the Ford Library as Wiki-ian in residence. It looks reasonably easy from my end. Do you qualify for the program? If so, I would love to introduce you to the Supervisory Archivist on Oct 17th in a preliminary meet-and-greet. Please let me know your thoughts. Bdcousineau (talk) 14:34, 9 October 2012 (UTC)[reply]
Yes, I would love to look into a PEP internship at the Ford Library as a Wikipedian/Wikimedian in residence! Considering that October 17th is a Wednesday, I have a morning class (8:30-11:30am) and an afternoon group session (5-7pm) that I will need to schedule around, but I would certainly love to meet the Supervisory Archivist! Thanks so much and take care! Michael Barera (talk) 17:02, 9 October 2012 (UTC)[reply]

Could you be at the Library say noonish? Let me know what you find out about if you qualify too. Watch this page later, I'm working on some things and will want your input. Bdcousineau (talk) 17:04, 9 October 2012 (UTC)[reply]