Commons:Batch uploading/Library of Congress

Library of Congress

Uploads after 2014 have been run using different methods, refer to User:Fæ/LOC.

This is a project coordination page to explain the process used and to keep track of issues and past uploads

• Source to upload from:
  • Library of Congress collections
  • There is an API, though web pages with metadata in MODS format are usable.
  • I have been in correspondence with the library on API access, it is limited to 15 enquiries per minute.

• Describe the works to be uploaded in detail (audio files, images by …):
  • Suitable collections listed at http://www.loc.gov/pictures/

    These are not all suitable, some collections have few images online, others are neither government works, nor before 1923.

• Which license tag(s) should be applied?
  • {{PD-1923}}

• Is there a template that could be used on the file description pages? Do you think a special template should be created?
  • {{LOC-image}}

Fæ (talk) 11:39, 18 June 2014 (UTC)

Technical

Initial uploads used custom scripts to upload, the most recent use the Special:GWToolset which requires an xml file to be generated.

Naming is of the form:

File:<descriptive title> LCCN<lccn>.tiff

For an explanation of the unique lccn identification, refer to http://lccn.loc.gov/

For early GWT uploads naming has been forced to use "-LCCN" rather than " LCCN".

Opinions and issues

These include tiffs and there has been on-going issues with regard to Commons' thumbnail generation of very large tiffs (>50MP) and whether we should host jpeg files for convenience in parallel with the tiffs.

Progress

• Live report of all Library of Congress uploads by Fæ (Deadlink as of September 2023)

Code	Assigned to	Progress	Bot	Category
cpbr	Fæ Uploads use {{Artwork}}. Supporting credit to WMUK.	Status:    Done	Custom	British Cartoon Prints Collection: 1,017R
pgz	Fæ Uploads use {{Photograph}} There has been significant post-upload "housekeeping" to: Add country sub-categorization. Add the parent category as this got left off after a re-run was needed when GWT was changed. Upgrade jpgs to the same size as tiffs, using the sips command under OSX (so local downloading and uploading is required). Supporting credit to WMUK.	Status:    Done	GWT	Photochrom prints collection: 20R Original total c.11,500. Current category total will be less due to volunteer recategorizations.
pan	Fæ Uploads use {{Photograph}} Initial analysis included an assessment of how many tiffs would be over 100MB, there are only a handful out of a couple of thousand images. A bulk of the collection appears to be pre-1923 (the test sample of 100 had 93 as published in 1923 or earlier). Where they exist, location categories may be applied, for example Los Angeles. This may be slightly controversial, however the alternative is to make country bucket categories which seems a worse option. No credit needed.	Status:    Done	GWT	Library of Congress panoramic photographs collection: 0R
tisc	Fæ Use {{Artwork}} Images relating to the history of ballooning	Status:    Done	GWT	Library of Congress Tissandier collection: 217R
item 02121	Fæ Create special 'page turner' script to find all sub-images within a LoC item Use {{Photograph}} Photographs of the September 11th attack on the World Trade Center - all from the same photographer	Status:    Done	GWT	Library of Congress images of September 11 attacks: 0R
HABS	Fæ Create special webscraping query to generate xml as the photographs do not have LCCNs Use {{Artwork}} - swapping to {{Photograph}} as only a small proportion of files are scans of building plans Use HABS license template 6,079 images were previously uploaded for HABS Post-upload geolocation templates Post-upload creation of PNG files from 50 MP+ TIFFs (via API, not GWT) catscan report of 50 MP+ TIFFs (Hard, this requires uploading an array of all existing files in memory) Cross link files from same scanned document using other_versions (example) Ensure post-upload categorization is skipped when other editors have touched the file Set up backlog page for HABS related category creation: Commons:Batch uploading/Library of Congress/HABS	Status:    In progress	GWT	Catscan query Files from the Historic American Buildings Survey: 303919 152% completed (estimate)