Commons:Wikimaps/Design

This document is out-of-date. Please review it for inspiration.

This document describes a workflow for taking historical maps, publishing them to the Commons, rectifying them for usage as layers in historical applications, creating geodata from them and linking that data to other resources, primarily to Wikidata.

The technical description and project plan have been moved to Commons:Wikimaps/Development.

The flow for historical maps in Wikimaps

A scheme showing the workflow for historical maps in Wikimaps

1. GLAM (memory organization) sends large amounts of maps to Wikimedia Commons and utilizes the GLAM-Wiki toolset with the Map template.  Done
2. In Wikimaps the map images can be given their map coordinates, through sending a map for georeferencing in Wikimaps Warper.  Done
3. In Wikimaps the maps could be annotated, in other words the users should be able to add information about them and discuss them.
4. The maps could be digitized, that is the historical map features could be turned into vector geometry. The OpenHistoricalMap is included in the Wikimaps Warper.  Done
5. The resulting historical geodata could be stored in or linked to items in Wikidata, thus creating a powerful platform for a historical gazetteer. For example, loosely dated historical period ("Roman Era") can be used as defined in wikidata.
6. Through the map service the annotated historical maps could be embedded in Wikipedia or elsewhere.
7. The enriched maps could be exported to the original GLAMs, or read by them and others.
8. The maps can be searched using interfaces made for the purpose and through APIs.

A diagram showing how things would link within Wikimaps

Uploading maps

Scanned map files can be uploaded with all the normal upload tools. Only GWToolset supports using the Map template in the upload process. It is also possible to use the template if you craft your uploads yourself.

Additional upload methods may emerge

• Through third party software
• Saving a map found in an open repository through the map search interface
• A browser bookmarklet
• In the article editing process

Map template

Maps and their metadata can be uploaded as batches with the GWToolset. Map metadata properties should be kept in single file uploads also.

We have devised the Template:Map scheme for historical maps. Later, the same scheme can be applied when using Wikibase as the data repository for Wikimedia Commons. We hope to participate in collaboration about harmonizing metadata import with other GLAM activists and GLAM institutions.

The metadata properties

The published version is at Template:Map

Info

• Title
• Description
• Map legend
• Author(s) Several roles needed.
• Creation date
• Source
• Permission

Geotemporal data

• Map date The date that the map depicts. Calendar system. Time range. Named era.
• Map location The location(s) the map depicts. Should be allowed to input what's written on the map, and related to the place id through WIkidata.
• Map type Such as city plan, road map, panoramic drawing ...
• Map projection EPSG. Named SRSs? Default.
• Map scale Must be expressed as 1 relative to n, or n units to n units. Ex. one verst per inch. One werst map. 1:42000.
• Heading The rotation of the map in degrees
• Bounding box Lower corner, upper corner (OSG BBox)
• Precise area that the map covers - perhaps Control Points that can be used to reproject the map into WGS84. Link into OSM, OHM, wikidata, Muninn or others to locate control points? (a proposal)

Bibliographic In printed maps, related to all maps by the same publication data.

• Containing publication (reference to map series. Should there be an index sheet?)
• Sheet reference (id, location in series: how to store, what's the interface)
• Language The map language(s)
• Publisher
• Printer
• Print date Date or year.
• Authority control Like in the Book template. ISBN, URI, etc. Should be possible to add several.

Archival Data on the specific copy of a map.

• Institution
• Archival reference
• Dimensions Amounts and unit.
• Medium Same as Artwork. Base and medium.
• Inscriptions
• Other versions

Things to consider

• Component type Such as legend, scale, compass rose, map, inscription, index ... For sub- and supersections of a map. Are they separate documents or annotated regions?
• Map copy Manuscript, original, copy, draft ... Vocabulary for most used options.
• How do we deal with free-form annotations?
• Can the properties be closely related with standard metadata properties? Which ones? How?

Map metadata storage

Maps are currently stored in Wikimedia Commons. Their metadata is a part of the stored item. The future arrangement for the Commons images is that metadata will be stored in a separate Wikibase (Wikidata knowledge base) instance at the Commons. This will mean all data within Wikimedia Commons will become structured and queryable, while searches combining items in Wikidata and in the Commons Wikibase are more difficult. So we are preparing for 2 environments: One that exists and one that is planned.

Map display in the Commons

Maps are shown in the Media Viewer with limited metadata and no access to further tools. The file page in Commons can host the additional information and tools required to work with the maps.

The metadata display

• Map metadata  Done
• Button to bring the map into Wikimedia Commons  Done
• Map locator map

Wikimaps gadget

We would like to create a gadget that allows converting one or more images into a map. It would

• Show a form to convert existing metadata properties to the Template:Map scheme and refine the existing metadata
• Add the Map locator map within the Map template

An image showing the Wikimaps Gadget in the Tools menu

You can install a gadget that creates a link to the Wikimaps Warper to all file pages by copying the following code to your common.js, found in Preferences under Appearance and Custom JavaScript.

//Wikimaps Gadget
importScriptURI("//commons.wikimedia.org/w/index.php?title=User:Andr%C3%A9_Costa_%28WMSE%29/Gadget-Wikimaps.js&action=raw&ctype=text/javascript");

Map locator map

This is a proposal for the use of a slippy map with historical maps. The map locator map should be shown on the File-page of the map image or the Media Viewer.

If the map has no data about the location

• A prompt is shown: The formulation is: "Help place..."
• The button start georeferencing is shown "Place the map"

If the location can be reasoned from metadata, in a textual representation in the metadata, the filename etc

• The location is pinned on a slippy map
• The Warper button is shown

If the metadata contains also information about the scale (and rotation)

• The map is displayed overlaid on a slippy map, rotated and scaled
• The bounding box will be shown (maybe only on hover)
• The Warper button is shown

If we decide to make a placing tool inside this locator map

• The tools for dragging, scaling and rotating are shown
• The bounding box will be shown
• An execute button for the placement is shown
• The display beneath the map shows the location based on coordinates, time, language and scale.

If the map has been warped in the Wikimaps Warper

• The bounding box will be shown
• A link to the Wikimaps Warper to adjust the warping

Wikimaps tools: Maps search

Wikimaps tools reside outside Wikimedia Commons, and they are accessed through OAuth.

Currently maps are searched in Wikimedia Commons through metadata attributes, but there should exist a map based search. We can create a map search that searches maps added to the Warper.

Search interface

Controls

• Zoom, pan
• Geolocation search
• Layer controls
  • Show/hide with lens, curtain
  • Transparency
• Layers
  • Base map, preferably in vectors
    • Current map
    • Historical map
    • Temporal controls
  • Different styles?
  • Raster maps
  • Vector data
  • POIs
• Actions
  • Export & embed layered view
  • Download files

Simple search

• General search grouping results by method found (see image above)

Search by free text

• Limiting searches to specific metadata fields, proposing existing Wikidata options

Search by location

• Framed on a base map

Search by place name

• Based on a historical gazetteer located in Wikidata
• Eg. Reval > 13th C – 1917 (Tallinn, Estonia) or 1941 – 1944 (Tallinn, Estonia)
• Free text hits using translations eg Rääveli > Reval > Tallinn
• Using phonetic algorithms, fuzzy searches

Wikimaps search input - Time

Search by time

• Time ranges
  • Before / after [date / event / era]
  • Between [date / event / era] and [date / event / era]
  • During [event / era]
• Dates
  • Copy/paste from sources > reasoning
  • Different conventions: DD/MM/YYYY, MM/DD/YYYY, DD.MM.YYYY etc
• Based on a historical event / era that has defined timeframe (Wikidata)
  • Medieval age in Sweden
  • The reign of …
  • Helsinki Olympics
  • … occupation in …
• Extended
  • Imprecision: approx, first, last, turn of, end of, latter half, first quarter
  • Granularity: century, decade, year, month, week
  • Fuzziness: Allow fuzzy period before and/or after: amount and unit (The model of Wikidata)
• Calendar selector
• Calendar: Gregorian, Julian…

Future directions for search

The maps could and should also be searched in open archives around the world for georeferencing. By finding a map suitable for your purposes, you could also save that map into Wikimedia Commons for further use and enrichment.

Wikidata can store the metadata of the files, with CC0 as the license. The map images, holding more restrictive licenses, could be viewed, or imported, if the licensing is permissive. There is a mapping proposal for the map properties in Wikidata. It has not yet been made into a Wikidata task force.

The maps search should allow displaying several layers of maps, geodata and thematic data.

Wikimaps tools: The Warper

Import

• Login through OAuth from Wikimedia Commons
• An option to import from any open source is studied

Adding control points on a base map and the map to be warped

• Add / modify / delete control points
• Alternatively give corner coordinates together with an EPSG for quick warping with modern maps

Applying transformations based on the control points

• GDAL transformations
• An option for orthorectification is studied
• Alternatively change the projection of the base map
• As a third alternative for an old map, no distortion, only locking the map at a POI, and scaling & rotating to the best average when being displayed on a current map

Masking the image

• Stitching a series of masked images into a whole

Export

• As transformed image files: GeoTIFF, PNG
• Transformation data
  • Control points
  • Transformation settings
  • File format to be studied
• Tiles
• KML
• WMS

Import

• Georeferencing data produced by other environments
• Map images from external open services

Warper data

• The transformed image file
• Transformation data: We are looking into a new data structure (an exchange format) for warping data: control points and warp settings.

Wikimaps Warper 2.0

The plan for the Wikimaps Warper 2.0 project.

Wikimaps tools: iD editor for historical maps

• The OHM database

Further tools

Here are some tools that we have come across for extending the environment, or ideas that could be pushed further in the environment

Vectorizing capabilities

• NYPL Building inspector
• Vectorizer code at GitHub
• Feature extractor / Mapbox / iD

Annotating maps

• Annotorious
• MapHub project

Toponym extraction

• Recogito
• Cymru1900
• Find place names / Building Inspector

Georeferencing historical images

• Ajapaik
• Second Take
• Mapillary

Mosaicking (is a feature in the Warper, needs to be enabled)

• Also MapKnitter

Map stories

• StoryMap JS
• Neatline

Tools for working with historical gazetteers

We investigate the possibility of having Wikidata as a central place name index. Wikidata can model place name and feature changes in a rich manner and that could be taken advantage of.

• Finnish National Gazetteer of Historical Places
• New York Public Library’s Chronology of Place / LOC Digital Gazetteer Blog post
• Histopo.nl
• Pleiades
• Histograph

Wikimaps Use cases & scenarios

will be found on a new sub page