Commons:Monuments database/Structured lists

Structured lists are the source of data that is gathered in the monuments database. They are kept on various Wikipedias, and provide information about cultural heritage monuments from specified countries in a clear, good-looking and consistent way.

Get the data

Somehow you need to get the data. The data should be in some sort of structured format: A database, an Excel sheet, a csv file, .... This data is the source of the lists.

Analyze the data

First take a good look at the data and see what's in there. Write down all the fields and put a note after the field what you think is in there. You might want to do that in several passes. You can see some examples at User:Multichill/Monuments notes. This process is to understand the data. You first need to understand it before you can go to the next step.

For each field write down if it's useful to include in the Wikipedia lists. You probably have too many fields so you have to decide what the fields are you really need.

Create the templates

Structured lists are the source of all data in the monuments database. They are kept on various Wikipedias, and provide information about cultural heritage monuments from specified countries in a clear, good-looking and consistent way. They are built with templates - so they can be maintained manually by Wiki-editors easily (easier than Wiki tables!) and automatically by a bot that harvests all the information and puts it into the database.

The lists are created in the form of wikitables, and consist of two templates:

• a header template containing the header of a table;
• a row template containing the actual information about a single monument.

The names of the templates vary from one language to another, and in the case of languages that are used in many countries (English, German, Spanish) even among countries. This page gives an idea of naming of the row templates.

The header template sets the top of the table:

{{Monuments header}}

You can include some fields in the header template. These will function as default value for the row templates after the header:

{{Monuments header
|municipality=Bloemendaal
}}

In the table row, every monument is structured in a block like this (example):

{{Monument row
|municipality=Bloemendaal
|description=Oud Leyduin: wooden house, partially with windows down to the ground, shed roof and double tympanum
|type_obj=G
|org_function=House
|build=18th century
|build_sort=1700
|architect=
|address=
|lat=52.34214
|lon=4.59454
|objno=364845
|image=
}}

These templates can be very simple – just generate a table row – or a bit more complex to fit your needs, for example if you include a coordinate template, some more template logic, or a specific upload button that passes some parameters.

Some examples

no:Mal:Kulturminner row

a simple row template

en:Template:Rijksmonument row

a sophisticated template with some logic and extra functions.

Other examples can be found at Commons:Wiki_Loves_Monuments_2011/Monuments_database#Sources_overview. It is recommended to get help from your local "template wizards" who know template syntax well. The names of the templates vary from one language to another, and in the case of languages that are used in many countries (English, German, Spanish) even among countries.

Create the lists from the excel file

Use Excel or Open/Libre Office Calc to process the data. If you have the data in an html table, just copypaste it into it, and fix the errors first. Make a backup copy of this file.

Make a choice of which columns you want to use in your Wikipedia table. Remove the columns that you do not want to use. Change the headers (row 1) or add them: use exactly the same name as the name of the parameters in the template. Move the columns so that they are in the right order you want them to be mentioned in the templated lists. Also add empty columns with only the header for the parameters that you simply don't have data for yet, but which you might want to add in the wiki later manually (such as latitude and longitude).

In Libre Office, you will need to:

• Select all (Control-A)
• Remove all formatting (right mouse click, remove all formatting)
• Select all
• Right Mouse click - Cell Formatting
• In the popup window: in the first tab, select 'text' as category

A good size for the monument lists (to make it humanly possible to load the page on a normal computer) is between the 100 and 200 monuments per page. This means you will have to split up the table into different lists. You will have to do this manually. Usually it makes sense to split the list up based on the district, municipality or another geographic characteristic.

this image shows which is the separator, text deliminator etc

Repeat these steps for each list page:

• Copy the 100 - 200 monuments into a temporary file (with the headers)
• Save the file as a 'csv' (comma separated values) file. If you have the choice, use ';' as a separator.
  • If you have quotation marks (") in your data, use another character as text deliminer (“quotations“) The default is a double quotation mark (") but you could use for example a dollar sign ($) or at-sign (@).
  • Use Unicode (UTF-8) as encoding for the output
  • In Libre Office, select 'display all text fields'
• Open the csv file in a notepad
• Copy the content of the file
• Paste this into the csv2template tool
• Add the name of your row template in the field "Template name" (for example: "ASI Monument row")
• Set the separator character. Most of the times this will be by default a comma. If you chose a different separator, take that.
• Set the quotation character if you changed the text deliminator to @, set that in this setting. (depends on your Excel/Calc settings)
• Set escape character if you turned this on in Excel/Calc. Most likely it is off by default.
• Click "Convert to templates"
• Create the list page you wanted to produce: add the header template
• Copy the output of this tool into this page and check the results.
• Add a category

And you should be done after clicking 'save page'.

Helpers

• csv2template – an online tool to convert csv files to structured lists, based on a template you choose. A (German language) screencast shows how to use it.