Commons:WMF support for Commons/Commons Impact Metrics

Objective/hypothesis

If we provide Commons Impact Metrics to the movement as data products, we can address outstanding metrics needs with the community. This includes the ability to scale metrics for all movement participants, reduce the cost to maintain existing Commons metrics tools, and increase accessibility.

Why do this?

Historically, the WMF has provided minimal support to affiliates and the wider movement concerning data and metrics. This minimal support often pushed affiliates and the movement to seek and build their solutions. This trend is evident in the area of Commons Impact Metrics, with multiple affiliates and communities developing their tools, websites, and technical solutions. For more details and context, see the WMF Support for Commons/GLAM Metrics Needs page on Wikimedia Commons.

As these tools have gained popularity, their fragility has been exposed, with increased outages, more bugs, decreased usability, and strained technical capacities.

The absence of these tools and data can impede movement participants across GLAMs and individuals from justifying their efforts on Commons, acquiring necessary funding, and more. These challenges can lead to significant knowledge and content gaps, as well as limiting movement participation. Therefore, providing these metrics is crucial for maintaining a thriving Commons community and evolving its content.

To build trust in our data, we must improve its perception and address issues like outages and overcounts.

Risks

Risk	Description	Status	Notes
Solution accessibility	We might deliver a solution without accessible entry points, forcing the community to create another fragile layer.
Metrics Alignment with the Community	Different roles and priorities can emerge, like individual contributors vs. partnerships vs. pipelines like Flickr.
Transitioning from existing data/tools	Potential data discrepancies can affect perception, especially if numbers aren't as significant as expected.		Trust and confidence will need building through a defined data process.

Success Criteria

For the hypothesis to be deemed successful, the following criteria should be met:

• Commons metrics data products should replicate specific analytics in existing GLAM metric tools for Commons.
• Metrics should be accessible to a more extensive part of the GLAM community.
• Commons Metrics should be available to non-GLAM community members.
• The WMF and the community should adopt shared metrics concerning Commons.

Target Outcomes

The project aims to produce reliable data products that detail essential Commons impact metrics, assisting GLAMs and movement participants in understanding the positive effects of their contributions to the open-knowledge movement. Expected outcomes are:

• Well-documented Commons Metrics data products.
• Clear guidelines on using and leveraging Commons metrics data products.
• Established mechanisms for data access (e.g., dumps, API).

What’s Out of Scope?

Certain elements are considered out-of-scope for this hypothesis validation:

• Generating metrics or data products unrelated to Commons' re-use or impact.
• Analytics data related to Wikidata.
• Maintaining, replacing, or migrating existing Commons metrics tools.
• Developing features on Commons.

Frequently Asked Questions

This section aims to provide you with quick and clear answers to help deepen your understanding of this initiative and its significance. Whether you're new to this topic or looking for specific information, we hope you find these insights helpful.

Who is defining the roadmap and who decided this project should exist?

The starting point for the Commons Impact Metrics roadmap was to meet the use cases served by commonly-used tools, such as BaGLAMa2, GLAMorgan and GLAM Wiki Dashboard. These tools have experienced maintenance and scaling issues over the last couple of years, leading to community discussions about a ‘crisis of GLAM metrics’. Following many community conversations about these issues, in calls, mailing lists and Telegram channels, a page was developed on Commons to document GLAM Metrics Needs.

What affiliates are consulted and involved, and why (if not all)?

For our initial design research interviews, we focused on affiliates and partners who had developed, or were planning to develop, their own metrics solutions.

Why am I just hearing about this endeavor?

Following many community conversations about these issues, in calls, mailing lists and Telegram channels, a page was developed on Commons to document GLAM Metrics Needs. The project was announced at a high-level in the Culture & Heritage section of the Foundation’s annual plan. There is also a sub-page under the Support for Commons page, where we have been communicating all Foundation work relating to Commons. In the coming weeks, we plan to broaden our engagement efforts, starting with a session at the global GLAM Wiki Conference in Montevideo. Following the conference, we will involve a diverse range of stakeholders, including Wikimedia affiliates and members of the general Commons community. Our commitment is to ensure an inclusive and collaborative process where every participant feels informed and integral to the project's development from its early stages.

Why can’t we just improve the best existing community made tools?

The existing tools created by the various GLAMs/Commons community have been valuable, but they have faced significant maintenance and scaling challenges, including extended outages. These issues highlight the need for more robust, reliable, and scalable solutions. The Commons Impact Metrics project aims to develop and maintain new data products that can effectively scale for all movement participants. This approach is expected to provide a more stable and efficient solution for measuring the impact of contributions on Commons.

Which version of the Glam Wiki Dashboard have we migrated to WMF cloud?

We supported Wikimedia Israel to migrate the latest version of their dashboard to the Wikimedia cloud. This removed prohibitive data costs and allowed them to restore their service for existing partners. We were able to work with Wikimedia Israel on this because they had a developer who was knowledgeable about, and still actively working on, their service.

How will requirements gathering look, and when will this occur?

In the first stage, we analyzed the community-developed GLAM Metrics Needs page and commonly-used tools, such as BaGLAMa2, GLAMorgan and GLAM Wiki Dashboard, to understand the underlying data needs. We then organized 8-10 design interviews with affiliates and partners who either used those tools, or were known to be thinking about creating new metrics tools. We used this information to produce a preliminary data model and dataset to be workshopped at the global GLAM Wiki Conference. Following this conference, all of the documentation will be published and we will facilitate 1-2 online versions of this workshop. We will then follow our usual processes for on-wiki product consultation.

Will the dataset be structured for flexibility?

We are planning to use categories and sub-categories, following commonly-used tools, such as BaGLAMa2, GLAMorgan and GLAM Wiki Dashboard.

Will this contain data for all of Commons? And, will users be able to access cross-project commons file usage data?

It will be possible to view cross-project utilization and views for Commons media. We won’t produce the data for every category that exists on Commons due to the immense and ever-growing number of categories. However, there will be a process for adding categories that should be available. We will consult the community on the best way to do this.

Will there be an easy means of accessing and analyzing the dataset?

The Commons Impact Metrics project this annual year is focused on the use cases served by commonly-used tools, such as BaGLAMa2, GLAMorgan and GLAM Wiki Dashboard. It will deliver the following metrics for categories and subcategories of images: utilization of images by Wikimedia projects (including different language versions) and articles; views by Wikimedia projects and languages; number of contributors and number of edits. By developing a shared definition for each metric, delivering data reliably without interruptions or over-counting; and providing clear documentation and guidelines, the project will support volunteers, affiliates, and partners to demonstrate the impact of their contributions. The project intends to make the metrics data accessible through mechanisms such as data dumps and APIs, which can be integrated in existing tools or new visualizations to support a range of research and reporting needs within the Commons community.

What is going on with the GLAM Wiki platform and Commons project as a whole?

The plans for this fiscal year were shared in the Culture & Heritage section of the Foundation’s annual plan. In addition to this metrics project, we are:

• Reducing the burden of deletion requests on Commons moderators through Upload Wizard Improvements project
• Partnering with the Flickr Foundation to develop and improved and maintained pipeline of images from Flickr to Wikimedia Commons (Flickypedia)
• Providing follow-on funding for OpenRefine to support the adoption of the Commons batch upload extension through training.
• Supporting the transcription of manuscripts, through improved OCR/HTR services, training, and targeted improvements to Wikisource.

You can follow WMF Support for Commons page, which is updated monthly.

Related Resources

• GLAM Prototype Data Model and Implementation Plan
• User Testing Instructions: Commons Impact Metrics Prototype
• GLAM Metrics Needs page

Project Contacts

• Desiree Abad
• Virginia Poundstone
• Fiona Romeo
• Miriam Redi
• Benedict Udeh