Commons:WMF support for Commons/Commons Impact Metrics/Instructions:Commons Impact Metrics Prototype User Testing

Thank you for testing the WMF Data Product Team’s prototype for the Commons Impact Metrics data product.

At this discovery stage, we are trying out a method for calculating various utilization and engagement metrics for Commons media starting with GLAM use cases. Media is organized on Commons using the Mediawiki category structure. The trickiest aspect of creating these datasets is the non-standardized application of categories on Commons.

With this prototype, we have tried an approach to working with categories. Now, it is time to gather feedback on this approach from Commons metrics data users across the GLAM communities. This feedback will be incorporated into our next phase of work and ensure we build a Commons Impact Metrics data product that is useful, easy to use, and meets the GLAM community’s needs and expectations.

Goals

We took the main use cases from three of the currently most used tools: BaGLAMa2, GLAMorgan, and the GLAM Wiki dashboard. Currently these tools all pull “raw data” and perform various calculation methods. Our goal is to provide:

• Centralized logic for “cooked” data so that all dashboards, tools and gadgets can use the same base calculations instead of “cooking” the data individually using various different calculation methods
• Reduce complexity by establishing shared definitions, reducing joins, and precomputing data into simple fit for purpose datasets.
• Serve this data via APIs and (or) data dumps
• Once completed, this will allow us to
• Mitigate calculation discrepancies across tools and dashboards
• Scale our ability to solve calculation issues centrally instead of solving similar problems in slightly different ways across various implementations
• Set service level objectives that ensure uptime, availability, and data quality
• Consolidate on wiki documentation about this data and how to access and use it into an easily discoverable and accessible hub, or developer portal, that reduces the archaeological digging or interpersonal connections currently necessary to find.

Format

The prototype is composed of five datasets using sample data from fourteen primary categories. This data is available in a set of five queryable google sheets (one for each dataset):

• GLAM Commons categories, gives information about the GLAM Commons categories at the time the snapshot is taken. It includes both primary categories and all their subcategories, recursively down the tree.
• GLAM Commons media files, gives information about the GLAM Commons media files at the time the snapshot is taken. It includes category information so that the user can filter by category.
• GLAM Commons article views by category, gives information about the number of views GLAM articles receive across time. And it breaks them down by the GLAM category. A GLAM article is a wiki article (ns=0 page) that contains at least one media file belonging to a GLAM Commons category.
• GLAM Commons article views by media file, gives information about the number of views GLAM articles receive across time. However, it breaks them down by the GLAM media file. A GLAM article is a wiki article (ns=0 page) that contains at least one media file belonging to a GLAM Commons category.
• GLAM Commons edits, a record for each edit (creation, update) to a Commons media file. It also records the user who did the edit and the categories the media file belongs to.

User Testing Process

How To Make A Copy Of The Commons Impact Metrics Prototype Template

A walkthrough on testing the Commons Impact Metrics prototype.

1. Make a copy of the prototype template. Save your copy in the Commons Impact Metrics Prototype folder replacing [template] in the title with your name. Together these six digital documents will be your individualized workspace.
2. Review the READMEs for each dataset on the first tab of each google sheet so you can familiarize yourself with the data dimensions and limitations.
3. Browse the example use cases available in each sheet
4. While you review and browse, add questions, concerns, or encouragement as comments in the google sheets.
5. Try out querying the data. Build your own use cases by either overwriting existing queries or creating new tabs. Remember: this is your individualized workspace, so you don’t have to worry about breaking anything.
6. Once you’ve had a chance to play around with the data please answer the questions in the section below.
7. That’s it! Follow our progress on wiki, stay in touch with us via the talk page or in our new Phabricator project board.

Questions to Answer

After you have reviewed the data model and tinkered with the sample data, please answer the following questions:

1. How do you feel about our approach to the category tree?
2. What questions are you able to answer with these datasets? Do the datasets miss any questions / use cases you may have?
3. How might we leverage Commons Impact Metrics data to support existing tools? What would be required to do so?

Stay in the know

• Watch the project page on Commons
• For deeper understanding of how we are approaching the complexity, dig into the Commons Impact Metrics - GLAM Prototype Data Model and Implementation Plan
• Email us directly: vpoundstone@wikimedia.org
• Phabricator project tags: #Commons-Impact-Metrics & #Data-Products

Thank you!