File:Enrichment-datastore.png

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search

Original file(1,150 × 904 pixels, file size: 68 KB, MIME type: image/png)

Captions

Captions

Add a one-line explanation of what this file represents

Summary[edit]

Description
English: Ideas of how an enrichment datastore might work in the context of Cirrus
Date
Source Own work
Author Guillaume Lederrey

Source for this diagram:

@startuml

participant Hadoop

participant "PageRank\nDatastore" as prds
database "PageRank DB" as prdb
database "PageRank DB\nn-1" as prdb1

database "JobQueue" as jq

participant MediaWiki
actor User

database Elasticsearch

== New page rank ready ==
Hadoop -> prds: send new page rank data
create prdb
prds -> prdb: create new database
prds -> prdb: import new page rank data
|||
prds -> prdb: get data
activate prds
prds -> prdb1: get data
note left : implementation could\nalso work on raw files
prds -> prds: compute diff between\ncurrent and previous\npage rank
prds -> prdb1: destroy previous page rank DB
destroy prdb1
prds -> jq: create update jobs for changed page ranks
deactivate prds

== Page updated ==
User -> MediaWiki: update page
MediaWiki -> jq: create update job fo modified page

== Process updates ==
jq -> MediaWiki: start processing page update
activate MediaWiki
MediaWiki -> MediaWiki: get page data
MediaWiki -> prds: get page rank data
prds -> prdb: get page rank data
MediaWiki -> Elasticsearch: update Elasticsearch
deactivate MediaWiki

@enduml

Licensing[edit]

I, the copyright holder of this work, hereby publish it under the following license:
w:en:Creative Commons
attribution share alike
This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.
You are free:
  • to share – to copy, distribute and transmit the work
  • to remix – to adapt the work
Under the following conditions:
  • attribution – You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
  • share alike – If you remix, transform, or build upon the material, you must distribute your contributions under the same or compatible license as the original.

File history

Click on a date/time to view the file as it appeared at that time.

Date/TimeThumbnailDimensionsUserComment
current11:20, 19 May 2016Thumbnail for version as of 11:20, 19 May 20161,150 × 904 (68 KB)GLederrey (WMF) (talk | contribs)User created page with UploadWizard

There are no pages that use this file.