Consumerium development wiki

Knowledge graphs: Difference between revisions

← Older edit
Visual
moved contents from Wikibase unchanged to gain better understanding of the possibilities and issues
+ '''SPARQL endpoint''' at databus.dbpedia.org
 
(26 intermediate revisions by the same user not shown)
Line 1: Line 1:
= DBpedia =
Acquiring access for our consumers to a [[w:semantic network|semantic network]] of relevant [[w:linked data|linked]] [[w:open data|open data]] compiled by other efforts and structured by a number of [[w:ontology (information science)|ontologies]] is obviously key to Consumerium. Reciprocally we aim to share the information we gather and compile available to other efforts.
'''[[w:DBpedia|DBpedia]]''' [http://dbpedia.org (.org)] is a community effort to enable the web moving ''"Towards a Public Data Infrastructure for a Large, Multilingual, Semantic Knowledge Graph"''.
 
Today the DBpedia '''[[w:data set|data set]]s''' contain a wealth of information structured into '''[[w:Ontology (information science)|ontologies]]'''. This [[w:structured data|structured data]] can be queried with [[SPARQL]] query language at their [http://dbpedia.org/sparql public DBpedia SPARQL endpoint].
 
There are many methods how the DBpedia ontology and datasets could be used in the Consumerium implementation stage wiki. One of these would be to install the [[mw:Extension:LinkedWiki|LinkedWiki extension]].
 
== Ontology classes useful for implementing Consumerium ==
'''[http://mappings.dbpedia.org/server/ontology/classes/ All DBpedia ontology classes]'''
 
* [http://mappings.dbpedia.org/server/ontology/classes/Company Company]
* [http://mappings.dbpedia.org/server/ontology/classes/Non-ProfitOrganisation Non-ProfitOrganisation]
* [http://mappings.dbpedia.org/server/ontology/classes/Food Food]
** [http://mappings.dbpedia.org/server/ontology/classes/Beverage Beverage]
 
* [http://mappings.dbpedia.org/server/ontology/classes/MeanOfTransportation MeanOfTransportation]
** [http://mappings.dbpedia.org/server/ontology/classes/Aircraft Aircraft]
** [http://mappings.dbpedia.org/server/ontology/classes/Automobile Automobile]
** [http://mappings.dbpedia.org/server/ontology/classes/Locomotive Locomotive]
** [http://mappings.dbpedia.org/server/ontology/classes/Ship Ship]


* [http://mappings.dbpedia.org/server/ontology/classes/Place Place]
See also: [[mw:Manual:Managing data in MediaWiki]] at mediawiki.org
** [http://mappings.dbpedia.org/server/ontology/classes/ArchitecturalStructure ArchitecturalStructure]
*** [http://mappings.dbpedia.org/server/ontology/classes/Building Building]
**** [http://mappings.dbpedia.org/server/ontology/classes/Hotel Hotel]
**** [http://mappings.dbpedia.org/server/ontology/classes/Restaurant Restaurant]
**** [http://mappings.dbpedia.org/server/ontology/classes/ShoppingMall ShoppingMall]
*** [http://mappings.dbpedia.org/server/ontology/classes/Infrastructure Infrastructure]
**** [http://mappings.dbpedia.org/server/ontology/classes/Airport Airport]
**** [http://mappings.dbpedia.org/server/ontology/classes/RouteOfTransportation RouteOfTransportation]
***** [http://mappings.dbpedia.org/server/ontology/classes/RailwayLine RailwayLine]
***** [http://mappings.dbpedia.org/server/ontology/classes/Road Road]
**** [http://mappings.dbpedia.org/server/ontology/classes/Station Station]
***** [http://mappings.dbpedia.org/server/ontology/classes/RailwayStation RailwayStation]


* [http://mappings.dbpedia.org/server/ontology/classes/PopulatedPlace PopulatedPlace]
** [http://mappings.dbpedia.org/server/ontology/classes/AdministrativeRegion AdministrativeRegion]
** [http://mappings.dbpedia.org/server/ontology/classes/Continent Continent]
** [http://mappings.dbpedia.org/server/ontology/classes/Country Country]
** [http://mappings.dbpedia.org/server/ontology/classes/Island Island]
** [http://mappings.dbpedia.org/server/ontology/classes/Settlement Settlement]
*** [http://mappings.dbpedia.org/server/ontology/classes/City City]
*** [http://mappings.dbpedia.org/server/ontology/classes/Town Town]
*** [http://mappings.dbpedia.org/server/ontology/classes/Village Village]
== DBpedia datasets ==
* '''[http://wiki.dbpedia.org/Datasets DBpedia datasets]''' have been released annually, sometimes with improvements more frequently. The latest (as of July 2018) [https://wiki.dbpedia.org/develop/datasets/dbpedia-version-2016-10 DBpedia dataset version is 2016-10] which was published in 2017.
== Databus ==
At DBpedia there is ongoing work on what is called '''[https://wiki.dbpedia.org/blog/dbpedia-databus-%E2%80%93-transforming-linked-data-networked-data-economy DBpedia Databus]''' to take their game to the next level. Databus Alpha was published in May 2018.
== History of DBpedia ==
DBpedia began as an effort to extract structured information from [[Wikipedia]] [[templates|infobox templates]] and [[categories]] and to make this information available on the Web with the initial release on January 10th 2007.
== More info on DBpedia  ==
* [https://wiki.dbpedia.org/ DBpedia wiki]
* [https://blog.dbpedia.org/ DBpedia blog]
* [https://github.com/dbpedia/ DBpedia code at GitHub]
== See also ==
* [[Database]] for a higher level view of what is going on with the databases
* [[Semantic MediaWiki]] is system for inputting and querying semantic data within the MediaWiki and it is implemented as extension(s).
* [[Wikidata]] and the underlying [[Wikibase|Wikibase extensions]]
----
= Wikidata =
= Wikidata =
[[File:Wikidata-logo-en.svg|thumb|right|260px|The [https://wikidata.org Wikidata] logo]]


'''[[w:Wikidata|Wikidata]]''' [http://wikidata.org (.org)] is a [[w:knowledge base|knowledge base]], an effort to store and serve structured data to [[Wikimedia]] wikis and to a more limited extent to other parties. Wikidata effort saw the daylight in [[2012]].  
'''[[w:Wikidata|Wikidata]]''' [https://wikidata.org (.org)] is a [[w:knowledge base|knowledge base]], an effort to store and serve structured data to [[Wikimedia]] wikis and to a more limited extent to other parties. Wikidata effort saw the daylight in [[2012]].  


The underlying software is the '''[[Wikibase]]''' which consists of 2 [[Mediawiki]] extensions, the repository extension and the client extension.  
The underlying software is the '''[[#Wikibase|Wikibase]]''' which consists of 2 [[Mediawiki]] extensions, the [[mw:Extension:Wikibase Repository|Wikibase Repository]] and the [[mw:Extension:Wikibase Client|Wikibase Client]].


Wikibase allows [[interwiki]] links to be managed with Wikidata removing much contributor annoyanges, redundancy and error-proneness.
Wikibase allows [[interwiki]] links to be managed with Wikidata removing much contributor annoyanges, redundancy and error-proneness.


Wikidata is obviously a main source of [[reference]] level data once it is technically possible for non-WMF wikis to access the data items.
Wikidata is obviously a very viable source of [[reference]] level data once it is technically possible for non-WMF wikis to access the data items. (See [[#LinkedWiki extension]] for a potential workaround for this limitation)


It can be accessed outside of WMF wikis with with  
It can be accessed outside of WMF wikis with with  
* [[wikidata:Wikidata:SPARQL_query_service/queries|Wikidata advice on how to query]] [https://query.wikidata.org/ Wikidata's endpoint] using [[SPARQL]].
* [https://query.wikidata.org/ Wikidata's SPARQL endpoint] using [[SPARQL]] ([[wikidata:Wikidata:SPARQL_query_service/queries|Wikidata advice on how to query]]).
* [[mw:Wikidata Toolkit]] is a way for Java programs to access data in the Wikidata repository.
* [[mw:Wikidata Toolkit]] is a way for Java programs to access data in the Wikidata repository.
* [https://rdflib.github.io/sparqlwrapper/ SPARQL Endpoint interface to Python]
* [https://rdflib.github.io/sparqlwrapper/ SPARQL Endpoint interface to Python]
* [[wikidata:Wikidata:Database download|Wikidata offers its database for download]]
* [[wikidata:Wikidata:Database download|Wikidata database download]]
 
* The [https://tools.wmflabs.org/wikidata-game/# Wikidata Game] [[w:gamification|gamifies]] adding information to Wikidata.
 


Main entry point of any Wikidata item is a [[w:JSON|JSON]] dictionary, that has this form:
Main entry point of any Wikidata item is a [[w:JSON|JSON]] dictionary, that has this form:
Line 92: Line 34:


“sitelinks”: by-language dictionary}</code>
“sitelinks”: by-language dictionary}</code>
== Lexicographical Wikidata ==
A '''lexeme''' is a unit of [[w:lexical semantics|lexical]] meaning that underlies a set of words that are related through [[w:inflection|inflection]]. It is a basic abstract unit of meaning,<ref>''The Cambridge Encyclopedia of The English Language''. Ed. [[w:David Crystal|]]. Cambridge: Cambridge University Press, 1995. p.&nbsp;118. {{ISBN|0521401798}}.</ref> a [[w:emic unit|unit]] of [[w:Morphology (linguistics)|morphological]] [[w:Semantic analysis (linguistics)|analysis]] in [[w:linguistics]] that roughly corresponds to a set of forms taken by a single root [[w:word]]. For example, in [[w:English language|English]], ''run'', ''runs'', ''ran'' and ''running'' are forms of the same lexeme, which can be represented as <span style="font-variant:small-caps; text-transform:lowercase;">RUN</span> (Wikipedia on 2019-12-29)
Since 2018, Wikidata has also stored a new type of data: words, phrases and sentences, in many languages, described in many languages. This information is stored in new types of entities, called '''Lexemes''' ('''L'''), '''Forms''' ('''F''') and '''Senses''' ('''S''').<ref>[[wikibooks:SPARQL/WIKIDATA Lexicographical data]]</ref>. This is enabled by [[mw:Extension:WikibaseLexeme|the WikibaseLexeme extension]].
* [[wikidata:Wikidata:Lexicographical_data/Documentation|The main documentation page for lexicographical data on Wikidata]]
* [[wikidata:Wikidata:Tools/Lexicographical data|Wikidata's list of lexicographical properties and tools using the lexicographical data]]
* [[wikibooks:SPARQL/WIKIDATA Lexicographical data|Wikibook on Wikidata's lexicographical data]]


== Useful information ==
== Useful information ==
Line 99: Line 51:
'''More info'''  
'''More info'''  
* [[m:Wikidata|Metawiki on Wikidata]]
* [[m:Wikidata|Metawiki on Wikidata]]
* [[w:Wikipedia:Wikidata|Wikipedia advice on Wikidata issues]]  
* [[w:Wikipedia:Wikidata|Wikipedia advice on Wikidata issues]]
 
== See also ==
* [[Database]] for a higher level view of what is going on with the databases
* [[Semantic MediaWiki]] is system for inputting and querying semantic data within the MediaWiki and it is implemented as extension(s).
* [[DBpedia]] the original structured data harvesting effort for the MediaWiki wikis


== Wikibase ==
== Wikibase ==
[[File:Wikibase_logo.png|thumb|right|250px|The [http://wikiba.se/ Wikibase] logo]]
'''[[mw:Wikibase|Wikibase]]''' [http://wikiba.se/ (wikiba.se)] is a system for storing and querying structured data that powers [[Wikidata]] and other wikis.
'''[[mw:Wikibase|Wikibase]]''' [http://wikiba.se/ (wikiba.se)] is a system for storing and querying structured data that powers [[Wikidata]] and other wikis.


Line 114: Line 62:
:# [[mw:Extension:Wikibase Client|Wikibase Client]] that allows a wiki to access structured data from a repository. The client can work only with repository databases it can access so they must be on the same machine or the same load balancer.
:# [[mw:Extension:Wikibase Client|Wikibase Client]] that allows a wiki to access structured data from a repository. The client can work only with repository databases it can access so they must be on the same machine or the same load balancer.


=== Installation ===
=== Installation of Wikibase ===
[[mw:Wikibase/Installation|Wikibase installation instructions at Mediawiki.org]] and [[mw:Wikibase/Installation/Advanced_configuration|advanced configuration of Wikibase]].  
[[mw:Wikibase/Installation|Wikibase installation instructions at Mediawiki.org]] and [[mw:Wikibase/Installation/Advanced_configuration|advanced configuration of Wikibase]].  


The installation instructions assume you are installing the dependencies with [[mw:Composer|Composer]], a PHP package manager that makes the installation of dependencies easy.
The installation instructions assume you are installing the dependencies with [[mw:Composer|Composer]], a PHP package manager that makes the installation of dependencies easy.


=== Related useful extensions ===
=== Useful extensions in conjunction with Wikibase ===
* [[mw:Extension:ArticlePlaceholder]] makes article placeholders from repository data and invites users to create the article.
* [[mw:Extension:ArticlePlaceholder]] makes article placeholders from repository data and invites users to create the article.
* [[mw:Extension:UniversalLanguageSelector]] is recommended to work in conjunction with Wikibase for user comfort.
* [[mw:Extension:UniversalLanguageSelector]] is recommended to work in conjunction with Wikibase for user comfort.


=== Alternative to using Wikibase Client ===
== Alternative to using Wikibase Client ==
* [[mw:Extension:LinkedWiki]] can be configured to access multiple SPARQL endpoints.
* [[#LinkedWiki extension]] can be configured to access multiple SPARQL endpoints.


=== Useful information ===
=== Useful information ===
Line 131: Line 79:
* [https://phabricator.wikimedia.org/T159240 Phabricator Task to "Document how to set up federated Wikibase instances"]
* [https://phabricator.wikimedia.org/T159240 Phabricator Task to "Document how to set up federated Wikibase instances"]
* [https://phabricator.wikimedia.org/T196997 Phabricator Task to "Wikidata/Wikibase federation and distribution"] and the [https://docs.google.com/document/d/1YYIuQzcWz2cH9zTUbbfiBrTecVxMWAu3_9-U4DcKzEU/edit#heading=h.z49pje934vur Google Docs document charting out the task]
* [https://phabricator.wikimedia.org/T196997 Phabricator Task to "Wikidata/Wikibase federation and distribution"] and the [https://docs.google.com/document/d/1YYIuQzcWz2cH9zTUbbfiBrTecVxMWAu3_9-U4DcKzEU/edit#heading=h.z49pje934vur Google Docs document charting out the task]
----
----


= Semantic MediaWiki =
= Semantic MediaWiki =
 
[[File:SemanticMediaWiki_Logo.png|thumb|right|200px|The [https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki Semantic MediaWiki] logo]]
[[File:SemanticMediaWiki_Logo.png|thumb|right|200px|Semantic MediaWiki logo]]


'''Semantic MediaWiki''' [https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki (.org)] (SMW) is a free, open-source [[MediaWiki extensions|extension]] to [[MediaWiki]] that lets you store & query semantic data within the wiki and it seems well suited to [[Consumerium]]'s information infrastructure needs.
'''Semantic MediaWiki''' [https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki (.org)] (SMW) is a free, open-source [[MediaWiki extensions|extension]] to [[MediaWiki]] that lets you store & query semantic data within the wiki and it seems well suited to [[Consumerium]]'s information infrastructure needs.


== Spinoff extensions ==
== Spinoff extensions ==
A variety of open-source MediaWiki extensions exist that use the data structure provided by Semantic MediaWiki.
A variety of open-source MediaWiki extensions exist that use the data structure provided by Semantic MediaWiki.


Line 150: Line 95:
* [[mw:Extension:Semantic Maps|Semantic Maps]] - displays geographic semantic data using various mapping services
* [[mw:Extension:Semantic Maps|Semantic Maps]] - displays geographic semantic data using various mapping services


== See also ==
----
= Open Food Facts =
{{:Open Food Facts}}
----
= DBpedia =
[[File:DBpediaLogo.svg|thumb|right|260px|The [https://www.dbpedia.org/ DBpedia] logo]]
'''[[w:DBpedia|DBpedia]]''' [https://www.dbpedia.org/ (.org)] is a community effort to enable the web moving ''"Towards a Public Data Infrastructure for a Large, Multilingual, Semantic Knowledge Graph"''.
 
Today the DBpedia '''[[w:data set|data set]]s''' contain a wealth of information structured into '''[[w:Ontology (information science)|ontologies]]'''. This [[w:structured data|structured data]] can be queried with [[SPARQL]] query language at their [http://dbpedia.org/sparql public DBpedia SPARQL endpoint].
 
DBpedia uses [[w:Virtuoso Universal Server]] to store and query the data. [https://www.w3.org/wiki/VirtuosoUniversalServer VirtuosoUniversalServer at w3.org]
 
There are many methods how the DBpedia ontology and datasets could be used in the Consumerium implementation stage wiki.
 
== Ontology classes useful for implementing Consumerium ==
'''[https://dief.tools.dbpedia.org/server/ontology/classes/ All DBpedia ontology classes]'''
 
* [http://mappings.dbpedia.org/server/ontology/classes/Company Company]
* [http://mappings.dbpedia.org/server/ontology/classes/Non-ProfitOrganisation Non-ProfitOrganisation]
* [http://mappings.dbpedia.org/server/ontology/classes/Food Food]
** [http://mappings.dbpedia.org/server/ontology/classes/Beverage Beverage]
 
* [http://mappings.dbpedia.org/server/ontology/classes/MeanOfTransportation MeanOfTransportation]
** [http://mappings.dbpedia.org/server/ontology/classes/Aircraft Aircraft]
** [http://mappings.dbpedia.org/server/ontology/classes/Automobile Automobile]
** [http://mappings.dbpedia.org/server/ontology/classes/Locomotive Locomotive]
** [http://mappings.dbpedia.org/server/ontology/classes/Ship Ship]
 
* [http://mappings.dbpedia.org/server/ontology/classes/Place Place]
** [http://mappings.dbpedia.org/server/ontology/classes/ArchitecturalStructure ArchitecturalStructure]
*** [http://mappings.dbpedia.org/server/ontology/classes/Building Building]
**** [http://mappings.dbpedia.org/server/ontology/classes/Hotel Hotel]
**** [http://mappings.dbpedia.org/server/ontology/classes/Restaurant Restaurant]
**** [http://mappings.dbpedia.org/server/ontology/classes/ShoppingMall ShoppingMall]
*** [http://mappings.dbpedia.org/server/ontology/classes/Infrastructure Infrastructure]
**** [http://mappings.dbpedia.org/server/ontology/classes/Airport Airport]
**** [http://mappings.dbpedia.org/server/ontology/classes/RouteOfTransportation RouteOfTransportation]
***** [http://mappings.dbpedia.org/server/ontology/classes/RailwayLine RailwayLine]
***** [http://mappings.dbpedia.org/server/ontology/classes/Road Road]
**** [http://mappings.dbpedia.org/server/ontology/classes/Station Station]
***** [http://mappings.dbpedia.org/server/ontology/classes/RailwayStation RailwayStation]
 
* [http://mappings.dbpedia.org/server/ontology/classes/PopulatedPlace PopulatedPlace]
** [http://mappings.dbpedia.org/server/ontology/classes/AdministrativeRegion AdministrativeRegion]
** [http://mappings.dbpedia.org/server/ontology/classes/Continent Continent]
** [http://mappings.dbpedia.org/server/ontology/classes/Country Country]
** [http://mappings.dbpedia.org/server/ontology/classes/Island Island]
** [http://mappings.dbpedia.org/server/ontology/classes/Settlement Settlement]
*** [http://mappings.dbpedia.org/server/ontology/classes/City City]
*** [http://mappings.dbpedia.org/server/ontology/classes/Town Town]
*** [http://mappings.dbpedia.org/server/ontology/classes/Village Village]
 
== DBpedia datasets ==
* '''[http://wiki.dbpedia.org/Datasets DBpedia datasets]''' have been released annually, sometimes with improvements more frequently. The latest (as of July 2018) [https://wiki.dbpedia.org/develop/datasets/dbpedia-version-2016-10 DBpedia dataset version is 2016-10] which was published in 2017.
 
== DBpedia Databus ==
The [https://databus.dbpedia.org/ '''DBpedia Databus''' at databus.dbpedia.org] ''is a data cataloging and versioning platform for data developers and consumers.''
 
[https://databus.dbpedia.org/sparql '''SPARQL endpoint''' at databus.dbpedia.org]
 
DBpedia developed the DBpedia Databus in the late 2010's with a Databus alpha published in May 2018.
 
== History of DBpedia ==
DBpedia began as an effort to extract structured information from [[Wikipedia]] [[templates|infobox templates]] and [[categories]] and to make this information available on the Web with the initial release on January 10th 2007.
== More info on DBpedia ==
* [https://wiki.dbpedia.org/ DBpedia wiki]
* [https://blog.dbpedia.org/ DBpedia blog]
* [https://github.com/dbpedia/ DBpedia code at GitHub]
 
----
= LinkedWiki extension =
[[File:LogoLinkedWiki.png|thumb|right|260px|Logo of the [[mw:Extension:LinkedWiki|LinkedWiki extension]]]]
A possible way to tap into various knowledge graphs is the [[mw:Extension:LinkedWiki|LinkedWiki extension]]. LinkedWiki has been developed since 2010 by [[mw:User:Karima Rafes]], a [[#Semantic MediaWiki]] developer and CEO of [http://www.bordercloud.com/ BorderCloud.com]


''' See also '''
* [https://linkedwiki.com/ LinkedWiki.com - The best place to discover available Linked Data and to share SPARQL queries]
* [[Database]] for a higher level view of what is going on with the databases
* [[Database]] for a higher level view of what is going on with the databases
* [[DBpedia]] the original [[w:structured data|structured data]] harvesting effort for the MediaWiki wikis
* [[Wikidata]] and the underlying [[MediaWiki/Extensions#Wikibase|Wikibase extension]]


== Screenshots ==
= References =
<references/>


[[File:AcaWiki Semantic Forms screenshot.png|thumb|right|650px|A form to edit a page, using the [[mw:Extension:Semantic Forms|Semantic Forms extension]]]]
[[Category:List]]
[[Category:Technology]]
Retrieved from "https://develop.consumerium.org/wiki/Knowledge_graphs"