Systems selections: Difference between revisions
(→Data sources: + https://www.w3.org/wiki/SparqlEndpoints) |
(→Data sources: + Commercial data sources + 'Google Maps Platform -> Web Services -> Distance Matrix API' at developers.google.com) |
||
| (23 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
This article is for choosing good systems (technical) for The [[Consumerium]] Effort - Enhancing Consumer Informedness | |||
== Factors affecting | == Factors affecting systems selections == | ||
* [[Copyleft]] 'free to modify' and 'free | * [[Copyleft]] 'free to modify' and 'free of cost' software is strongly preferred over other solutions. | ||
* Minimal modifications | * Minimal modifications made (MMM) is an preference as well. This will ensure that we don't get swamped down on patching ad-hoccy, unstandard aged solutions to keep 'em working with the new stuff. | ||
---- | ---- | ||
== Encodings == | == Encodings == | ||
| Line 10: | Line 11: | ||
* [[w:UTF-8]] encoded [[w:Unicode]] for text. UTF-8 is efficient and allows 2^32 different characters.. unless escaped to 40-bit in future revision | * [[w:UTF-8]] encoded [[w:Unicode]] for text. UTF-8 is efficient and allows 2^32 different characters.. unless escaped to 40-bit in future revision | ||
== | == Copyright and copyleft protections system selections == | ||
* [[Copyleft]] [[licensing]] for content, code and [[Consumerium Commons]] media repository | |||
* Automatic plagiarism testing which usually probably also translates to copyvio | |||
* Disqualify and rapidly remove all copyvio | |||
* Discourage piratism. Piratism on copyleft much stronger than on copyright. | |||
== Data identifiers == | |||
* [[ | * [[GTIN]] (contains [[EAN]] and [[UPC]] spaces) identifies uniquely a [[product]] to the systems. Also contains the country of the [[marketer]] (2 first digits of EAN) and address of the marketers address space. It may be possible to acquire namespace size from examining these values. Possible values are 100 ids and 10,000 ids (EAN). | ||
* [[w: | * [[ISO]] identifiers exist for at least [[languages]], [[countries]], country-subdivision [[areas]], [[securities]] and many more things [[w:List of International Organization for Standardization standards]] | ||
* [[ | * [[UN/LOCODE]] (United Nations Code for Trade and Transport Locations) identifies globally nearly all locations of interest to [[logistics]]. | ||
''' | == Product and service classifications == | ||
'''[[Product and service classifications]]''' | |||
== Databases == | == Databases == | ||
'''Main article: [[Databases]]''' | |||
'''Main article: [[ | |||
Currently using all major free software RDBMS: | Currently using all major free software RDBMS: | ||
| Line 32: | Line 36: | ||
'''Considered databases''' | '''Considered databases''' | ||
* [[Databases#Graph databases|Graph databases]] and [[Database#Subject-predicate-object databases|Subject-predicate-object databases]] seem to perform the same thing just with different architecture and suitability for certain searches. See [[Talk:Databases]] for more on this. | * [[Databases#Graph databases|Graph databases]] and [[Database#Subject-predicate-object databases|Subject-predicate-object databases]] seem to perform the same thing just with different architecture and suitability for certain searches. See [[Talk:Databases]] for more on this. | ||
'''Candidates for testing''' | |||
* All copyleft licensed triplestore and more advanced i.e. graph | |||
---- | |||
== Languages == | |||
Programming languages | |||
''' Currently used languages ''' | |||
* [[w:PHP]] (5) | |||
* [[w:Ruby (programming language)]] | |||
* [[w:Python (programming language)]] | |||
'''Considered languages / languages of interest''' | |||
---- | ---- | ||
| Line 37: | Line 54: | ||
== Frameworks == | == Frameworks == | ||
'''Frameworks''' | '''Frameworks''' for glueing into various systems | ||
* [[Semantic MediaWiki]] | * [[Semantic MediaWiki]] compatibilities: PHP, Semantic Forms, enter data in [[wiki code]] -> RDF <-> Graph -> Outputs wikirenders and public [[SPARQL]] endpoints | ||
---- | ---- | ||
== Software == | == Software == | ||
''' | '''Current server software''' | ||
* [[w:Operating system|Operating system]]: [[w:Debian|Debian]] [[w:GNU/Linux|GNU/Linux]] | |||
* [[w:Hypervisor|Hypervisor]]s (by hosting guys) [[w:Xen|Xen]] and [[w:Kernel-based Virtual Machine|KVM]] | |||
* Webservers: [[w:Nginx|Nginx]] and [[w:Apache HTTP Server|Apache]] | |||
* [[MediaWiki]] ([[w:PHP|PHP]], [[w:MariaDB|MariaDB]] and more) | |||
* [[w:diaspora (software)]] ([[w:Ruby on Rails|Ruby on Rails]] ,[[w:Ruby (programming language)|Ruby]], MariaDB, [[w:Node.js|Node.js]] and more) - role: [[consumer protection]] oriented [[free social media]] since 2010 | |||
* [[w:Draft:Hubzilla]] (PHP, MariaDB) - Excellent channel-based distributed free social media where subscription is to the whole network allowing nomadic and double citizenship. A model the others should follow suit. | |||
* [[w:Friendica]] (PHP, MariaDB, fully contained in database w/ images stored as blobs. The least learning curve free social media for [[facebook refugees]]. Initiated by the same guy as Hubzilla. | |||
* [[w:GNU social]] MariaDB - [[w:Microblogging|Microblogging]] platform | |||
* [[w:GNU MediaGoblin]] (Python2 (Installing Python3 was impossible last time I looked at it, theoretically compatible with Py3 but broken because outside code supplier.) and [[w:PostgreSQL|PostgreSQL]]) Role: [[Copyleft]] [[code]], [[content]] and [[codecs]] based media repository. | |||
''' Future software ''' | |||
* Once [[MaidSafe]] goes persistent stage Consumerium will be deploying many nodes in order to earn SafeCoin to pay for future storage needs. Provision of information services to the denizens of the SAFE Network is also naturally a priority but basically patching a normal webapp onto the SAFE Network is straightforward and easy. Also: commiemism. | |||
* [[ | |||
... | ... | ||
| Line 60: | Line 83: | ||
APIs to communicate with: | APIs to communicate with: | ||
* [https://www.mediawiki.org/wiki/API:Main_page MediaWiki API] | |||
* Free social media ones | * Free social media ones | ||
* Commercial social media ones | |||
---- | ---- | ||
== Data sources == | == Data sources == | ||
''' Data sources: Own own data source ''' | |||
* [[Semantic MediaWiki]] well thought out and set up on implementation wiki (c.consumerium.org) | * [[Semantic MediaWiki]] well thought out and set up on implementation wiki (c.consumerium.org) | ||
* [[Development Wiki]] - User database, userspace sharing or importing and Unified Login are being looked at as options. | * [[Development Wiki]] - User database, userspace sharing or importing and Unified Login are being looked at as options. | ||
''' Data sources: Semantic web ''' | |||
* Other copyleft Semantic MediaWikis | |||
* '''[[DBpedia]]''' [http://dbpedia.org/sparql Public SPARQL endpoint of DBpedia.org] | The [[w:semantic web]] at large and the [[w:semantic wiki]]s are treasure troves of interesting data which Consumerium should tap into early on to avoid redundant manual replication and duplicate data insertions. | ||
* Other copyleft Semantic MediaWikis | |||
** [http://smw.referata.com/wiki/Special:BrowseData/Sites Semantic MediaWiki Community wiki list of SMW sites] | |||
** [https://wikiapiary.com/wiki/Semantic_statistics Wikiapiary list of SMW sites] | |||
** [http://wikiindex.org/Category:Semantic_MediaWiki Wikiindex.org category of SMWs] | |||
* '''[[DBpedia]]''' [http://dbpedia.org/sparql Public SPARQL endpoint of DBpedia.org] and [http://wiki.dbpedia.org/projects Projects and efforts using DBpedia] are very relevant. | |||
* '''[[Wikidata]]''' [https://query.wikidata.org/ Public SPARQL endpoint of Wikidata.org (Wikidata Query Service)] | * '''[[Wikidata]]''' [https://query.wikidata.org/ Public SPARQL endpoint of Wikidata.org (Wikidata Query Service)] | ||
''' Commercial data sources ''' | |||
* [https://developers.google.com/maps/documentation/distance-matrix/overview 'Google Maps Platform -> Web Services -> Distance Matrix API' at developers.google.com] | |||
'''Lists of Sparql endpoints''' | '''Lists of Sparql endpoints''' | ||
| Line 86: | Line 121: | ||
* [[Consumerium Commons]] | * [[Consumerium Commons]] | ||
* Upcoming federated [[w:GNU MediaGoblin]] | * Upcoming federated [[w:GNU MediaGoblin]] | ||
''' Free social media ''' | |||
* [[Consumium free social]] | |||
Latest revision as of 12:11, 30 July 2020
This article is for choosing good systems (technical) for The Consumerium Effort - Enhancing Consumer Informedness
Factors affecting systems selections[edit | edit source]
- Copyleft 'free to modify' and 'free of cost' software is strongly preferred over other solutions.
- Minimal modifications made (MMM) is an preference as well. This will ensure that we don't get swamped down on patching ad-hoccy, unstandard aged solutions to keep 'em working with the new stuff.
Encodings[edit | edit source]
- Integers w:BASE10 for integers, size depends on available solutions. Escape integer ceiling by bluntly encoding the integer as string of BASE-10 chars, dirty hack but could do the trick.
- Condense URL encoding w:BASE64 and 64 possible characters, exactly 6 bits from w:ISO/IEC 646 (hard compatible with 7-bit ASCII)
- w:UTF-8 encoded w:Unicode for text. UTF-8 is efficient and allows 2^32 different characters.. unless escaped to 40-bit in future revision
Copyright and copyleft protections system selections[edit | edit source]
- Copyleft licensing for content, code and Consumerium Commons media repository
- Automatic plagiarism testing which usually probably also translates to copyvio
- Disqualify and rapidly remove all copyvio
- Discourage piratism. Piratism on copyleft much stronger than on copyright.
Data identifiers[edit | edit source]
- GTIN (contains EAN and UPC spaces) identifies uniquely a product to the systems. Also contains the country of the marketer (2 first digits of EAN) and address of the marketers address space. It may be possible to acquire namespace size from examining these values. Possible values are 100 ids and 10,000 ids (EAN).
- ISO identifiers exist for at least languages, countries, country-subdivision areas, securities and many more things w:List of International Organization for Standardization standards
- UN/LOCODE (United Nations Code for Trade and Transport Locations) identifies globally nearly all locations of interest to logistics.
Product and service classifications[edit | edit source]
Product and service classifications
Databases[edit | edit source]
Main article: Databases
Currently using all major free software RDBMS:
Considered databases
- Graph databases and Subject-predicate-object databases seem to perform the same thing just with different architecture and suitability for certain searches. See Talk:Databases for more on this.
Candidates for testing
- All copyleft licensed triplestore and more advanced i.e. graph
Languages[edit | edit source]
Programming languages
Currently used languages
Considered languages / languages of interest
Frameworks[edit | edit source]
Frameworks for glueing into various systems
- Semantic MediaWiki compatibilities: PHP, Semantic Forms, enter data in wiki code -> RDF <-> Graph -> Outputs wikirenders and public SPARQL endpoints
Software[edit | edit source]
Current server software
- Operating system: Debian GNU/Linux
- Hypervisors (by hosting guys) Xen and KVM
- Webservers: Nginx and Apache
- MediaWiki (PHP, MariaDB and more)
- w:diaspora (software) (Ruby on Rails ,Ruby, MariaDB, Node.js and more) - role: consumer protection oriented free social media since 2010
- w:Draft:Hubzilla (PHP, MariaDB) - Excellent channel-based distributed free social media where subscription is to the whole network allowing nomadic and double citizenship. A model the others should follow suit.
- w:Friendica (PHP, MariaDB, fully contained in database w/ images stored as blobs. The least learning curve free social media for facebook refugees. Initiated by the same guy as Hubzilla.
- w:GNU social MariaDB - Microblogging platform
- w:GNU MediaGoblin (Python2 (Installing Python3 was impossible last time I looked at it, theoretically compatible with Py3 but broken because outside code supplier.) and PostgreSQL) Role: Copyleft code, content and codecs based media repository.
Future software
- Once MaidSafe goes persistent stage Consumerium will be deploying many nodes in order to earn SafeCoin to pay for future storage needs. Provision of information services to the denizens of the SAFE Network is also naturally a priority but basically patching a normal webapp onto the SAFE Network is straightforward and easy. Also: commiemism.
...
APIs[edit | edit source]
APIs to communicate with:
- MediaWiki API
- Free social media ones
- Commercial social media ones
Data sources[edit | edit source]
Data sources: Own own data source
- Semantic MediaWiki well thought out and set up on implementation wiki (c.consumerium.org)
- Development Wiki - User database, userspace sharing or importing and Unified Login are being looked at as options.
Data sources: Semantic web
The w:semantic web at large and the w:semantic wikis are treasure troves of interesting data which Consumerium should tap into early on to avoid redundant manual replication and duplicate data insertions.
- Other copyleft Semantic MediaWikis
- DBpedia Public SPARQL endpoint of DBpedia.org and Projects and efforts using DBpedia are very relevant.
- Wikidata Public SPARQL endpoint of Wikidata.org (Wikidata Query Service)
Commercial data sources
Lists of Sparql endpoints
Media sources[edit | edit source]
- w:Wikimedia Commons
- Consumerium Commons
- Upcoming federated w:GNU MediaGoblin
Free social media