Systems selections: Difference between revisions

    From Consumerium development wiki R&D Wiki
    (→‎Data sources: + https://www.w3.org/wiki/SparqlEndpoints)
    (→‎Data sources: + Commercial data sources + 'Google Maps Platform -> Web Services -> Distance Matrix API' at developers.google.com)
     
    (23 intermediate revisions by the same user not shown)
    Line 1: Line 1:
    This article is for choosing good systems (technical) for The [[Consumerium]] Effort - Enhancing Consumer Informedness


    == Factors affecting database systems selections ==  
    == Factors affecting systems selections ==  


    * [[Copyleft]] 'free to modify' and 'free in cost' software is strongly preferred over other solutions.  
    * [[Copyleft]] 'free to modify' and 'free of cost' software is strongly preferred over other solutions.  
    * Minimal modifications required could be another preference as that means maintenance of the chosen solution is minimized in that aspect.
    * Minimal modifications made (MMM) is an preference as well. This will ensure that we don't get swamped down on patching ad-hoccy, unstandard aged solutions to keep 'em working with the new stuff.
    ----
    ----
    == Encodings ==
    == Encodings ==
    Line 10: Line 11:
    * [[w:UTF-8]] encoded [[w:Unicode]] for text. UTF-8 is efficient and allows 2^32 different characters.. unless escaped to 40-bit in future revision
    * [[w:UTF-8]] encoded [[w:Unicode]] for text. UTF-8 is efficient and allows 2^32 different characters.. unless escaped to 40-bit in future revision


    == Languages ==
    == Copyright and copyleft protections system selections ==
    Programming languages
    * [[Copyleft]] [[licensing]] for content, code and [[Consumerium Commons]] media repository
    * Automatic plagiarism testing which usually probably also translates to copyvio
    * Disqualify and rapidly remove all copyvio
    * Discourage piratism. Piratism on copyleft much stronger than on copyright.


    ''' Currently used languages '''
    == Data identifiers ==
    * [[w:PHP]] (5)  
    * [[GTIN]] (contains [[EAN]] and [[UPC]] spaces) identifies uniquely a [[product]] to the systems. Also contains the country of the [[marketer]] (2 first digits of EAN) and address of the marketers address space. It may be possible to acquire namespace size from examining these values. Possible values are 100 ids and 10,000 ids (EAN).
    * [[w:Ruby (programming language)]]
    * [[ISO]] identifiers exist for at least [[languages]], [[countries]], country-subdivision [[areas]], [[securities]] and many more things [[w:List of International Organization for Standardization standards]]
    * [[w:Python (programming language)]]
    * [[UN/LOCODE]] (United Nations Code for Trade and Transport Locations) identifies globally nearly all locations of interest to [[logistics]].


    '''Considered languages / languages of interest'''
    == Product and service classifications ==
    '''[[Product and service classifications]]'''


    == Databases ==
    == Databases ==


    === Current databases ===
    '''Main article: [[Databases]]'''
    '''Main article: [[Database]]'''


    Currently using all major free software RDBMS:  
    Currently using all major free software RDBMS:  
    Line 32: Line 36:
    '''Considered databases'''
    '''Considered databases'''
    * [[Databases#Graph databases|Graph databases]] and [[Database#Subject-predicate-object databases|Subject-predicate-object databases]] seem to perform the same thing just with different architecture and suitability for certain searches. See [[Talk:Databases]] for more on this.
    * [[Databases#Graph databases|Graph databases]] and [[Database#Subject-predicate-object databases|Subject-predicate-object databases]] seem to perform the same thing just with different architecture and suitability for certain searches. See [[Talk:Databases]] for more on this.
    '''Candidates for testing'''
    * All copyleft licensed triplestore and more advanced i.e. graph
    ----
    == Languages ==
    Programming languages
    ''' Currently used languages '''
    * [[w:PHP]] (5)
    * [[w:Ruby (programming language)]]
    * [[w:Python (programming language)]]
    '''Considered languages / languages of interest'''


    ----
    ----
    Line 37: Line 54:
    == Frameworks ==
    == Frameworks ==


    '''Frameworks'''
    '''Frameworks''' for glueing into various systems
    * [[Semantic MediaWiki]]
    * [[Semantic MediaWiki]] compatibilities: PHP, Semantic Forms, enter data in [[wiki code]] -> RDF <-> Graph -> Outputs wikirenders and public [[SPARQL]] endpoints


    ----
    ----
    == Software ==
    == Software ==
    '''Main article: [[Software]]'''
    '''Current server software'''
     
    * [[w:Operating system|Operating system]]: [[w:Debian|Debian]] [[w:GNU/Linux|GNU/Linux]]
    * [[w:Hypervisor|Hypervisor]]s (by hosting guys) [[w:Xen|Xen]] and [[w:Kernel-based Virtual Machine|KVM]]
    * Webservers: [[w:Nginx|Nginx]] and [[w:Apache HTTP Server|Apache]]


    === Current software ===
    * [[MediaWiki]] ([[w:PHP|PHP]], [[w:MariaDB|MariaDB]] and more)
    * [[w:diaspora (software)]]  ([[w:Ruby on Rails|Ruby on Rails]] ,[[w:Ruby (programming language)|Ruby]], MariaDB, [[w:Node.js|Node.js]] and more) - role: [[consumer protection]] oriented [[free social media]] since 2010
    * [[w:Draft:Hubzilla]] (PHP, MariaDB) - Excellent channel-based distributed free social media where subscription is to the whole network allowing nomadic and double citizenship. A model the others should follow suit.
    * [[w:Friendica]] (PHP, MariaDB, fully contained in database w/ images stored as blobs. The least learning curve free social media for [[facebook refugees]]. Initiated by the same guy as Hubzilla.
    * [[w:GNU social]] MariaDB - [[w:Microblogging|Microblogging]] platform
    * [[w:GNU MediaGoblin]] (Python2 (Installing Python3 was impossible last time I looked at it, theoretically compatible with Py3 but broken because outside code supplier.) and [[w:PostgreSQL|PostgreSQL]]) Role: [[Copyleft]] [[code]], [[content]] and [[codecs]] based media repository.


    * [[MediaWiki]]
    ''' Future software '''
    * [[w:diaspora (software)]]*
    * Once [[MaidSafe]] goes persistent stage Consumerium will be deploying many nodes in order to earn SafeCoin to pay for future storage needs. Provision of information services to the denizens of the SAFE Network is also naturally a priority but basically patching a normal webapp onto the SAFE Network is straightforward and easy. Also: commiemism.
    * [[w:Draft:Hubzilla]] (pauvre ecrivan.. les deletionists vous savez..)
    * [[w:Friendica]]
    * [[w:GNU social]]
    * [[w:GNU MediaGoblin]]
    ...
    ...


    Line 60: Line 83:
    APIs to communicate with:
    APIs to communicate with:


    * [https://www.mediawiki.org/wiki/API:Main_page MediaWiki API]
    * Free social media ones
    * Free social media ones
    * Commercial social media ones


    ----
    ----


    == Data sources ==
    == Data sources ==
    === Data sources: own ===
    ''' Data sources: Own own data source '''
    * [[Semantic MediaWiki]] well thought out and set up on implementation wiki (c.consumerium.org)
    * [[Semantic MediaWiki]] well thought out and set up on implementation wiki (c.consumerium.org)
    * [[Development Wiki]] - User database, userspace sharing or importing and Unified Login are being looked at as options.
    * [[Development Wiki]] - User database, userspace sharing or importing and Unified Login are being looked at as options.


    === Data sources: Semantic web ===
    ''' Data sources: Semantic web '''
    * Other copyleft Semantic MediaWikis
     
    * '''[[DBpedia]]''' [http://dbpedia.org/sparql Public SPARQL endpoint of DBpedia.org]
    The [[w:semantic web]] at large and the [[w:semantic wiki]]s are treasure troves of interesting data which Consumerium should tap into early on to avoid redundant manual replication and duplicate data insertions.
     
    * Other copyleft Semantic MediaWikis  
    ** [http://smw.referata.com/wiki/Special:BrowseData/Sites Semantic MediaWiki Community wiki list of SMW sites]
    ** [https://wikiapiary.com/wiki/Semantic_statistics Wikiapiary list of SMW sites]
    ** [http://wikiindex.org/Category:Semantic_MediaWiki Wikiindex.org category of SMWs]
     
    * '''[[DBpedia]]''' [http://dbpedia.org/sparql Public SPARQL endpoint of DBpedia.org] and [http://wiki.dbpedia.org/projects Projects and efforts using DBpedia] are very relevant.
    * '''[[Wikidata]]''' [https://query.wikidata.org/ Public SPARQL endpoint of Wikidata.org (Wikidata Query Service)]
    * '''[[Wikidata]]''' [https://query.wikidata.org/ Public SPARQL endpoint of Wikidata.org (Wikidata Query Service)]
    ''' Commercial data sources '''
    * [https://developers.google.com/maps/documentation/distance-matrix/overview 'Google Maps Platform ->  Web Services -> Distance Matrix API' at developers.google.com]


    '''Lists of Sparql endpoints'''
    '''Lists of Sparql endpoints'''
    Line 86: Line 121:
    * [[Consumerium Commons]]
    * [[Consumerium Commons]]
    * Upcoming federated [[w:GNU MediaGoblin]]
    * Upcoming federated [[w:GNU MediaGoblin]]
    ''' Free social media '''
    * [[Consumium free social]]

    Latest revision as of 12:11, 30 July 2020

    This article is for choosing good systems (technical) for The Consumerium Effort - Enhancing Consumer Informedness

    Factors affecting systems selections[edit | edit source]

    • Copyleft 'free to modify' and 'free of cost' software is strongly preferred over other solutions.
    • Minimal modifications made (MMM) is an preference as well. This will ensure that we don't get swamped down on patching ad-hoccy, unstandard aged solutions to keep 'em working with the new stuff.

    Encodings[edit | edit source]

    • Integers w:BASE10 for integers, size depends on available solutions. Escape integer ceiling by bluntly encoding the integer as string of BASE-10 chars, dirty hack but could do the trick.
    • Condense URL encoding w:BASE64 and 64 possible characters, exactly 6 bits from w:ISO/IEC 646 (hard compatible with 7-bit ASCII)
    • w:UTF-8 encoded w:Unicode for text. UTF-8 is efficient and allows 2^32 different characters.. unless escaped to 40-bit in future revision

    Copyright and copyleft protections system selections[edit | edit source]

    • Copyleft licensing for content, code and Consumerium Commons media repository
    • Automatic plagiarism testing which usually probably also translates to copyvio
    • Disqualify and rapidly remove all copyvio
    • Discourage piratism. Piratism on copyleft much stronger than on copyright.

    Data identifiers[edit | edit source]

    Product and service classifications[edit | edit source]

    Product and service classifications

    Databases[edit | edit source]

    Main article: Databases

    Currently using all major free software RDBMS:

    Considered databases

    Candidates for testing

    • All copyleft licensed triplestore and more advanced i.e. graph

    Languages[edit | edit source]

    Programming languages

    Currently used languages

    Considered languages / languages of interest


    Frameworks[edit | edit source]

    Frameworks for glueing into various systems


    Software[edit | edit source]

    Current server software

    Future software

    • Once MaidSafe goes persistent stage Consumerium will be deploying many nodes in order to earn SafeCoin to pay for future storage needs. Provision of information services to the denizens of the SAFE Network is also naturally a priority but basically patching a normal webapp onto the SAFE Network is straightforward and easy. Also: commiemism.

    ...


    APIs[edit | edit source]

    APIs to communicate with:

    • MediaWiki API
    • Free social media ones
    • Commercial social media ones

    Data sources[edit | edit source]

    Data sources: Own own data source

    • Semantic MediaWiki well thought out and set up on implementation wiki (c.consumerium.org)
    • Development Wiki - User database, userspace sharing or importing and Unified Login are being looked at as options.

    Data sources: Semantic web

    The w:semantic web at large and the w:semantic wikis are treasure troves of interesting data which Consumerium should tap into early on to avoid redundant manual replication and duplicate data insertions.

    Commercial data sources

    Lists of Sparql endpoints


    Media sources[edit | edit source]

    Copyleft:

    Free social media