diff --git a/salad/README.rst b/salad/README.rst index 7901a76..d428770 100644 --- a/salad/README.rst +++ b/salad/README.rst @@ -1,23 +1,26 @@ -|Build Status| |Build status| +|Linux Build Status| |Code coverage| |CII Best Practices| -.. |Build Status| image:: https://img.shields.io/travis/common-workflow-language/schema_salad/master.svg?label=unix%20build - :target: https://travis-ci.org/common-workflow-language/schema_salad -.. |Build status| image:: https://img.shields.io/appveyor/ci/mr-c/schema-salad/master.svg?label=windows%20build - :target: https://ci.appveyor.com/project/mr-c/schema-salad/branch/master +.. |Linux Build Status| image:: https://github.com/common-workflow-language/schema-salad/actions/workflows/ci-tests.yml/badge.svg?branch=main + :target: https://github.com/common-workflow-language/schema-salad/actions/workflows/ci-tests.yml +.. |Code coverage| image:: https://codecov.io/gh/common-workflow-language/schema_salad/branch/main/graph/badge.svg + :target: https://codecov.io/gh/common-workflow-language/schema_salad +.. |CII Best Practices| image:: https://bestpractices.coreinfrastructure.org/projects/1867/badge + :target: https://bestpractices.coreinfrastructure.org/projects/1867 Schema Salad ------------ -Salad is a schema language for describing JSON or YAML structured linked data -documents. Salad is based originally on JSON-LD_ and the Apache Avro_ data -serialization system. +Salad is a schema language for describing JSON or YAML structured +linked data documents. Salad schema describes rules for +preprocessing, structural validation, and hyperlink checking for +documents described by a Salad schema. Salad supports rich data +modeling with inheritance, template specialization, object +identifiers, object references, documentation generation, code +generation, and transformation to RDF_. Salad provides a bridge +between document and record oriented data modeling and the Semantic +Web. -Salad schema describes rules for preprocessing, structural validation, and link -checking for documents described by a Salad schema. Salad features for rich -data modeling such as inheritance, template specialization, object identifiers, -object references, documentation generation, and transformation to RDF_. Salad -provides a bridge between document and record oriented data modeling and the -Semantic Web. +The Schema Salad library is Python 3.6+ only. Usage ----- @@ -25,21 +28,142 @@ Usage :: $ pip install schema_salad + +To install from source:: + + git clone https://github.com/common-workflow-language/schema_salad + cd schema_salad + python3 setup.py install + +Commands +-------- + +Schema salad can be used as a command line tool or imported as a Python module:: + $ schema-salad-tool usage: schema-salad-tool [-h] [--rdf-serializer RDF_SERIALIZER] - [--print-jsonld-context | --print-doc | --print-rdfs | --print-avro | --print-rdf | --print-pre | --print-index | --print-metadata | --version] - [--strict | --non-strict] - [--verbose | --quiet | --debug] - schema [document] + [--print-jsonld-context | --print-rdfs | --print-avro + | --print-rdf | --print-pre | --print-index + | --print-metadata | --print-inheritance-dot + | --print-fieldrefs-dot | --codegen language + | --print-oneline] + [--strict | --non-strict] [--verbose | --quiet + | --debug] + [--version] + [schema] [document] + $ python >>> import schema_salad -To install from source:: +Validate a schema:: + + $ schema-salad-tool myschema.yml + +Validate a document using a schema:: + + $ schema-salad-tool myschema.yml mydocument.yml + +Generate HTML documentation:: + + $ schema-salad-tool myschema.yml > myschema.html + +Get JSON-LD context:: + + $ schema-salad-tool --print-jsonld-context myschema.yml mydocument.yml + +Convert a document to JSON-LD:: + + $ schema-salad-tool --print-pre myschema.yml mydocument.yml > mydocument.jsonld + +Generate Python classes for loading/generating documents described by the schema:: + + $ schema-salad-tool --codegen=python myschema.yml > myschema.py + +Display inheritance relationship between classes as a graphviz 'dot' file and +render as SVG:: + + $ schema-salad-tool --print-inheritance-dot myschema.yml | dot -Tsvg > myschema.svg + + +Quick Start +----------- + +Let's say you have a 'basket' record that can contain items measured either by +weight or by count. Here's an example:: + + basket: + - product: bananas + price: 0.39 + per: pound + weight: 1 + - product: cucumbers + price: 0.79 + per: item + count: 3 + +We want to validate that all the expected fields are present, the +measurement is known, and that "count" cannot be a fractional value. +Here is an example schema to do that:: + + - name: Product + doc: | + The base type for a product. This is an abstract type, so it + can't be used directly, but can be used to define other types. + type: record + abstract: true + fields: + product: string + price: float + + - name: ByWeight + doc: | + A product, sold by weight. Products may be sold by pound or by + kilogram. Weights may be fractional. + type: record + extends: Product + fields: + per: + type: + type: enum + symbols: + - pound + - kilogram + jsonldPredicate: '#per' + weight: float + + - name: ByCount + doc: | + A product, sold by count. The count must be a integer value. + type: record + extends: Product + fields: + per: + type: + type: enum + symbols: + - item + jsonldPredicate: '#per' + count: int + + - name: Basket + doc: | + A basket of products. The 'documentRoot' field indicates it is a + valid starting point for a document. The 'basket' field will + validate subtypes of 'Product' (ByWeight and ByCount). + type: record + documentRoot: true + fields: + basket: + type: + type: array + items: Product + +You can check the schema and document in schema_salad/tests/basket_schema.yml +and schema_salad/tests/basket.yml:: + + $ schema-salad-tool basket_schema.yml basket.yml + Document `basket.yml` is valid - git clone https://github.com/common-workflow-language/schema_salad - cd schema_salad - python setup.py install - Documentation ------------- @@ -47,6 +171,7 @@ Documentation See the specification_ and the metaschema_ (salad schema for itself). For an example application of Schema Salad see the Common Workflow Language_. + Rationale --------- @@ -83,7 +208,7 @@ provides for robust support of inline documentation. .. _JSON-LD: http://json-ld.org .. _Avro: http://avro.apache.org -.. _metaschema: https://github.com/common-workflow-language/schema_salad/blob/master/schema_salad/metaschema/metaschema.yml +.. _metaschema: https://github.com/common-workflow-language/schema_salad/blob/main/schema_salad/metaschema/metaschema.yml .. _specification: http://www.commonwl.org/v1.0/SchemaSalad.html -.. _Language: https://github.com/common-workflow-language/common-workflow-language/blob/master/v1.0/CommandLineTool.yml +.. _Language: https://github.com/common-workflow-language/common-workflow-language/blob/main/v1.0/CommandLineTool.yml .. _RDF: https://www.w3.org/RDF/ diff --git a/salad/schema_salad/metaschema/field_name.yml b/salad/schema_salad/metaschema/field_name.yml index 44e95a2..60b9478 100644 --- a/salad/schema_salad/metaschema/field_name.yml +++ b/salad/schema_salad/metaschema/field_name.yml @@ -9,7 +9,7 @@ which are not part of the vocabulary are resolved using the following rules: - * If an field name URI begins with a namespace prefix declared in the + * If a field name URI begins with a namespace prefix declared in the document context (`@context`) followed by a colon `:`, the prefix and colon must be replaced by the namespace declared in `@context`. diff --git a/salad/schema_salad/metaschema/field_name_schema.yml b/salad/schema_salad/metaschema/field_name_schema.yml index 5089c4b..a494238 100644 --- a/salad/schema_salad/metaschema/field_name_schema.yml +++ b/salad/schema_salad/metaschema/field_name_schema.yml @@ -5,6 +5,7 @@ "$graph": [{ "name": "ExampleType", "type": "record", + "documentRoot": true, "fields": [{ "name": "base", "type": "string", diff --git a/salad/schema_salad/metaschema/ident_res.yml b/salad/schema_salad/metaschema/ident_res.yml index 86aefe6..3955f89 100644 --- a/salad/schema_salad/metaschema/ident_res.yml +++ b/salad/schema_salad/metaschema/ident_res.yml @@ -9,7 +9,7 @@ fragment identifier. It is resolved relative to the base URI by setting or replacing the fragment portion of the base URI. - * If an identifier URI contains `#` in some other position is a + * If an identifier URI contains `#` in some other position it is a relative URI with fragment identifier. It is resolved relative to the base URI by stripping the last path segment from the base URI and adding the identifier followed by the fragment. diff --git a/salad/schema_salad/metaschema/ident_res_schema.yml b/salad/schema_salad/metaschema/ident_res_schema.yml index 704116d..9e4a28b 100644 --- a/salad/schema_salad/metaschema/ident_res_schema.yml +++ b/salad/schema_salad/metaschema/ident_res_schema.yml @@ -5,6 +5,7 @@ "$graph": [{ "name": "ExampleType", "type": "record", + "documentRoot": true, "fields": [{ "name": "id", "type": "string", diff --git a/salad/schema_salad/metaschema/import_include.md b/salad/schema_salad/metaschema/import_include.md index 0ad06bf..84397f7 100644 --- a/salad/schema_salad/metaschema/import_include.md +++ b/salad/schema_salad/metaschema/import_include.md @@ -28,7 +28,7 @@ or resource fragment which does not exist or is not accessible. ### Import example -import.yml: +import.json: ``` { "hello": "world" @@ -36,12 +36,12 @@ import.yml: ``` -parent.yml: +parent.json: ``` { "form": { "bar": { - "$import": "import.yml" + "$import": "import.json" } } } @@ -83,7 +83,7 @@ which does not exist or is not accessible. ### Include example -parent.yml: +parent.json: ``` { "form": { diff --git a/salad/schema_salad/metaschema/link_res.yml b/salad/schema_salad/metaschema/link_res.yml index 9346f8a..a8e698e 100644 --- a/salad/schema_salad/metaschema/link_res.yml +++ b/salad/schema_salad/metaschema/link_res.yml @@ -1,7 +1,7 @@ - | ## Link resolution - The schema may designate one or more fields as link fields reference other + The schema may designate one or more fields as link fields that reference other objects. Processing must resolve links to either absolute URIs using the following rules: diff --git a/salad/schema_salad/metaschema/map_res.yml b/salad/schema_salad/metaschema/map_res.yml index bbcee48..636cc71 100644 --- a/salad/schema_salad/metaschema/map_res.yml +++ b/salad/schema_salad/metaschema/map_res.yml @@ -4,7 +4,7 @@ The schema may designate certain fields as having a `mapSubject`. If the value of the field is a JSON object, it must be transformed into an array of JSON objects. Each key-value pair from the source JSON object is a list - item, each list item must be a JSON objects, and the value of the key is + item, each list item must be a JSON object, and the value of the key is assigned to the field specified by `mapSubject`. Fields which have `mapSubject` specified may also supply a `mapPredicate`. diff --git a/salad/schema_salad/metaschema/metaschema_base.yml b/salad/schema_salad/metaschema/metaschema_base.yml index 67008b6..5bcbe49 100644 --- a/salad/schema_salad/metaschema/metaschema_base.yml +++ b/salad/schema_salad/metaschema/metaschema_base.yml @@ -173,4 +173,5 @@ $graph: _id: "sld:items" _type: "@vocab" refScope: 2 + typeDSL: True doc: "Defines the type of the array elements." diff --git a/salad/schema_salad/metaschema/salad.md b/salad/schema_salad/metaschema/salad.md index 68233c9..a931636 100644 --- a/salad/schema_salad/metaschema/salad.md +++ b/salad/schema_salad/metaschema/salad.md @@ -247,7 +247,7 @@ rules: * If the value of `jsonldPredicate` is `@id`, the field is an identifier field. - * If the value of `jsonldPredicate` is an object, and contains that + * If the value of `jsonldPredicate` is an object, and that object contains the field `_type` with the value `@id`, the field is a link field subject to [link validation](#Link_validation). diff --git a/salad/schema_salad/metaschema/sfdsl_res_src.yml b/salad/schema_salad/metaschema/sfdsl_res_src.yml index dc4c5f3..c12869c 100644 --- a/salad/schema_salad/metaschema/sfdsl_res_src.yml +++ b/salad/schema_salad/metaschema/sfdsl_res_src.yml @@ -5,8 +5,8 @@ }, { "secondaryFiles": { "pattern": ".bai?" -}, { +}}, { "secondaryFiles": { "pattern": ".bai?", "required": true -}] +}}] diff --git a/salad/schema_salad/metaschema/typedsl_res.yml b/salad/schema_salad/metaschema/typedsl_res.yml index c30c60b..de5d62b 100644 --- a/salad/schema_salad/metaschema/typedsl_res.yml +++ b/salad/schema_salad/metaschema/typedsl_res.yml @@ -5,7 +5,7 @@ following micro-DSL for schema salad types: * If the type ends with a question mark `?`, the question mark is stripped off and the type is expanded to a union with `null` - * If the type ends with square brackets `[]` it is expanded to an array with items of the preceeding type symbol + * If the type ends with square brackets `[]` it is expanded to an array with items of the preceding type symbol * The type may end with both `[]?` to indicate it is an optional array. * Identifier resolution is applied after type DSL expansion.