diff --git a/docs/automate/cmemc-command-line-interface/command-reference/package/index.md b/docs/automate/cmemc-command-line-interface/command-reference/package/index.md new file mode 100644 index 000000000..afa1c86e2 --- /dev/null +++ b/docs/automate/cmemc-command-line-interface/command-reference/package/index.md @@ -0,0 +1,190 @@ +--- +title: "cmemc: Command Group - package" +description: "List, (un)install, export, create, or inspect packages." +icon: material/shopping +tags: + - cmemc + - Package +--- +# package Command Group + + +List, (un)install, export, create, or inspect packages. + + +## package create + +Initialize an empty package directory with a minimal manifest. + +```shell-session title="Usage" +$ cmemc package create [OPTIONS] PACKAGE_ID +``` + + + + + +??? info "Options" + ```text + + --name TEXT The package name in english. Defaults to package ID. + --version TEXT Semantic version identifier string of the package, but + limited to proper releases. [default: 0.0.1] + --description TEXT The package description in english. [default: This is + the first version of a wonderful eccenca Corporate + Memory package 🤓] + ``` + +## package inspect + +Inspect the manifest of a package. + +```shell-session title="Usage" +$ cmemc package inspect [OPTIONS] PACKAGE_PATH +``` + + + + + +??? info "Options" + ```text + + --key TEXT Get a specific key only from the manifest. + --raw Outputs raw JSON. + ``` + +## package list + +List installed packages. + +```shell-session title="Usage" +$ cmemc package list [OPTIONS] +``` + + + + + +??? info "Options" + ```text + + --filter ... Filter installed packages by one of the following + filter names and a corresponding value: type, name, + id. + --id-only Lists only package IDs. This is useful for piping + the IDs into other commands. + --raw Outputs raw JSON. + ``` + +## package install + +Install packages. + +```shell-session title="Usage" +$ cmemc package install [OPTIONS] [PACKAGE_ID] +``` + + + + +This command installs a package either from the marketplace or from local package archives (.cpa) or directories. + + + +??? info "Options" + ```text + + -i, --input PATH Install a package from a package archive (.cpa) or + directory. + --replace Replace (overwrite) existing package version, if present. + ``` + +## package uninstall + +Uninstall installed packages. + +```shell-session title="Usage" +$ cmemc package uninstall [OPTIONS] [PACKAGE_ID] +``` + + + + + +??? info "Options" + ```text + + --filter ... Filter installed packages by one of the following + filter names and a corresponding value: type, name, + id. + -a, --all Uninstall all packages. This is a dangerous option, + so use it with care. + ``` + +## package export + +Export installed packages to package directories. + +```shell-session title="Usage" +$ cmemc package export [OPTIONS] [PACKAGE_ID] +``` + + + + + +??? info "Options" + ```text + + --filter ... Filter installed packages by one of the following + filter names and a corresponding value: type, name, + id. + -a, --all Export all installed packages. + --replace Replace (overwrite) existing files, if present. + ``` + +## package build + +Build a package archive from a package directory. + +```shell-session title="Usage" +$ cmemc package build [OPTIONS] PACKAGE_DIRECTORY +``` + + + + +This command processes a package directory, validates its content including the manifest, and creates a versioned Corporate Memory package archive (.cpa) with the following naming convention: {package_id}-v{version}.cpa + +Package archives can be published to the marketplace using the `package publish` command. + + + +??? info "Options" + ```text + + --version TEXT Set the package version. + --replace Replace package archive, if present. + --output-dir DIRECTORY Create the package archive in a specific directory. + [default: .] + ``` + +## package publish + +Publish a package archive to the marketplace server. + +```shell-session title="Usage" +$ cmemc package publish [OPTIONS] PACKAGE_ARCHIVE +``` + + + + + +??? info "Options" + ```text + + --marketplace-url TEXT Alternative Marketplace URL. + ``` + diff --git a/docs/develop/.pages b/docs/develop/.pages index a11a2dfb9..fa1d34c03 100644 --- a/docs/develop/.pages +++ b/docs/develop/.pages @@ -2,6 +2,7 @@ nav: - Develop: index.md - Accessing Graphs with Java Applications: accessing-graphs-with-java-applications - Python Plugins: python-plugins + - Marketplace Packages: marketplace-packages - cmempy - Python API: cmempy-python-api - cmemc - Python Scripts: cmemc-scripts - Build (DataIntegration) APIs: dataintegration-apis diff --git a/docs/develop/marketplace-packages/.pages b/docs/develop/marketplace-packages/.pages new file mode 100644 index 000000000..1f4c61800 --- /dev/null +++ b/docs/develop/marketplace-packages/.pages @@ -0,0 +1,4 @@ +nav: + - Marketplace Packages: index.md + - Installation and Management: installation + - Development and Publication: development diff --git a/docs/develop/marketplace-packages/development/index.md b/docs/develop/marketplace-packages/development/index.md new file mode 100644 index 000000000..98af76a4c --- /dev/null +++ b/docs/develop/marketplace-packages/development/index.md @@ -0,0 +1,198 @@ +--- +title: "Marketplace Packages: Development and Publication" +icon: material/code-json +tags: + - Package +--- +# Development and Publication of Marketplace Packages + +## Introduction + +Marketplace Packages are archives that bundle content, functionality, and configuration from Corporate Memory for sharing and reuse. + +Each package has its own release cycle. +Packages can be installed and uninstalled during runtime. + +In order to support the development and publication of Marketplace Packages, we published a [package-template](https://github.com/eccenca/cmem-package-template). +Please have a look at this template to get started. + +This page gives an overview of the concepts you need to understand in order to develop packages. + +## Package Structure + +Use the [package-template](https://github.com/eccenca/cmem-package-template) to create the boilerplate for a package repository. + +### License + +!!! info "No publication without license" + + Packages without a license declaration cannot be published to a Corporate Memory Marketplace Server. + +Our template will bootstrap your package with an _Apache License 2.0 ([`Apache-2.0`](https://spdx.org/licenses/Apache-2.0.html))_. +See if you need a different license. +You can remove a license entirely; however, a package that does not declare a license cannot be published. + +### Manifest + +The `manifest.json` is the central package definition. +It contains all relevant package metadata and describes the package contents. +It is used to present package details and contents to the `inspect` command, to install, configure and uninstall all parts of a package. + +#### Metadata + +`package_type` +: `project` + : A package that may ship any content, mainly intended to contain Build projects, (instance/data) graphs, SHACL shapes, workspace configuration, query catalogs, etc. + + `vocabulary` + : A package that is supposed to contribute vocabulary / ontology contents, such as `rdf:`, `org:`, `sso:`, etc. Such a package may contain multiple vocabularies / ontologies. Packaging related SHACL shapes is reasonable, too. + +`package_id` +: Unique package identifier + +`package_version` +: Semantic version identifier string of the package, but limited to proper releases. + +`metadata.name` +: The package name in English. + +`metadata.description` +: The package description in English. + +`metadata_comment` +: A maintainer or publisher comment. + +#### Files + +A package can contain graphs or Build projects. These contents are referenced in the `manifest.json`. + +##### Graphs + +Use the following structure to include a graph. +`register_as_vocabulary` and `import_into` are optional instructions. +We suggest to organize graphs in a respective sub-folder (here `graphs/`), but this is up to you: + +```json +"files": [ + … + { + "file_type": "graph", + "file_path": "graphs/file.ttl", + "graph_iri": "http://www.example.org/file/", + "register_as_vocabulary": true, + "import_into": [ + "http://www.example.org/integration_graph/" + ] + }, + … +] +``` + +##### Projects + +Use the following structure to include a project. +We suggest to organize projects in a respective sub-folder (here `projects/`), but this is up to you: + +```json +"files": [ + … + { + "file_type": "project", + "file_path": "projects/my-build-project.zip", + "project_id": "my-build-project" + }, + … +] +``` + +#### Dependencies + +Dependencies to other (vocabulary) packages or to Python plugins can be declared in the `copier copy` answers. +The dependencies are added to the `manifest.json` as described in the next sections. + +##### Python Plugin Packages + +Use the following to declare a dependency to a Python plugin: + +```json +"dependencies": [ + … + { + "dependency_type": "python-package", + "pypi_id": "cmem-plugin-pyshacl" + }, + … +] +``` + +##### (Vocabulary) Packages + +Use the following to declare a dependency to a (vocabulary) package: + +```json +"dependencies": [ + … + { + "dependency_type": "vocabulary", + "package_id": "w3c-rdfs-vocab" + } + … +] +``` + +## Package Development Cycle + +!!! info "`cmemc package` reference" + + Refer to [TODO: link](./) for the complete reference of the `package` command group. + +Some packages are simply wrapping existing artifacts into a managed structure (e.g. existing vocabulary/ontology). + +Most (solution) package development and evolution will be a back and forth between a package repository (making changes to `manifest.json` in terms of adding/removing dependencies, graph files, or Build project files) and a Corporate Memory (package development) instance. + +The following pages give an overview about this feature: + +![Corporate Memory Marketplace Package Lifecycle](../mpp-lifecycle.svg){ width="50%" } + +### Install (local) Packages + +Use the following command to install a local package folder content (or built `.cpa` file) to a Corporate Memory (package development) instance. + +```sh +cmemc package install --input PATH +``` + +Make changes to graphs, configuration, or Build projects as needed. +Newly created or imported graphs or Build projects need to be registered in `manifest.json` so they will be fetched by `export`. + +### Export Contents into a Package + +Use the following command to export the file artifacts declared in `manifest.json` from a Corporate Memory (package development) instance to a local package folder. + +```sh +cmemc package export PACKAGE_ID +``` + +Run this to initially populate package contents from a solution configuration. You can also use it to update contents after making changes on your Corporate Memory (package development) instance, capturing them for building and releasing as a Marketplace Package. + +### Inspect Packages + +Review and verify the contents of a package with the following command: + +```sh +cmemc package inspect PACKAGE_PATH +``` + +### Build Packages + +During development you can install a package from a local path (plain folder or a `.cpa` package) using the `cmemc package install --input PATH` command. + +Use the `cmemc package build` command. +This will build a package archive from a package directory. + +This command processes a package directory, validates its content including the manifest, and creates a versioned Corporate Memory package archive (`.cpa`) with the following naming convention: `{package_id}-v{version}.cpa`. + +### Publish Packages + +Package archives can be published to the Marketplace Server using the `cmemc package publish` command. +After being published packages can be found and installed directly from the Marketplace Server (potential users do not need to have the local package folder or `.cpa` file available). diff --git a/docs/develop/marketplace-packages/index.md b/docs/develop/marketplace-packages/index.md new file mode 100644 index 000000000..e00af74b5 --- /dev/null +++ b/docs/develop/marketplace-packages/index.md @@ -0,0 +1,49 @@ +--- +status: new +title: "Marketplace Packages: Overview" +icon: material/shopping +tags: + - Package +hide: + - toc +--- + +# Marketplace Packages + +Starting with version 26.1, we support the creation and use of Marketplace Packages. + +Marketplace Packages bundle everything for a specific Corporate Memory–based solution or project into a single shareable, managed artifact: + +- Vocabularies / Ontologies +- (SKOS) Taxonomies +- (Instance / Data) Graphs +- Build Projects +- Dependencies on + - [python-plugins](../python-plugins/index.md) + - (other) Marketplace Packages + +The lifecycle of a Corporate Memory Marketplace Package is shown in the following flowchart. + +![Corporate Memory Marketplace Package Lifecycle](mpp-lifecycle.svg){ width="50%" } + +The following pages give an overview of this feature: + +
+ +- :material-download-circle-outline: [Installation and Management](installation/index.md) + + --- + + Intended for Linked Data Experts and Corporate Memory Admins, this page outlines how to (un)install and manage Marketplace Packages. + + This section discusses the lifecycle commands and stages `install`, `list` and `uninstall`. + +- :material-code-json: [Development and Publication](development/index.md) + + --- + + Intended for Linked Data Experts, Consultants, and Partners, this page gives an overview of how to start developing and publish Marketplace Packages. + + This section discusses the lifecycle commands and stages `copier copy`, _Package Definition and Release_, `inspect`, `install --input PATH` (from local), _Solution Development and Configuration_, `export`, `build`, and `publish`. + +
diff --git a/docs/develop/marketplace-packages/installation/index.md b/docs/develop/marketplace-packages/installation/index.md new file mode 100644 index 000000000..24d6002d2 --- /dev/null +++ b/docs/develop/marketplace-packages/installation/index.md @@ -0,0 +1,51 @@ +--- +title: "Marketplace Packages: Installation and Management" +icon: material/download-circle-outline +tags: + - Package +--- +# Installation and Management of Marketplace Packages + +## Introduction + +Marketplace Packages can be installed directly from a Corporate Memory Marketplace Server (e.g. [https://eccenca.market](https://eccenca.market)), or from local **C**orporate Memory **P**ackage **A**rchives (`.cpa` files). + +This page describes how to install, list, and uninstall packages using `cmemc`. + +!!! info "`cmemc package` reference" + + The [cmemc package command group](../../../automate/cmemc-command-line-interface/command-reference/package/index.md) contains of a needed commands to support + the complate package lifecycle. + +## Install Packages + +Use the following command to install a package from a Marketplace Server: + +``` sh +cmemc package install PACKAGE_ID +``` + +For installing local package archives (`.cpa` files) or package directories, use the `--input` option: + +``` sh +cmemc package install --input PATH +``` + +## List Packages + +Use the following command to list all installed packages: + +``` sh +cmemc package list +``` + +## Uninstall Packages + +Use the following command to uninstall a package: + +``` sh +cmemc package uninstall PACKAGE_ID +``` + +This removes all package contents from the Corporate Memory instance, including graphs and Build projects that were installed as part of the package. + diff --git a/docs/develop/marketplace-packages/mmd.txt b/docs/develop/marketplace-packages/mmd.txt new file mode 100644 index 000000000..fc2ee5b30 --- /dev/null +++ b/docs/develop/marketplace-packages/mmd.txt @@ -0,0 +1,13 @@ +# mermaid + +## live editor + +https://mermaid.live/edit#pako:eNqFU12L2zAQ_CuLIHVb7DSxL3YiykFJHnpwgaMfL636oNgbR0SWjCIdlwv575WcuA6FUvxg7WhmZ3clnUipKySUJEnClBVWIoWlNq023CKssdHmCGtu9mhbyUuEJ17ueY3wKLZYHkuJTHXa0egklLAUThBJXT_iM8qIQlThxtVRDJHdYYMBUeis4TKCM5xHI6YO1lutBK8Nb5LnlKnwdSAw0tutcBvSC60-bsw9VxV8QYn8gCF8K3XJ5TtGgB-g3dc3creR4rDDqqPd9NGT3WYgj8uW9_0NaeENLB8-LFdXhecMiq9aulCTL-8ZQlFLrbai7sR_T7HXN9gMCR7CSsprff-QCHUI84Ve9F2JXnYhOCXCzH6-_wVJch8mABRK3Qo04XdkKkDDlpe3WFqmQsMBDkVd8JAWQjEeF6p1Fp4-ffs8JPg_s2MMXvjiW7I3CbwlhY0Tshr8_TF4sL0cVmglAGHD-wxuLPR9A3uyHbDQPQ2j6MkkJrURFaHWOIxJg6bhISQnpsCPsbuQjFC_vF5JRpg6e1nL1Q-tm15ptKt3hG65PPjItdVwX_-gBlWFZqmdsoSmWZp1WQg9kRdCs2k-nswWxSy7mxaLfF7MYnIkdDpPx5NJUeR5kWZ5ls7PMXntfCfjYpIWd9lsvkizrJjli5hgJaw268tz7V7t-TfNwDqo + +## cli + +https://github.com/mermaid-js/mermaid-cli + +```sh +mmdc -i mpp-lifecycle.mmd -o mpp-lifecycle.svg -b transparent +``` diff --git a/docs/develop/marketplace-packages/mpp-lifecycle.mmd b/docs/develop/marketplace-packages/mpp-lifecycle.mmd new file mode 100644 index 000000000..22960e9d9 --- /dev/null +++ b/docs/develop/marketplace-packages/mpp-lifecycle.mmd @@ -0,0 +1,25 @@ +--- +#title: Corporate Memory Marketplace Package Lifecycle +--- +%%{init: { 'logLevel': 'debug', 'theme': 'neutral' } }%% +stateDiagram-v2 + + +state "Package Definition
and Release
(local)" as pkg +state "Published
(Marketplace)" as pub +state ".cpa Package
(local & CI/CD)" as cpa +state "Solution Dev and Config
(Corporate Memory)" as cmem +state "Installed
(Corporate Memory)" as ins +%% state "Uninstalled" as uni + +[*] --> pkg : copier copy +pkg --> pkg : inspect +cpa --> cmem : install
--input PATH +pkg --> cmem : install
--input PATH +cmem --> pkg : export +pkg --> cpa : build +cpa --> pub : publish + +pub --> ins : install +ins --> ins : list +ins --> [*] : uninstall diff --git a/docs/develop/marketplace-packages/mpp-lifecycle.svg b/docs/develop/marketplace-packages/mpp-lifecycle.svg new file mode 100644 index 000000000..a6835805a --- /dev/null +++ b/docs/develop/marketplace-packages/mpp-lifecycle.svg @@ -0,0 +1 @@ +

copier copy

inspect

install
--input PATH

install
--input PATH

export

build

publish

install

list

uninstall

Package Definition
and Release
(local)

Published
(Marketplace)

.cpa Package
(local & CI/CD)

Solution Dev and Config
(Corporate Memory)

Installed
(Corporate Memory)

 \ No newline at end of file diff --git a/docs/develop/python-plugins/installation/index.md b/docs/develop/python-plugins/installation/index.md index 538538263..770665282 100644 --- a/docs/develop/python-plugins/installation/index.md +++ b/docs/develop/python-plugins/installation/index.md @@ -8,7 +8,7 @@ tags: # Installation and Usage of Python Plugins Plugins are a released as parts of Python packages. -They can but do not need to be open-source and published on [pypi.org](https://pypi.org/search/?q=%22cmem-plugin-%22) (a widely used Python Package Index). One package can contain of multiple plugins. +They can but do not need to be open-source and published on [pypi.org](https://pypi.org/search/?q=%22cmem-plugin-%22) (a widely used Python Package Index). One package can contain multiple plugins. ## Installation