Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/develop/.pages
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ nav:
- Develop: index.md
- Accessing Graphs with Java Applications: accessing-graphs-with-java-applications
- Python Plugins: python-plugins
- Marketplace Packages: marketplace-packages
- cmempy - Python API: cmempy-python-api
- cmemc - Python Scripts: cmemc-scripts
- Build (DataIntegration) APIs: dataintegration-apis
Expand Down
4 changes: 4 additions & 0 deletions docs/develop/marketplace-packages/.pages
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
nav:
- Marketplace Packages: index.md
- Installation and Management: installation
- Development and Publication: development
197 changes: 197 additions & 0 deletions docs/develop/marketplace-packages/development/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
---
title: "Marketplace Packages: Development and Publication"
icon: material/code-json
tags:
- Package
---
# Development and Publication of Marketplace Packages

## Introduction

Marketplace Packages are archives that bundle content, functionality, and configuration from Corporate Memory for sharing and reuse.

Each package has its own release cycle.
Packages can be installed and uninstalled during runtime.

In order to support the development and publication of Marketplace Packages, we published a [package-template](https://github.com/eccenca/cmem-package-template).
Please have a look at this template to get started.

This page gives an overview of the concepts you need to understand in order to develop packages.

## Package Structure

Use the [package-template](https://github.com/eccenca/cmem-package-template) to create the boilerplate for a package repository.

### License

!!! info "No publication without license"

Packages without a license declaration cannot be published to a Corporate Memory Marketplace Server.

Our template will bootstrap with an _Apache License 2.0 ([`Apache-2.0`](https://spdx.org/licenses/Apache-2.0.html))_. See <https://spdx.org/licenses/> if you need a different license.
You can remove a license entirely; however, a package that does not declare a license cannot be published.

### Manifest

The `manifest.json` is the central package definition.
It contains all relevant package metadata and describes the package contents.
It is used to present package details and contents to the `inspect` command<!-- or in the marketplace frontends-->, to install, configure and uninstall all parts of a package.

#### Metadata

`package_type`
: `project`
: A package that may ship any content, mainly intended to contain Build projects, (instance/data) graphs, SHACL shapes, workspace configuration, query catalogs, etc.

`vocabulary`
: A package that is supposed to contribute vocabulary / ontology contents, such as `rdf:`, `org:`, `sso:`, etc. Such a package may contain multiple vocabularies / ontologies. Packaging related SHACL shapes is reasonable, too.

`package_id`
: Unique package identifier

`package_version`
: Semantic version identifier string of the package, but limited to proper releases.

`metadata.name`
: The package name in English.

`metadata.description`
: The package description in English.

`metadata_comment`
: A maintainer or publisher comment.

#### Files

A package can contain graphs or Build projects. These contents are referenced in the `manifest.json`.

##### Graphs

Use the following structure to include a graph.
`register_as_vocabulary` and `import_into` are optional instructions.
We suggest to organize graphs in a respective sub-folder (here `graphs/`), but this is up to you:

```json
"files": [
{
"file_type": "graph",
"file_path": "graphs/file.ttl",
"graph_iri": "http://www.example.org/file/",
"register_as_vocabulary": true,
"import_into": [
"http://www.example.org/integration_graph/"
]
},
]
```

##### Projects

Use the following structure to include a project.
We suggest to organize projects in a respective sub-folder (here `projects/`), but this is up to you:

```json
"files": [
{
"file_type": "project",
"file_path": "projects/my-build-project.zip",
"project_id": "my-build-project"
},
]
```

#### Dependencies

Dependencies to other (vocabulary) packages or to Python plugins can be declared in the `copier copy` answers.
The dependencies are added to the `manifest.json` as described in the next sections.

##### Python Plugin Packages

Use the following to declare a dependency to a Python plugin:

```json
"dependencies": [
{
"dependency_type": "python-package",
"pypi_id": "cmem-plugin-pyshacl"
},
]
```

##### (Vocabulary) Packages

Use the following to declare a dependency to a (vocabulary) package:

```json
"dependencies": [
{
"dependency_type": "vocabulary",
"package_id": "w3c-rdfs-vocab"
}
]
```

## Package Development Cycle

!!! info "`cmemc package` reference"

Refer to [TODO: link](./) for the complete reference of the `package` command group.

Some packages are simply wrapping existing artifacts into a managed structure (e.g. existing vocabulary/ontology).

Most (solution) package development and evolution will be a back and forth between a package repository (making changes to `manifest.json` in terms of adding/removing dependencies, graph files, or Build project files) and a Corporate Memory (package development) instance.

The following pages give an overview about this feature:

![Corporate Memory Marketplace Package Lifecycle](../mpp-lifecycle.svg){ width="50%" }

### Install (local) Packages

Use the following command to install a local package folder content (or built `.cpa` file) to a Corporate Memory (package development) instance.

```sh
cmemc package install --input PATH
```

Make changes to graphs, configuration, or Build projects as needed.
Newly created or imported graphs or Build projects need to be registered in `manifest.json` so they will be fetched by `export`.

### Export Contents into a Package

Use the following command to export the file artifacts declared in `manifest.json` from a Corporate Memory (package development) instance to a local package folder.

```sh
cmemc package export PACKAGE_ID
```

Run this to initially populate package contents from a solution configuration. You can also use it to update contents after making changes on your Corporate Memory (package development) instance, capturing them for building and releasing as a Marketplace Package.

### Inspect Packages

Review and verify the contents of a package with the following command:

```sh
cmemc package inspect PACKAGE_PATH
```

### Build Packages

During development you can install a package from a local path (plain folder or a `.cpa` package) using the `cmemc package install --input PATH` command.

Use the `cmemc package build` command.
This will build a package archive from a package directory.

This command processes a package directory, validates its content including the manifest, and creates a versioned Corporate Memory package archive (`.cpa`) with the following naming convention: `{package_id}-v{version}.cpa`.

### Publish Packages

Package archives can be published to the Marketplace Server using the `cmemc package publish` command.
After being published packages can be found and installed directly from the Marketplace Server (potential users do not need to have the local package folder or `.cpa` file available).
49 changes: 49 additions & 0 deletions docs/develop/marketplace-packages/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
status: new
title: "Marketplace Packages: Overview"
icon: material/shopping
tags:
- Package
hide:
- toc
---

# Marketplace Packages

Starting with version 26.1, we support the creation and use of Marketplace Packages.

Marketplace Packages bundle everything for a specific Corporate Memory–based solution or project into a single shareable, managed artifact:

- Vocabularies / Ontologies
- (SKOS) Taxonomies
- (Instance / Data) Graphs
- Build Projects
- Dependencies on
- [python-plugins](../python-plugins/index.md)
- (other) Marketplace Packages

The lifecycle of a Corporate Memory Marketplace Package is shown in the following flowchart.

![Corporate Memory Marketplace Package Lifecycle](mpp-lifecycle.svg){ width="50%" }

The following pages give an overview of this feature:

<div class="grid cards" markdown>

- :material-download-circle-outline: [Installation and Management](installation/index.md)

---

Intended for Linked Data Experts and Corporate Memory Admins, this page outlines how to (un)install and manage Marketplace Packages.

This section discusses the lifecycle commands and stages `install`, `list` and `uninstall`.

- :material-code-json: [Development and Publication](development/index.md)

---

Intended for Linked Data Experts, Consultants, and Partners, this page gives an overview of how to start developing and publish Marketplace Packages.

This section discusses the lifecycle commands and stages `copier copy`, _Package Definition and Release_, `inspect`, `install --input PATH` (from local), _Solution Development and Configuration_, `export`, `build`, and `publish`.

</div>
49 changes: 49 additions & 0 deletions docs/develop/marketplace-packages/installation/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
title: "Marketplace Packages: Installation and Management"
icon: material/download-circle-outline
tags:
- Package
---
# Installation and Management of Marketplace Packages

## Introduction

Marketplace Packages can be installed directly from a Corporate Memory Marketplace Server or from local package archives (`.cpa` files).

This page describes how to install, list, and uninstall packages using `cmemc`.

!!! info "`cmemc package` reference"

Refer to [TODO: link](./) for the complete reference of the `package` command group.

## Install Packages

Use the following command to install a package from a Marketplace Server:

```sh
cmemc package install PACKAGE_ID
```

For installing local package archives (`.cpa` files) or package directories, use the `--input` option:

```sh
cmemc package install --input PATH
```

## List Packages

Use the following command to list all installed packages:

```sh
cmemc package list
```

## Uninstall Packages

Use the following command to uninstall a package:

```sh
cmemc package uninstall PACKAGE_ID
```

This removes all package contents from the Corporate Memory instance, including graphs and Build projects that were installed as part of the package.
13 changes: 13 additions & 0 deletions docs/develop/marketplace-packages/mmd.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# mermaid

## live editor

https://mermaid.live/edit#pako:eNqFU12L2zAQ_CuLIHVb7DSxL3YiykFJHnpwgaMfL636oNgbR0SWjCIdlwv575WcuA6FUvxg7WhmZ3clnUipKySUJEnClBVWIoWlNq023CKssdHmCGtu9mhbyUuEJ17ueY3wKLZYHkuJTHXa0egklLAUThBJXT_iM8qIQlThxtVRDJHdYYMBUeis4TKCM5xHI6YO1lutBK8Nb5LnlKnwdSAw0tutcBvSC60-bsw9VxV8QYn8gCF8K3XJ5TtGgB-g3dc3creR4rDDqqPd9NGT3WYgj8uW9_0NaeENLB8-LFdXhecMiq9aulCTL-8ZQlFLrbai7sR_T7HXN9gMCR7CSsprff-QCHUI84Ve9F2JXnYhOCXCzH6-_wVJch8mABRK3Qo04XdkKkDDlpe3WFqmQsMBDkVd8JAWQjEeF6p1Fp4-ffs8JPg_s2MMXvjiW7I3CbwlhY0Tshr8_TF4sL0cVmglAGHD-wxuLPR9A3uyHbDQPQ2j6MkkJrURFaHWOIxJg6bhISQnpsCPsbuQjFC_vF5JRpg6e1nL1Q-tm15ptKt3hG65PPjItdVwX_-gBlWFZqmdsoSmWZp1WQg9kRdCs2k-nswWxSy7mxaLfF7MYnIkdDpPx5NJUeR5kWZ5ls7PMXntfCfjYpIWd9lsvkizrJjli5hgJaw268tz7V7t-TfNwDqo

## cli

https://github.com/mermaid-js/mermaid-cli

```sh
mmdc -i mpp-lifecycle.mmd -o mpp-lifecycle.svg -b transparent
```
25 changes: 25 additions & 0 deletions docs/develop/marketplace-packages/mpp-lifecycle.mmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
#title: Corporate Memory Marketplace Package Lifecycle
---
%%{init: { 'logLevel': 'debug', 'theme': 'neutral' } }%%
stateDiagram-v2


state "Package Definition<br>and Release<br>(local)" as pkg
state "Published<br>(Marketplace)" as pub
state ".cpa Package<br>(local & CI/CD)" as cpa
state "Solution Dev and Config<br>(Corporate Memory)" as cmem
state "Installed<br>(Corporate Memory)" as ins
%% state "Uninstalled" as uni

[*] --> pkg : copier copy
pkg --> pkg : inspect
cpa --> cmem : install <br> --input PATH
pkg --> cmem : install <br> --input PATH
cmem --> pkg : export
pkg --> cpa : build
cpa --> pub : publish

pub --> ins : install
ins --> ins : list
ins --> [*] : uninstall
1 change: 1 addition & 0 deletions docs/develop/marketplace-packages/mpp-lifecycle.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/develop/python-plugins/installation/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ tags:
# Installation and Usage of Python Plugins

Plugins are a released as parts of Python packages.
They can but do not need to be open-source and published on [pypi.org](https://pypi.org/search/?q=%22cmem-plugin-%22) (a widely used Python Package Index). One package can contain of multiple plugins.
They can but do not need to be open-source and published on [pypi.org](https://pypi.org/search/?q=%22cmem-plugin-%22) (a widely used Python Package Index). One package can contain multiple plugins.

## Installation

Expand Down
Loading