Add support for global and per-layer metadata (#359)

- added low-level property management commands: `propset`, `propget`, `proplist`, `propdel`, `prop clear`
- introduced system properties:
  - `vp:color` (`color`): the color to be used for the layer
  - `vp:pen_width` (`float`): the pen width to be used for the layer
  - `vp:name` (`float`): the name of the layer (in the `inkscape:label` SVG attribute sense)
  - `vp:page_size`: the page size (formerly stored as an instance attribute to the `vp.Document` class 
- The `show` and `write` command now take into account the above layer system properties.
- The page size related commands (`pagesize` and `layout`) now act on the `vp:page_size` global system property.
- New, high-level commands are introduced to interact with the new layer system properties:
  - `name`: sets the name of a given layer
  - `color`: changes the color for a given layer
  - `penwidth`: changes the pen-width of a given layer
  - `pens`: apply a pre-defined set of per-layer name, color, and/or pen width to the pipelines, based on a template defined in a config file (`rgb` and `cmyk` templates are bundled in the new config file)
- In addition to populating the system properties (when possible), the `read` command now extracts a number of SVG attributes and stores them as properties (prefixed with `svg:`).
- Optionally, the `write` command can attempt to restore those properties when possible. Note: since properties are per layer, SVG attributes are saved only if they are shared by *all* the geometries contained in a given layer.
- The layer operation commands are updated to handle metadata:
  - When a single source layer is specified and `--prob` is not used, the `lcopy` and `lmove` commands now copy the source layer's properties to the destination layer (possibly overwriting existing properties).
  - When `--prob` is not used, the `lswap` command now swaps the layer properties as well.
  - These behaviours may be disabled with the `--no-prop` option.
- API changes: `vpype.Document` and `vpype.LineCollection` have been extended to support operation on their metadata (through the use of a mix-in class)
- the bundled config file is renamed to `vpype_config.toml` (since it's no longer specific to HPGL features)
- `stat` command now display all the metadata in the pipeline
- added a warning when a layer passed to `--layer` doesn't exist
- tests: added a few SVG test cases
- doc: several addition/changes relative to metadata
- doc: some minor fixes

Fixes #145 and #213

.github/pull_request_template.md

@@ -12,4 +12,5 @@ _to be completed_
     - [ ] command docstring and option/argument `help`
     - [ ] README.md updated (Feature Overview)
     - [ ] CHANGELOG.md updated
+    - [ ] added new command to doc
     - [ ] RTD doc updated and building with no error (`make clean && make html` in `docs/`)

.gitignore

@@ -146,3 +146,5 @@ cython_debug/
 /my_work
 /pip-wheel-metadata
 /test_report_img_sim/
+/*.svg
+

CHANGELOG.md

@@ -2,12 +2,46 @@
 
 #### 1.9 (UNRELEASED)
 
+**Note**: This is the last version of *vpype* to support Python 3.7.
+
 New features and improvements:
-* ...
+* Updated the internal data model to support global and per-layer metadata (#359)
+
+  This feature is intended as a generic mechanism whereby a set of properties may be attached to specific layers (layer property) or all of them (global property). Properties are identified by a name and may be of arbitrary type (e.g. integer, floating point, color, etc.). This new infrastructure is used by several of the features introduced in this release, paves the way for future features, and further empowers plug-in writers. See the [documentation](https://vpype.readthedocs.io/en/latest/metadata) for more background information on metadata.
+
+* Layer color, pen width, and name are now customizable (#359)
+  * The `read` commands now sets layer color, pen width, and name based on the input SVG if possible.
+  * The new `color`, `penwdith`, and `name` commands can be used to modify layer color, pen width, and name.
+  * The new `pens` command can apply a predefined or custom scheme on multiple layers at once. Two schemes, `rgb` and `cmyk`, are included and others may be defined in the configuration file.
+  * The `show` and `write` commands were updated to take into account these new layer properties.
+
+* The `read` and `write` commands now preserve a sub-set of SVG attributes (experimental) (#359)
+
+  The `read` command now seeks for SVG attributes (e.g. `stroke-dasharray`) which are shared by all geometries in each layer. When found, such attributes are saved as layer properties (with their name prefixed with `svg:`, e.g. `svg:stroke-dasharray`). The `write` command can optionally restore these attributes in the output SVG (using the `--restore-attribs`), thereby maintaining some of the visual aspects of the original SVG (e.g. dashed lines).
+
+* Introduced new commands for low-level inspection and modification of metadata (#359)
+
+  * `propget`: gets the value of a given global or layer property
+  * `proplist`: lists all global and/or layer properties and their value
+  * `propset`: sets the value of a given global or layer property
+  * `propdel`: deletes a given global or layer property
+  * `propclear`: removes all global and/or layer properties
+
+* Updated layer operation commands to handle metadata (#359)
+
+  * When a single source layer is specified and `--prob` is not used, the `lcopy` and `lmove` commands now copy the source layer's properties to the destination layer (possibly overwriting existing properties).
+  * When `--prob` is not used, the `lswap` command now swaps the layer properties as well.
+  * These behaviors can be disabled with the `--no-prop` option.
+
+* Providing a non-existent layer ID to any `--layer` parameter now generates a warning (#359)
+
+API changes:
+* `vpype.Document` and `vpype.LineCollection` have additional members to manage properties through the `vpype._MetadataMixin` mix-in class (#359)
 
 Other changes:
+* Renamed the bundled config file to `vpype_config.toml` (#359)
 * Changed dependencies to dataclasses (instead of attrs) and tomli (instead of toml) (#362)
-* Various documentation improvements (#363)
+* Various documentation improvements (#359, #363)
 
 
 #### 1.8.1 (2022-01-13)

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2019-2020 Antoine Beyeler & Contributors
+Copyright (c) 2019-2022 Antoine Beyeler & Contributors
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

README.md

@@ -4,12 +4,14 @@
 # _vpype_
 
 [![PyPI](https://img.shields.io/pypi/v/vpype?label=PyPI&logo=pypi)](https://pypi.org/project/vpype/)
-![python](https://img.shields.io/github/languages/top/abey79/vpype)
+[![python](https://img.shields.io/github/languages/top/abey79/vpype)](https://www.python.org)
+[![Downloads](https://pepy.tech/badge/vpype)](https://pepy.tech/project/vpype)
 [![license](https://img.shields.io/github/license/abey79/vpype)](https://vpype.readthedocs.io/en/stable/license.html)
 ![Test](https://img.shields.io/github/workflow/status/abey79/vpype/Lint%20and%20Tests?label=Tests&logo=github)
 [![codecov](https://codecov.io/gh/abey79/vpype/branch/master/graph/badge.svg?token=CE7FD9D6XO)](https://codecov.io/gh/abey79/vpype)
 [![Sonarcloud Status](https://sonarcloud.io/api/project_badges/measure?project=abey79_vpype&metric=alert_status)](https://sonarcloud.io/dashboard?id=abey79_vpype)
 [![Documentation Status](https://img.shields.io/readthedocs/vpype?label=Read%20the%20Docs&logo=read-the-docs)](https://vpype.readthedocs.io/en/latest/?badge=latest)
+[![Code style: Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 
 _vpype_ is the Swiss-Army-knife command-line tool for plotter vector graphics.
 
@@ -192,6 +194,13 @@ and much more.
 - Arbitrary **page size** definition ([`pagesize`](https://vpype.readthedocs.io/en/stable/reference.html#pagesize)). 
 
 
+#### Metadata
+
+- Adjust layer **color**, **pen width** and **name** ([`color`](https://vpype.readthedocs.io/en/stable/reference.html#color), [`penwidth`](https://vpype.readthedocs.io/en/stable/reference.html#penwidth), [`name`](https://vpype.readthedocs.io/en/stable/reference.html#name)).
+- Apply provided or fully customisable **pen configurations** ([`pens`](https://vpype.readthedocs.io/en/stable/reference.html#pens)).
+- Manipulate global and per-layer **properties** ([`propset`](https://vpype.readthedocs.io/en/stable/reference.html#propset), [`propget`](https://vpype.readthedocs.io/en/stable/reference.html#propget), [`proplist`](https://vpype.readthedocs.io/en/stable/reference.html#proplist), [`propdel`](https://vpype.readthedocs.io/en/stable/reference.html#propdel), [`propclear`](https://vpype.readthedocs.io/en/stable/reference.html#propclear)).
+
+
 #### Plotting optimization
 
 - **Line merging** with optional path reversal and configurable merging threshold ([`linemerge`](https://vpype.readthedocs.io/en/stable/reference.html#linemerge)).

docs/conf.py

@@ -20,7 +20,7 @@
 
 project = "vpype"
 # noinspection PyShadowingBuiltins
-copyright = "2020, Antoine Beyeler"
+copyright = "2020-2022, Antoine Beyeler"
 author = "Antoine Beyeler"
 
 # -- General configuration ---------------------------------------------------
@@ -108,6 +108,11 @@ def autodoc_skip_member(app, what, name, obj, skip, options):
         "__doc__",
         "__module__",
         "__weakref__",
+        "__init__",
+        "__getitem__",
+        "__iter__",
+        "__len__",
+        "__repr__",
     )
     exclude = name in exclusions
     return skip or exclude

docs/cookbook.rst

@@ -153,10 +153,41 @@ Note that *vpype* does not "remember" of such configuration file and the ``--con
 
 .. note::
 
-   *vpype* is bundled with a `configuration file <https://github.com/abey79/vpype/blob/master/vpype/hpgl_devices.toml>`_. It is strongly discouraged to edit this file as it will be overwritten each time *vpype* is installed or updated.
+   *vpype* is bundled with a `configuration file <https://github.com/abey79/vpype/blob/master/vpype/vpype_config.toml>`_. It is strongly discouraged to edit this file as it will be overwritten each time *vpype* is installed or updated.
 
 
+.. _faq_custom_pen_config:
 
+Creating a custom pen configuration
+===================================
+
+Pen configurations associate names, colors, and/or pen widths to specific layers and are applied by the :ref:`cmd_pens`
+command. For example, the included ``cmyk`` pen configuration sets the name and color or layers 1 to 4 to cyan, magenta,
+yellow, resp. black, while leaving pen widths unchanged. New pen configurations can be defined in a custom config file
+(see :ref:`faq_custom_config_file`).
+
+Pen configurations must conform to the following format to be valid:
+
+  .. code-block:: toml
+
+    [pen_config.my_pen_config]  # my_pen_config is this pen configuration's name
+    layers = [
+        # for each layer, a layer_id must be provided, but name, color and
+        # pen_width are optional
+        { layer_id = 1, name = "black layer", color = "black", pen_width = "0.15mm" },
+
+        # any valid CSS color string and length unit may be used
+        { layer_id = 2, name = "red layer", color = "#e00", pen_width = "0.05in" },
+
+        # any attribute may be omitted, except layer_id
+        { layer_id = 4, color = "#00de00" },
+
+        # etc. (a pen configuration may have an arbitrary number of layers defined)
+    ]
+
+The above pen configuration can be used by referring to its name, in this case ``my_pen_config``::
+
+  $ vpype [...] pens my_pen_config [...]
 
 
 Converting a SVG to HPGL

docs/fundamentals.rst

@@ -156,6 +156,49 @@ Likewise, angles are interpreted as degrees by default but alternative units may
   $ vpype rect 0 0 50 100 rotate 0.785398rad
 
 
+.. _fundamentals_metadata:
+
+Metadata
+========
+
+Metadata is data which provides information about other data. In the case of *vpype*, metadata takes the form of *properties* that are either attached to a given layer, or global. Properties are identified by a name and their value can be of arbitrary type (e.g. integer, floating point, color, etc.). There can be any number of global and/or layer properties and it is up to commands (and plug-ins) how they act based (or upon) these properties.
+
+
+System properties
+-----------------
+
+Some properties are referred to as *system properties*. Their name is prefixed with ``vp:`` and they are widely used throughout *vpype*. Currently, the following system properties are defined:
+
+  * ``vp:color``: the color of a layer (layer property)
+  * ``vp:pen_width``: the pen width of a layer (layer property)
+  * ``vp:name``: the name of a layer (layer property)
+  * ``vp:page_size``: the page size (global property)
+
+Many commands acts on these properties. For example, the :ref:`cmd_read` command sets these properties according to the imported SVG file's content. The :ref:`cmd_color`, :ref:`cmd_penwidth`, :ref:`cmd_name`, and :ref:`cmd_pens` commands can set these properties to arbitrary values. In particular, the :ref:`cmd_pens` commands can apply a predefined set of values on multiple layers at once, for example to apply a CMYK color scheme (see :ref:`faq_custom_pen_config` for more information). The page size global property is set by the :ref:`cmd_pagesize` and :ref:`cmd_layout` commands.
+
+
+SVG attributes properties
+-------------------------
+
+The :ref:`cmd_read` command identifies SVG attributes common to all geometries in a given layer and store their value as layer property with a ``svg:`` prefix. For example, if all geometries in a given layer share a ``stroke-dasharray="3 1"`` SVG attribute (either because it is set at the level of the group element, or because it is set in every single geometry elements), a property named ``svg:stroke-dasharray`` with a value of ``"3 1"`` is added to the layer.
+
+These properties are set for informational and extension purposes, and are mostly ignored by *vpype* commands. One exception is the :ref:`cmd_write` command, which can optionally restore these attributes in the exported SVG file.
+
+An example of future extension could be a plug-in which detects the ``svg:stroke-dasharray`` property and turns the corresponding layer's lines into their dashed equivalent. Another example would be a plug-in looking for a ``svg:fill`` property and adding the corresponding hatching patterns to reproduce the filled area.
+
+
+Interacting with properties
+---------------------------
+
+High-level commands such as :ref:`cmd_penwidth` are not the only means of interacting with properties. *vpype* includes a set of low-level commands to inspect and modify global and layer properties:
+
+  * :ref:`cmd_propget`: reads the value of a single global or layer property
+  * :ref:`cmd_proplist`: lists all the global or layer properties
+  * :ref:`cmd_propset`: sets the value of a given global or layer property
+  * :ref:`cmd_propdel`: deletes a given global or layer property
+  * :ref:`cmd_propclear`: deletes all global or layer properties
+
+
 .. _fundamentals_blocks:
 
 Blocks

docs/reference.rst

@@ -23,6 +23,10 @@ CLI reference
 .. click:: vpype_cli:circle
    :prog: circle
 
+.. _cmd_color:
+.. click:: vpype_cli:color
+   :prog: color
+
 .. _cmd_crop:
 .. click:: vpype_cli:crop
    :prog: crop
@@ -78,11 +82,11 @@ CLI reference
 .. _cmd_lmove:
 .. click:: vpype_cli:lmove
    :prog: lmove
-   
+
 .. _cmd_lreverse:
 .. click:: vpype_cli:lreverse
    :prog: lreverse
-   
+
 .. _cmd_lswap:
 .. click:: vpype_cli:lswap
    :prog: lswap
@@ -91,10 +95,42 @@ CLI reference
 .. click:: vpype_cli:multipass
    :prog: multipass
 
+.. _cmd_name:
+.. click:: vpype_cli:name
+   :prog: name
+
 .. _cmd_pagesize:
 .. click:: vpype_cli:pagesize
    :prog: pagesize
 
+.. _cmd_pens:
+.. click:: vpype_cli:pens
+   :prog: pens
+
+.. _cmd_penwidth:
+.. click:: vpype_cli:penwidth
+   :prog: penwidth
+
+.. _cmd_propclear:
+.. click:: vpype_cli:propclear
+   :prog: propclear
+
+.. _cmd_propdel:
+.. click:: vpype_cli:propdel
+   :prog: propdel
+
+.. _cmd_propget:
+.. click:: vpype_cli:propget
+   :prog: propget
+
+.. _cmd_proplist:
+.. click:: vpype_cli:proplist
+   :prog: proplist
+
+.. _cmd_propset:
+.. click:: vpype_cli:propset
+   :prog: propset
+
 .. _cmd_random:
 .. click:: vpype_cli:random
    :prog: random