Small Improvements

This is a minor release to fix a few issues in Snapcraft and sets some groundwork fixes to easily enable new extensions

---

Sergio Schvezov

• meta: transparently support command-chain
• extensions: structure the base extension class
• extensions: refactor checks into the extension base class
• extensions: support confinement restrictions
• appstream: xslt support for ul nested in p

Colin Watson

• store: send snapcraft-started-at in push requests

Chris Patterson

• scriplets: run override-pull on update_pull (#2653)
• cli: replace HiddenOption with click 7.0's new flag (#2654)

Paper cuts

This is a minor release to fix a few issues in Snapcraft.

Sergio Schvezov

• ant plugin: correct default channel and improve help
• cli: improve help for push-metadata
• project, cli: clean up snap asset messages
• file utils: better error for NotADirectory
• cli: only say sorry for in-snapcraft issues

Chris Patterson

• deprecations: add deprecation notice for version-script (dn10)

Carlo Lobrano

• meta: add InvalidAppCommand errors for non-existent and not-found (#2539)

A New Hope

Most of the improvements seen in this release are thanks to the great work done in collaboration with the attendees of the Snapcraft Summit that took place in Montreal this past northern hemisphere Spring time.

Core

Support for more building scenarios

So far Snapcraft has used the existence of the base keyword in snapcraft.yaml to trigger new behavior, this behavior did not consider the creation of bases, that is, a snapcraft.yaml that has:

type: base

In this scenario, the name set in snapcraft.yaml should be used to determine what build environment should be setup, thus:

name: core18
type: base
# base: not set

Shall create an Ubuntu 18.04 based build environment.

For the cases where the base has not been bootstrapped, it shall be possible to set build-base to the name of a base to use in order to setup the build environment, thus, the following will fail due to there not currently being a build environment for core20:

name: core20
type: base
# base: not set

But, can work with:

name: core20
type: base
build-base: core18
# base: not set

Additionally, a new value for type is now allowed, snapd, which would allow the snapd team to move forward in using snapcraft to create a snap for it.

Enhanced support for apps

Added snapcraft schema support (snapcraft.yaml) for recently introduced app properties supported by snapd (snap.yaml), along with additional improvements with regards to error handling and schema checks:

• Explicitly specify type string for existing options with unspecified type:
  • stop-command
  • reload-command
• Add daemon type dbus
• Add daemon options:
  • bus-name (for use with daemon: dbus) with regex pattern found in snapd
  • restart-delay
  • start-timeout
  • timer
  • watchdog-timeout
• Add on-watchdog for restart-condition
• Add autostart for apps installing autostart desktop files
• Specify dependencies on daemon for new daemon options, as well
  as existing options (stop-timeout, restart-condition)
• Update some descriptions to match https://docs.snapcraft.io/snap-format
• Add regex patterns for stop timeout (to match introduced timeouts).

Faster iterations when using --use-lxd

Since the general availability of snapd 2.39 it was made possible to retrieve snaps with a proper API in order to inject them into a build environment without requiring root. This provided a common implementation for
build providers (i.e.; LXD and Multipass) to use. In effect this brings the same experience of snap injection (no wasted time re-downloading snaps) when working with snapcraft on a snap enabled system (i.e.; when running snapcraft from the snap).
The end result is that iterating on a snap should be much faster now when using LXD as there is no network overhead.

Improved missing file experience

After priming, instead of simply providing a list of missing dependencies that likely need to be included in the snap, the user is now offered suggestions of stage-packages that may satisfy these dependencies (and build a functioning snap).

This work will be extended in upcoming versions of Snapcraft in order to take into account plugs using the content interface.

Plugins

crystal (new plugin)

This is a new plugin developed by the crystal upstream to work with their recently released crystal snap.

Currently, the following keyword is accepted by the plugin:

• crystal-channel:
  (string, default: latest/stable)
  The Snap Store channel to install Crystal from.

The crystal developer that attended the recent Snapcraft Summit, wrote a nice summary and example on how to use this plugin, available at https://crystal-lang.org/2019/06/19/snapcraft-summit-montreal.html

conda (new plugin)

This is a new plugin co-developed during the summit with the Anaconda developers.

The conda plugin is pretty straight forward, the following keywords are understood by the plugin:

• conda-packages
  (list of strings)
  List of conda packages to install.
• conda-python-version
  (string)
  The python version to use for the conda packages.
  Defaults to the latest supported by miniconda.
• conda-miniconda-version
  (string)
  The version of miniconda to bootstrap.
  Defaults to the latest release of miniconda.

rust

A rust developer has reviewed the plugin and suggested improvements which have made it into this release.

One of those improvements was defaulting to the usage of the rust-toolchain file (if present), unless explicitly overridden by use of rust-channel or rust-revision.

Rebuilding is now also possible using this plugin.

ant

The publisher for ant has released a snap and after a review of the plugin from the publisher, the ant plugin was updated to support the use of this new snap for building ant-based projects.

The following new keywords are accepted by the plugin:

• ant-channel:
  (string)
  The channel to use for ant in the snap store, if not using tarball from
  the ant archive (see ant-version and ant-version-checksum).
  Defaults to latest/stable.

The publisher for ant has also reviewed the plugin to satisfaction.

colcon

Support for dashing was added to the colcon plugin in order to support this latest ROS release.

Bug fixes

There have been many bug fixes in this release, of those worth mentioning are:

• improved error handling.
• additional appstream icon extraction scenarios taken into account.
• modified handling of in-snap symlinks, specifically to better accommodate the merged /usr directory scheme.
• click.prompt and click.confirm expanded to query the existence of tty for stdin.

--

The full list of features and issues worked on are listed below

Sergio Schvezov

• static: use beta channel for black (#2606)
• catkin spread tests: dump apt-config on failures for legacy (#2610)
• rust plugin: use toml to dump the config (#2611)
• rust plugin: use rust-toolchain by default if present (#2613)
• conda plugin: new plugin (#2608)
• build providers: support injection for LXD (#2621)
• schema: remove support for os when using bases (#2626)
• appstream extractor: skip non icon file paths (#2630)
• spread tests: enable LXD build provider tests (#2631)
• build environment: detect base type and use name as base
• plugins: use get_build_base to determine base support
• project: add support for build-base
• repo: add support for querying file ownership
• pluginhandler: suggest stage-packages for missing DT_NEEDED
• tests: add python3-toml for autopkgtests
• spread tests: limit conda plugin to non autopkgtests x86-64 systems
• spread tests: crystal tests should only run on x86-64

Chris Patterson

• black: minor format changes from updated black (#2603)
• sources: introduce SnapcraftSourceNotFoundError (#2604)
• spread: use more workers to reduce job times
• catkin/legacy-pull: set test to manual
• cli: convert users of click.confirm/prompt to echo.confirm/prompt
• echo: respect SNAPCRAFT_HAS_TTY for is_tty_connected()
• ant plugin: switch to using ant snap for building (by default)
• general spread tests: set base for cwd test (#2618)
• errors: refactor exception/error handling (#2602)
• tests/unit/pluginhandler: introduce tests to repro symlink preservation bug
• file_utils/create_similar_directory: drop follow_symlinks option
• pluginhandler: honour symlink directory paths for filesets (LP: #1833408)
• test_pluginhandler: remove faulty (redundant) tests
• schema: synchronizing snapd supported schema to snapcraft (#2627)

Brian J. Cardiff

• crystal plugin: new plugin (#2598)

Mike Miller

• build providers: enforce well-known temp dir (#2607) (LP: #1833292)

Pawel Stolowski

• schema: allow snapd as snap type (#2609)

Claudio Matsuoka

• echo: add wrappers for click.prompt() and click.confirm()

Kyle Fazzari

• colcon plugin: add support for dashing (#2593)

anatoly techtonik

• cli: add -h short option for help (#2527) (LP: #1807423)

Stefan Bodewig

• use the stable risk level now that ant has been released

Chris MacNaughton

• rust plugin: add ability to rebuild (#2620) (LP: #1825858)

Carlo Lobrano

• tools: let environment-setup.sh skip unnecessary steps (#2625)

Less paper cuts

Hello Snapcrafters! The Snapcraft team is pleased to announce that version 3.6 is out.
This is a minor release to fix some outstanding issues and bring in an improvement to the ant plugin.

New in this release

Plugins

ant

It is now possible to specify the build file location with the ant-buildfile keyword.

---

This is the full change set for 3.6

Sergio Schvezov

• docker images: update to be self contained (#2591)
• static: update to newer black (#2599)
• repo: set priority to critical for debs (LP: #1821313)

Kyle Fazzari

• docker images: use build stages and generate locale (#2588)
• {catkin,colcon} plugin: remove old ROS key (#2586)
• catkin plugin: check workspace for dependencies (#2585) (LP: #1832044)

Stefan Bodewig

• ant plugin: make build file location configurable (#2596)

Chris Patterson

• tools: add black snap to environment-setup.sh (#2595)

dependabot[bot]

• requirements: bump pyxdg from 0.25 to 0.26 (#2587)

Minor improvements

Hello Snapcrafters! The Snapcraft team is pleased to announce that version 3.5.1 is out.
This is a minor point release to fix some outstanding issues.

New in this release

Core

Legacy re-execution CLI

The CLI logic has been reworked in a way that re-execution into the legacy code base is only done for the build commands. This enables users of snapcraft with projects that have not migrated to bases to use the newer versions of the store commands.

Warnings when not using bases and links to upgrade paths are also presented now.

Destructive Mode

The clean command is now enabled with --destructive-mode.

---

This is the full change set for 3.5.1

Sergio Schvezov

• requirements: update to requests-toolbelt 0.8.0 (#2565)
• kernel plugin: correctly download the os.snap (#2566) (LP: #1828843)
• docs: consolidate on a simple HACKING.md (#2568)
• plainbox spread tests: lock down python packages (#2579)
• cli: refactor re-execution into legacy (#2571)
• plainbox spread tests: use https for the git source entries (#2583)
• catkin spread tests: finer system filter for legacy-pull (#2584)
• unit tests: stricter file checking on store download tests
• LEGACY common: add is_deb method
• LEGACY lifecycle: warn about bases
• LEGACY unit tests: stricter testing for common.is_deb
• LEGACY common: take into account dpkg not being found
• LEGACY common: check if running from snap before deb
• LEGACY common: check for relative argv0 in is_deb

Ian Metcalf

• dotnet plugin: fix parsing of newer sdk releases (#2537)

Kyle Fazzari

• catkin spread tests: use kinetic instead of indigo (#2575) (LP: #1830769)
• keyrings: update ROS signing key (#2578)
• LEGACY tests: stop testing indigo in catkin spread (#2572)
• LEGACY catkin plugin: remove default rosdistro (#2576)

Adam Collard

• cli: add --destructive-mode to clean (#2577)

Michael Vogt

• cli: do not prompt when there is no tty available on stdin (#2570)

dalance

• rust plugin: fix linker on i386 (#2580)

A new release

New in this release

Core

snapcraft promote

Promotion is introduced as a hidden command in this release to be able to release a set of revisions.
A build set is a set of snap revisions that meet a certain criteria, the most simple form of a build set is a set of revisions released to a channel.

When this command is used a warning will be displayed as the command syntax may change
Here's the feature on used to promote snapcraft from beta to candidate:

snapcraft promote --from-channel beta --to-channel candidate snapcraft
snapcraft promote does not have a stable CLI interface. Use with caution in scripts.
Build set information for 'beta'
Arch       Revision    Version
amd64          2947        3.5
arm64          2952        3.5
armhf          2951        3.5
i386           2949        3.5
ppc64el        2950        3.5
s390x          2948        3.5
Do you want to promote the current set to the 'candidate' channel? [y/N]: 

Execution time stamping

Improved build-aux support

---

The issues and features worked on for 3.5 can be seen on the 3.5 launchpad milestone which are reflected in the following change list:

Sergio Schvezov

• vcs: ignore .idea (#2558)
• ci: remove dependency on LXD from travis tests (#2557)
• cli: snapcraft promote (#2556) (LP: #1827513)
• manifest: expose snapcraft-started-at (#2559) (LP: #1806658)
• storeapi: allow promotion from branches
• tests: release and promote to grade devel channels
• tests: fix the status test
• tests: add ppc64el to the fake server info results

Claudio Matsuoka

• project: read local plugins from build-aux (#2551) (LP: #1827095)
• extensions: block direct use of private extensions (#2555)

Daniel Llewellyn

• meta: take ${SNAP} into account .desktop icon checks (#2541)

Minor improvements

Hello Snapcrafters! The Snapcraft team is pleased to announce that version 3.4.1 is out.
This is a minor point release to fix some outstanding issues.

New in this release

Core

LXD build environment

Removed warning about unused attributes from pylxd

Appstream

When using the appstream extractor in a part combined with the common-id keyword for an app entry, a warning will be raised when the common-id does not match any extracted appstream ID.

Environment

SNAPCRAFT_PROJECT_DIR is now exported as an environment variable, usable when doing override-<step>.

Plugins

rust

The plugin now works once more when using the nightly channel through the plugin's rust-channel parameter.

catkin

The catkin plugin has been updated to reflect some repository changes with regards to python and now uses the build environments compiler to execute the build instead of staging it into the parts building area through the plugin.

---

The issues and features worked on for 3.4.1 can be seen on the 3.4.1 launchpad milestone which are reflected in the following change list:

Sergio Schvezov

• tests: add a set_confinement helper
• tests: classic confinement spread tests for ant and maven (LP: #1805206)
• build providers: get rid of attribute warnings from pylxd (#2535)
• rust plugin: fix usage of rust-channel (#2538) (LP: #1825062)
• storeapi: move from details (v1) to info (v2) (#2550)

Tony Simpson

• project loader: add SNAPCRAFT_PROJECT_DIR environment variable (#2534) (LP: #1824417)

Kyle Fazzari

• tests: allow python2 as well as python for catkin spread tests (#2540)
• integration tests: use correct series in get_package_version (#2548)
• catkin plugin: use build-packages for compilers (#2545) (LP: #1827148)

Claudio Matsuoka

• extractors: fix typo in appstream icon unit test (#2543)
• meta: Validate common ID against appstream metadata (#2542) (LP: #1814902)

Say hello to snapcraft 3.4

New in this release

snapcraft snap

The snapcraft snap has now moved to using base: core, this unlocks snapcraft into using the snapcraft 3.0 features in its own snapcraft.yaml in order to build snapcraft from snapcraft (lots of snapcraft in that sentence!)

Core

New build provider support: LXD

You can now use LXD as a build provider, operations supported by the default mode (using multipass) are also supported with LXD, such as:

• --shell
• --shell-after
• --debug

To trigger use of LXD, the snapcraft lifecycle commands (pull, build, stage and prime, together with clean) need use the --use-lxd option.

This feature is introduced early to get feedback, but is not production ready unless it is a discardable CI environment. Early adopters may see future (with no automatic migration) changes to:

• storage setups
• default profiles
• use of LXD projects

snapcraft try

By default, when triggering builds in a clean environment, it is sometimes desirable to have the prime directory locally to be able to snap try prime. For this reason snapcraft try will run through the lifecycle all the way to prime(if not run before) and offer the prime directory locally.

While this version of snapcraft is not on stable, when combined with --use-lxd, one must pass SNAPCRAFT_BUILD_ENVIRONMENT_CHANNEL_SNAPCRAFT=latest/candidate to snapcraft. Support for snap injection for LXD will be worked on when snapd 2.38 is released.

Plugins

go

The go plugin now works more broadly when using classic confinement. This should avoid the necessity of specifying no-patchelf for parts failing to patch correctly.

catkin

The catkin plugin has been enhanced to support stage-snaps to satisfy dependencies, a detailed write up can be found on the snapcraft blog.

---

The issues and features worked on for 3.4 can be seen on the 3.4 launchpad milestone which are reflected in the following change list:

Sergio Schvezov

• build providers: modify the _run signature (#2511) (LP: #1821401)
• build providers: support for provider setup (#2515) (LP: #1821586)
• readme: add snap store badge (#2516)
• build providers: initial support for LXD (#2509) (LP: #1805221)
• cli: cleanup environment detection (#2521)
• build providers: add API for friendly instance type names (#2522)
• snap: set core as a base (#2520)
• ci: improve travis integration conditionals (#2523)
• cli: snapcraft try (#2524) (LP: #1805212)
• build providers: idempotent destroy for LXD (#2529)
• tests: add missing pylxd Build-Depends
• tests: restrict catking stage-snap tests arches

Claudio Matsuoka

• repo: handle deb package fetch error (#2513)
• project: ensure yaml load returns a dictionary (#2517)
• many: better handling of appstream icons (#2512) (LP: #1814898)
• go plugin, elf: use patchelf 0.10 and relink dynamic go binaries (#2519)
  (LP: #1805205)
• snap: use snapcraft's 0.10 patchelf branch (#2528)
• snap: revert to patchelf 0.9 with local patches (#2531)

adanhawth

• schema: add more detail wrt numeric version errors (#2506)

Kyle Fazzari

• catkin plugin: check stage-snaps for ROS dependencies (#2525)

Introducing snapcraft 3.3

New in this release

Core

base: core

In order to use the new features of snapcraft, introduced with 3.0, and still use 16.04 as a base, support for use of base: core has been added. This will trigger the same semantics as using the base keyword in snapcraft.

Alternate directory for snapcraft.yaml

build-aux/snap is now supported as an alternative directory for snapcraft.yaml and its assets (i.e.; hooks, gui, ...). To avoid confusions, snapcraft now display what directory it is picking for assets depending on where the snapcraft.yaml is found. It will only pick build-aux/snap for assets if the snapcraft.yaml is found in that path.

string validations

Snapcraft now produces a better error for when the type detected for the version string is not a string.

Plugins

python

A minor fix which should bring rebuilding capabilities to projects using the python plugin.
Long gone should be the days of seeing messages of the following construct:

You must give at least one requirement to install (maybe you meant "pip install …")

python and go

Expanded schema errors for users of the go and python, allowing for early discovery of non-valid uses of these plugins. Most importantly, this eliminates the cryptic error during build time when not using a combination of the source and <plugin-name>-packages keywords.

nodejs

Entries of type string are now supported in package.json for the bin keyword (previously, the plugin could only parse dictionary entries), this means that constructs from package.json of the form:

{
  "name": "unnamed",
  "version": "1.0",
  "description": "Using string bin entries.",
  "main": "index.js",
  "bin": "bin/index.js",
}

should be parse and interpreted correctly by snapcraft.

Store

Register against a specific store

Brand stores are a commercial feature of the Snap Store.

The following syntax is now allowed as part of the register command:

snapcraft register [--store <store>] <snap-name>

Which will allow users to register snaps for specific brand stores.

The issues and features worked on for 3.3 can be seen on the 3.3 launchpad milestone which are reflected in the following change list:

Sergio Schvezov

• many: support for "base: core" in snapcraft.yaml (#2499) (LP: #1819290)
• python plugin: graceful ret when no packages set (#2498) (LP: #1794216)
• many: support the use of build-aux/snap (#2496) (LP: #1805219)
• nodejs pluging: support for type str bin entries (#2501) (LP: #1817553)
• store: support registering to a specific store (#2479) (LP: #1820107)
• meta: fix management of snap/local (#2502)
• tests: improve login pexpect errors
• tests: correctly retry registers
• build providers: enhance provider errors (#2508) (LP: #1821217)
• build providers: improve handling in snap logic (#2507) (LP: #1820864)
• tests: filter per arch and fix snap build deps

Claudio Matsuoka

• sources: handle network request errors (#2494)
• store: handle invalid snap file errors (#2492)
• tests: fix multipass error handling spread test (#2491)
• plugins: improve python and go schema validation (#2473) (LP: #1806055)
• cli: disable raven if not running from package (#2503)

Facundo Batista

• schema: better 'version' error messages: wrong type and incorrect length (#2497)
  (LP: #1815812)

An expedited release

This is an expedited release, including some features, but more importantly the necessary fix to unbreak snap builds using classic confinement that use no base. These snaps essentially target core, which is based on Ubuntu 16.04.

New in this release

stage-snaps

This feature is equivalent to the stage-packages keyword but instead of using packages from the build environment's repositories, uses snaps hosted on the Snap Store.

The semantics are the same as those available for build-snaps. When declaring a snap to be staged, the snap will be retrieved from the Snap Store and unpacked into the the snap currently being built. The meta and snap directories from the snap will be available as meta.<snap-name> and snap.<snap-name> for the cases where assets from those locations are desired for reuse.

schema migrated from yaml to json

This has been done to make it easier to reuse in editors such as Visual Studio Code to get syntax/error highlighting in place. Here's how it would look like with the appropriate extensions in place:

Once this plugin is ready to use it will be available from the Snapcraft docs. In the meantime, an early preview is available.

New colcon plugin

This plugin enables the new build system that targets ROS2, it has been introduced in experimental form as the build system is actively being worked on by the OSRF.

These are the options the plugin offers for a part needing to build with colcon:

    - colcon-packages:
      (list of strings)
      List of colcon packages to build. If not specified, all packages in the
      workspace will be built. If set to an empty list ([]), no packages will
      be built, which could be useful if you only want ROS debs in the snap.
    - colcon-source-space:
      (string)
      The source space containing colcon packages (defaults to 'src').
    - colcon-rosdistro:
      (string)
      The ROS distro to use. Available options are bouncy and crystal (defaults to
      crystal), both of which are only compatible with core18 as the base.
    - colcon-cmake-args:
      (list of strings)
      Arguments to pass to cmake projects. Note that any arguments here which match
      colcon arguments need to be prefixed with a space. This can be done by quoting
      each argument with a leading space.
    - colcon-catkin-cmake-args:
      (list of strings)
      Arguments to pass to catkin packages. Note that any arguments here which match
      colcon arguments need to be prefixed with a space. This can be done by quoting
      each argument with a leading space.
    - colcon-ament-cmake-args:
      (list of strings)
      Arguments to pass to ament_cmake packages. Note that any arguments here which
      match colcon arguments need to be prefixed with a space. This can be done by
      quoting each argument with a leading space.

The issues and features worked on for 3.2 can be seen on the 3.2 launchpad milestone which are reflected in the following change list:

• many: support for stage-snaps (#2468) (LP: #1805214)
• project loader: do not leak a part's build-environment (#2472) (LP: #1815658)
• build providers: remove dead code (#2474)
• tests: disable spread colcon tests (#2476)
• ci: shallow clones for CLA checks on travis (#2477)
• meta: handle symlinked hooks (#2478)
• project loader: remove special LD_LIBRARY_FLAGS handling for classic (#2485) (LP: #1817300)
• sources: avoid marking changes to the snap directory as dirty (#2475) (LP: #1806746, #1816397)
• cli: clean up snapcraft push output (#2469) (LP: #1804439)
• cli: handle legitimate provider exec errors (#2483)
• schema: convert yaml jsonschema document to a json equivalent (#2448)
• tests: make before/after items an array in schema test (#2465)
• ruby plugin: support new download URL (#2466) (LP: #1815336)
• colcon plugin: new plugin (#2456) (LP: #1805213)
• colcon plugin: support build-time chaining (#2486) (LP: #1816565)