RFC: Specification for the version management

docs/specification/version-management.adoc

@@ -0,0 +1,121 @@
+= SDK Versioning And Cache Management
+Hans Larsen <hans@dfinity.org>
+v0.1, 2019-08-29
+
+-- DRAFT
+
+== Overview
+In order to use the SDK, the user has to download a version of DFX for his platform. Normally that means installing the latest version and using it, but the end user might be working on a project that was using an older version of the DFX.
+
+For example, a user could have created a project with `dfx` v0.1, but the latest is v1.0 and the user is not ready to upgrade his project just yet.
+
+This document details the design and decisions made to solve the problem of having multiple `dfx` versions, and how upstream dependencies integrate with `dfx`.
+
+== User-facing Version
+The version of DFX is the only version the user should be aware of. Any upstream version will be hidden behind the DFX version.
+
+For example, assuming `v1.1.5` and `v1.2.3` of DFX are supported stable versions, with unstable being `v1.3.0-beta.1`. Assuming a version of the DFINITY client could be released with bug fixes that affect both the previous stable, current stable and unstable releases. Without changes to the DFX client, we should release a `v1.1.6`, `v1.2.4` and `v1.3.0-beta.2` versions of the DFX containing the DFINITY client with the fix included.
+
+So the user would never actually see the version of DFINITY client that they are running.
+
+== Installing DFX
+The user should initially download the SDK by using the following command:
+[source,bash]
+curl https://sdk.dfinity.org/install.sh | sh
+
+This will install a binary compatible with your operating system, and add it to `/usr/local/bin`(or equivalent).footnote:disclaimer[Other systems, such as `brew`, `dpkg` or simply downloading a binary directly, should be made available.]
+
+This binary contains a number of things:
+
+. The global CLI. This is a CLI made to manage the cache, read the project file if necessary, andforward any calls to a versioned CLI.
+. A tar gz release file containing a distribution versioned with the latest (available) version ofthe SDK. This tar file include the versioned CLI, the actorscript compiler, the node manager andclient, and any other binaries necessary.
+
+== Version Resolution
+
+The `dfx` global CLI then tries to find the version it should use (in order);
+
+. If a `DFX_VERSION` environment variable is set, use that version directly. If that version isnot available in the cache, report the error and do not continue.
+. If there is a local DFINITY configuration file, it will use the version of the SDK specified init (the `dfx_version` field). If that version is not available in the cache, it will try todownload it from the Internet. If there is no connection, report the error and do not continue.
+. If there is an internet connection, to a maximum of once a week, the `dfx` will reach out to`sdk.dfinity.org` to find the latest version available.
+. If there is a user level cache available, the `dfx` will use the greatest version alreadydownloaded by the user, that is not greater than the `dfx` global CLI version.
+. Finally, the global `dfx` comes versioned and will use that version number as a last resort. Ifthat version is not part of the cache, it will use its internal tar file to bootstrap theuser level cache.
+
+The global CLI then defer the call to a local, user-level cached, versioned CLI.
+
+This whole process implies that:
+
+. The CLI will always create a new project using the latest available client and SDK.
+. The CLI will still work without an internet connection.
+. The global CLI only need to be updated on major changes; either the `cache` command changes, oran URL needs to be updated. If a new version need to be downloaded we can also tell the userhow to perform the upgrade.
+
+== Upgrade Subcommand
+An `upgrade` command can be made available to ping our servers, download the latest release, install it in the user-level cache, and update the project's `dfx_version` field (if in a project) to the new version. If no internet is available at that time it should error out.
+
+A migration script should also be included between two versions to upgrade the project. This is out of scope for version 1.0.
+
+== Versions
+=== Operators
+Semantic versioning operators should be supported in the `dfx_version` field. For example, using the following configuration file:
+[source,json]
+{
+  "dfx_version": ">=1.0.0 <1.4.0"
+}
+
+Should allow the user to use any `dfx` version between 1.0.0 and less than 1.4.
+
+=== Directories
+Having a directory (starting with either `.`, `~` or `/`) in the `dfx_version` field should be allowed to let the user (and most importantly our own integration tests) use a custom version of the installed binaries. No other verification than having the versioned CLI in it should be necessary.
+
+=== Tags (_optional_)
+We might want to have tracks of software that the user can use, resolving to versions through the server. The following tags could be made available from the start:
+
+. `latest`. The latest stable version.
+. `unstable`. The next alpha, beta or RC version.
+. `lts`. The last Long Term Supported version, if such a version exists.
+
+By default, prior to creating a project, the `latest` field will be used when contacting the servers to gather the latest stable version.
+
+== Remote Commands
+Commands like wallet, key management, upgrade and deployment of canisters, and other commands that affect a remote net (either the main net or a hosted test net) should validate that the version of the SDK is compatible with the version of the net being used remotely.
+
+A few ways of doing this can be considered:
+
+. The client API has a call to get the version, and we consider any delta X to be incompatible. 2majors is normally a good delta to guarantee if we implement a 1 major deprecation policy, but1 major could also work. This implies that the client and SDK are loosely versioned together.
+. The client API has a call that list all versions of the SDK it is backward compatible with.
+. The client API stays backward compatible forever.
+
+There could be other schemes that work. This is out of scope for this particular proposal, but should be addressed prior to launching the main net.
+
+== URL Scheme
+The `sdk.dfinity.org` should have a well-defined URL scheme that will avoid regressions:
+
+.URL Schemes
+|===
+| URL | Description
+
+| `sdk.dfinity.org/install.{sh,bash,fish,bat,...}` | should return a shell script that installs the
+global DFX CLI according to platform and shell environment.
+| `sdk.dfinity.org/v/` | Root of all the versions. The `index.html` should list all available
+versions.
+| `sdk.dfinity.org/v/1.2.3/x86_64-darwin.tgz` | The release for version 1.2.3 for OSX.
+| `sdk.dfinity.org/tags/` | Root of all tags released.
+| `sdk.dfinity.org/tags/latest/manifest.json` | The manifest file containing the version number and
+any flags necessary to get the version currently tagged latest.
+|===
+
+== Cache
+A cache directory will exist on the user's home folder. On Linux and OSX, it will likely be in `$HOME/.cache/dfinity`, while on Windows would likely be in `C:\Users\$USER\AppData\Local\DFINITY`.
+
+That cache folder should contain `./v/$VERSION/` folders for each version downloaded, and
+
+=== Upkeep
+A `cache` subcommand should be available to users to manage their cache. Example of subcommands:
+[source,bash]
+----
+dfx cache clear  # Delete the cache folder entirely.
+dfx cache list  # List all version installed.
+dfx cache install 1.2.3  # Download and install version 1.2.3 in the cache
+dfx cache remove 1.2.3  # Delete all the cache elements for version 1.2.3
+----
+
+Because of the delegation between global and versioned CLI, the `cache` subcommand should be strongly defined in the global CLI.