Merge pull request #1521 from Plutonomicon/dshuiski/purs-0.15-migration

Migrate to PureScript 0.15

.gitignore

@@ -27,5 +27,5 @@ test-data/keys
 tmp
 node_modules
 plutip-server/dist-newstyle/
+plutip-server/dist/
 plutip-server/.stack-work/
-

CHANGELOG.md

@@ -12,47 +12,52 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
   - [Changed](#changed)
   - [Fixed](#fixed)
   - [Removed](#removed)
-- [[v6.0.0]](#v600)
+- [[v7.0.0]](#v700)
   - [Added](#added-1)
   - [Changed](#changed-1)
   - [Fixed](#fixed-1)
   - [Removed](#removed-1)
-- [[v5.0.0]](#v500)
+- [[v6.0.0]](#v600)
   - [Added](#added-2)
   - [Changed](#changed-2)
-  - [Removed](#removed-2)
   - [Fixed](#fixed-2)
-  - [Runtime Dependencies](#runtime-dependencies)
-- [[v4.0.2] - 2023-01-17](#v402---2023-01-17)
-  - [Fixed](#fixed-3)
-- [[v4.0.1] - 2022-12-20](#v401---2022-12-20)
+  - [Removed](#removed-2)
+- [[v5.0.0]](#v500)
   - [Added](#added-3)
-- [[v4.0.0] - 2022-12-15](#v400---2022-12-15)
-  - [Added](#added-4)
   - [Changed](#changed-3)
   - [Removed](#removed-3)
+  - [Fixed](#fixed-3)
+  - [Runtime Dependencies](#runtime-dependencies)
+- [[v4.0.2] - 2023-01-17](#v402---2023-01-17)
   - [Fixed](#fixed-4)
-  - [Runtime Dependencies](#runtime-dependencies-1)
-- [[3.0.0] - 2022-11-21](#300---2022-11-21)
+- [[v4.0.1] - 2022-12-20](#v401---2022-12-20)
+  - [Added](#added-4)
+- [[v4.0.0] - 2022-12-15](#v400---2022-12-15)
   - [Added](#added-5)
   - [Changed](#changed-4)
   - [Removed](#removed-4)
   - [Fixed](#fixed-5)
-  - [Runtime Dependencies](#runtime-dependencies-2)
-- [[2.0.0] - 2022-09-12](#200---2022-09-12)
+  - [Runtime Dependencies](#runtime-dependencies-1)
+- [[3.0.0] - 2022-11-21](#300---2022-11-21)
   - [Added](#added-6)
   - [Changed](#changed-5)
   - [Removed](#removed-5)
   - [Fixed](#fixed-6)
-- [[2.0.0-alpha] - 2022-07-05](#200-alpha---2022-07-05)
+  - [Runtime Dependencies](#runtime-dependencies-2)
+- [[2.0.0] - 2022-09-12](#200---2022-09-12)
   - [Added](#added-7)
-  - [Removed](#removed-6)
   - [Changed](#changed-6)
+  - [Removed](#removed-6)
   - [Fixed](#fixed-7)
-- [[1.1.0] - 2022-06-30](#110---2022-06-30)
+- [[2.0.0-alpha] - 2022-07-05](#200-alpha---2022-07-05)
+  - [Added](#added-8)
+  - [Removed](#removed-7)
+  - [Changed](#changed-7)
   - [Fixed](#fixed-8)
-- [[1.0.1] - 2022-06-17](#101---2022-06-17)
+- [[1.1.0] - 2022-06-30](#110---2022-06-30)
   - [Fixed](#fixed-9)
+- [[1.0.1] - 2022-06-17](#101---2022-06-17)
+  - [Fixed](#fixed-10)
 - [[1.0.0] - 2022-06-10](#100---2022-06-10)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
@@ -67,6 +72,74 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ### Removed
 
+## [v7.0.0]
+
+### Added
+
+- [esbuild](https://esbuild.github.io/) bundler support.
+  - To update your package to use esbuild, add new `devDependencies` to your `package.json`:
+
+```diff
++    "esbuild": "0.18.11",
++    "esbuild-plugin-polyfill-node": "^0.3.0",
++    "esbuild-plugin-wasm": "^1.1.0",
+```
+
+    Then consult with [the template's build scripts](./templates/ctl-scaffold/esbuild/) - also see the new [Makefile](./templates/ctl-scaffold/Makefile) setup and [NPM scripts](./templates/ctl-scaffold/package.json).
+
+### Changed
+
+- PureScript compiler version has been updated to v0.15.8. ([#1521](https://github.com/Plutonomicon/cardano-transaction-lib/pull/1521))
+  - PureScript v0.15.x outputs and expects ES modules (instead of CommonJS), which means that FFI code in dependent projects should be updated to use [ES module-style imports and exports](https://nodejs.org/api/esm.html), and consumers of CTL-based bundles should expect ES modules as well.
+  - Due to limitations of ES modules (inability to patch at runtime), CTL now uses vendored versions of CSL for node and the browser:
+
+```diff
+-    "@emurgo/cardano-serialization-lib-browser": "11.2.1",
+-    "@emurgo/cardano-serialization-lib-nodejs": "11.2.1",
++    "@mlabs-haskell/cardano-serialization-lib-gc-browser": "^1.0.6",
++    "@mlabs-haskell/cardano-serialization-lib-gc-nodejs": "^1.0.6",
+```
+
+  - Our vendored version of `json-bigint` should be updated:
+
+```diff
+-    "@mlabs-haskell/json-bigint": "1.0.0",
++    "@mlabs-haskell/json-bigint": "2.0.0",
+```
+
+  - `utf-8-validate` NPM package has been added (used during bundling):
+
+```diff
++    "utf-8-validate": "^5.0.10",
+```
+
+- WebPack bundling machinery updates:
+
+```diff
+-    "webpack": "5.67.0",
+-    "webpack-cli": "4.10",
+-    "webpack-dev-server": "4.7.4"
++    "webpack": "5.88.1",
++    "webpack-cli": "5.1.4",
++    "webpack-dev-server": "4.15.1"
+```
+
+- Nix machinery refactorings ([#1521](https://github.com/Plutonomicon/cardano-transaction-lib/pull/1521)):
+  - `buildPursDependencies` is a new function that allows to build everything except the source files of a project itself. It allows to skip rebuilding all the dependencies when using Nix every time. `buildPursProject` calls `buildPursDependencies` automatically.
+  - `censorCodes` and `strictComp` arguments have been removed from `purescriptProject` and added to `buildPursDependencies` and `buildPursProject` for more granular control.
+  - `runPursTest`: new `psEntryPoint` argument, for PureScript-level entry point function ("main" by default).
+  - `runE2ETest`: new `runnerMain` and `runnerPsEntryPoint` arguments to control which module should be the runner.
+  - `runE2ETest`: new arguments for configuring the E2E test suite: `envFile`, `emptySettingsFile` and `testTimeout`.
+  - `bundlePursProjectEsbuild` is a bundling function that uses newly introduced `esbuild`.
+  - `bundlePursProject` is renamed to `bundlePursProjectWebpack`
+- `Data.BigInt` (`purescript-bigints`) dependency was replaced with `JS.BigInt` (`purescript-js-bigints`), that uses native JavaScript BigInt instead of `BigInteger.js` library. You can remove `big-integer` NPM dependency from your project, if you don't use it elsewhere. ([#1551](https://github.com/Plutonomicon/cardano-transaction-lib/pull/1551))
+
+### Fixed
+
+### Removed
+
+- Temporarily removed `buildSearchablePursDocs` and `launchSearchablePursDocs` from nix machinery - see [#1578](https://github.com/Plutonomicon/cardano-transaction-lib/issues/1578) for context. Use `buildPursDocs` for now. ([#1521](https://github.com/Plutonomicon/cardano-transaction-lib/pull/1521))
+
 ## [v6.0.0]
 
 ### Added

Makefile

@@ -1,21 +1,61 @@
 SHELL := bash
 .ONESHELL:
-.PHONY: run-dev run-build check-format format query-testnet-tip clean check-explicit-exports
+.PHONY: esbuild-bundle esbuild-serve webpack-bundle webpack-serve check-format format query-testnet-tip clean check-explicit-exports spago-build create-bundle-entrypoint create-html-entrypoint delete-bundle-entrypoint
 .SHELLFLAGS := -eu -o pipefail -c
 
 ps-sources := $(shell fd --no-ignore-parent -epurs)
 nix-sources := $(shell fd --no-ignore-parent -enix --exclude='spago*')
-js-sources := $(shell fd --no-ignore-parent -ejs)
-ps-entrypoint := Ctl.Examples.ByUrl # points to one of the example PureScript modules in examples/
-ps-bundle = spago bundle-module -m ${ps-entrypoint} --to output.js
+js-sources := $(shell fd --no-ignore-parent -ejs -ecjs)
+
+### Bundler setup
+
+# The main Purescript module
+ps-entrypoint := Ctl.Examples.ByUrl
+# The entry point function in the main PureScript module
+ps-entrypoint-function := main
+# Whether to bundle for the browser
+browser-runtime := 1 # Use "1" for true and "" for false
+
 preview-node-ipc = $(shell docker volume inspect store_node-preview-ipc | jq -r '.[0].Mountpoint')
 preprod-node-ipc = $(shell docker volume inspect store_node-preprod-ipc | jq -r '.[0].Mountpoint')
+serve-port := 4008
+
+spago-build:
+	@spago build
+
+create-bundle-entrypoint:
+	@mkdir -p dist/
+	@echo 'import("../output/${ps-entrypoint}/index.js").then(m => m.${ps-entrypoint-function}());' > ./dist/entrypoint.js
 
-run-dev:
-	@${ps-bundle} && BROWSER_RUNTIME=1 webpack-dev-server --progress
+delete-bundle-entrypoint:
+	@rm -f ./dist/entrypoint.js
 
-run-build:
-	@${ps-bundle} && BROWSER_RUNTIME=1 webpack --mode=production
+create-html-entrypoint:
+	@mkdir -p dist/
+	@cat << EOF > dist/index.html
+	<!DOCTYPE html>
+	<html>
+	  <body><script type="module" src="./index.js"></script></body>
+	</html>
+	EOF
+
+esbuild-bundle: spago-build create-bundle-entrypoint
+	@mkdir -p dist/
+	BROWSER_RUNTIME=${browser-runtime} node esbuild/bundle.js ./dist/entrypoint.js dist/index.js
+	@make delete-bundle-entrypoint
+
+esbuild-serve: spago-build create-bundle-entrypoint create-html-entrypoint
+	BROWSER_RUNTIME=1 node esbuild/serve.js ./dist/entrypoint.js dist/index.js dist/ ${serve-port}
+
+webpack-bundle: spago-build create-bundle-entrypoint
+	BROWSER_RUNTIME=${browser-runtime} webpack --mode=production \
+		-o dist/ --env entry=./dist/entrypoint.js
+	@make delete-bundle-entrypoint
+
+webpack-serve: spago-build create-bundle-entrypoint create-html-entrypoint
+	BROWSER_RUNTIME=1 webpack-dev-server --progress \
+		--port ${serve-port} \
+		-o dist/ --env entry=./dist/entrypoint.js
 
 .ONESHELL:
 check-explicit-exports:
@@ -35,8 +75,8 @@ check-whitespace:
 check-format: check-explicit-exports check-examples-imports check-whitespace
 	@purs-tidy check ${ps-sources}
 	@nixpkgs-fmt --check ${nix-sources}
-	@prettier --log-level warn -c ${js-sources}
-	@eslint --quiet ${js-sources}
+	@prettier --loglevel warn -c ${js-sources}
+	@eslint --quiet ${js-sources} --parser-options 'sourceType: module'
 
 format:
 	@purs-tidy format-in-place ${ps-sources}
@@ -66,7 +106,6 @@ run-ci-actions:
 	nix build -L .#checks.x86_64-linux.ctl-staking-test
 	nix build -L .#checks.x86_64-linux.examples-imports-check
 
-
 clean:
 	@ rm -r .psc-ide-port || true
 	@ rm -rf .psci_modules || true
@@ -75,3 +114,4 @@ clean:
 	@ rm -rf .spago2nix || true
 	@ rm -rf node_modules || true
 	@ rm -rf output || true
+	@ rm -rf dist || true

README.md

@@ -118,7 +118,7 @@ You can find help, more information and ongoing discusion about the project here
 
 ## Funding acknowledgements
 
-CTL is being developed by MLabs. The following companies/funds have contributed significan resources to development:
+CTL is being developed by MLabs. The following companies/funds have contributed significant resources to development:
 
 - [IOHK](https://iohk.io/en/about/)
 - [Catalyst Fund8](https://cardano.ideascale.com/c/idea/396607)

doc/babbage-features.md

@@ -14,7 +14,7 @@ This document is a reference/explainer for the new CTL APIs introduced for Babba
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 ## Reference Inputs
 
-[Reference inputs](https://cips.cardano.org/cip/CIP-0031/#reference-inputs) allow looking at an output without spending it in Plutus scripts.
+[Reference inputs](https://cips.cardano.org/cip/CIP-0031#reference-inputs) allow looking at an output without spending it in Plutus scripts.
 
 There are two ways to use an input as a reference in the constraints API:
 
@@ -28,7 +28,7 @@ There are two ways to use an input as a reference in the constraints API:
 
 ## Reference Scripts
 
-[Reference Scripts](https://developers.cardano.org/docs/governance/cardano-improvement-proposals/cip-0033/) allows the use of scripts without attaching them to the transaction (and using a reference instead).
+[Reference Scripts](https://cips.cardano.org/cip/CIP-0033) allows the use of scripts without attaching them to the transaction (and using a reference instead).
 
 Reference scripts can be utilized in CTL by first creating a reference point for the script to be used later via `mustPayToScriptWithScriptRef` (or its variants).
 
@@ -40,7 +40,7 @@ Then, `mustSpendScriptOutputUsingScriptRef` (or its variants) can be used to use
 
 ## Inline Data
 
-[CIP-32](https://developers.cardano.org/docs/governance/cardano-improvement-proposals/cip-0032/) introduces the inline data feature that allows storing datum values directly in transaction outputs, instead of just the hashes.
+[CIP-32](https://cips.cardano.org/cip/CIP-0032) introduces the inline data feature that allows storing datum values directly in transaction outputs, instead of just the hashes.
 
 In CTL, alternating between datum storage options can be achieved by specifying a `DatumPresence` value with constraints that accept it, like `mustPayToPubKeyWithDatum`.
 

doc/ctl-as-dependency.md

@@ -24,7 +24,7 @@ The following caveats alway applies when using CTL from your project:
 
 CTL exposes two `overlay`s from its flake. You can use these in the Nix setup of your own project to use the same setup as we do, e.g. the same packages and PS builders:
 
-- `overlays.purescript` contains Purescript builders to compile Purescript sources, build bundles with Webpack (`bundlePursProject`), run unit tests using NodeJS (`runPursTest`), run CTL contracts on a private testnet using Plutip (`runPlutipTest`), or build Pursuit documentation (`buildSearchablePursDocs` and `launchSearchablePursDocs`)
+- `overlays.purescript` contains Purescript builders to compile Purescript sources, build bundles with Webpack (`bundlePursProject`), run unit tests using NodeJS (`runPursTest`), and run CTL contracts on a private testnet using Plutip (`runPlutipTest`).
 - `overlays.runtime` contains various packages and other tools used in CTL's runtime, including `ogmios`, `kupo`, and `plutip-server`. It also defines `buildCtlRuntime` and `launchCtlRuntime` to help you quickly launch all runtime services (see the [runtime docs](./runtime.md))
 
 We've split the overlays into two components to allow users to more easily choose which parts of CTL's Nix infrastructure they would like to directly consume. For example, some users do not require a pre-packaged runtime and would prefer to build it themselves with more control over its components (e.g. by directly using `ogmios` from their own `inputs`). Such users might still like to use our `purescript` overlay -- splitting the `overlays` allows us to support this. `overlays.runtime` also contains several haskell.nix packages which may cause issues with `hackage.nix` versions in your own project.

doc/development.md

@@ -77,7 +77,7 @@ To **build** the project **without bundling and for a NodeJS environment**:
 - `npm run staking-test` to run [Plutip](./plutip-testing.md)-powered tests for ADA staking functionality - [entry point](../test/Plutip/Staking.purs)
 - `npm run blockfrost-test` for [Blockfrost-powered tests](./blockfrost.md) (does not require a runtime, but needs [some setup](./blockfrost.md#setting-up-a-blockfrost-powered-test-suite)) - [entry point](../test/Blockfrost/Contract.purs)
 - `npm run blockfrost-local-test` for self-hosted [Blockfrost-powered tests](./blockfrost.md) (requires a [local Blockfrost runtime](./blockfrost.md#running-blockfrost-locally)) - [entry point](../test/Blockfrost/Contract.purs)
-- `npm run e2e-test` for [tests with a headless browser](./e2e-testing.md) (requires a runtime and the tests served via HTTP)
+- `npm run e2e-test` for [tests with a headless browser](./e2e-testing.md) (requires a runtime and the tests served via HTTP: `npm run start-runtime` and `npm run webpack-serve` or `esbuild-serve`)
 
 #### With Nix
 
@@ -100,16 +100,16 @@ Here and below, `<SYSTEM>` should be replaced with [one of the supported systems
 
 To run or build/bundle the project for the browser:
 
-- `npm run e2e-serve` will start a Webpack development server at `localhost:4008`, which is required for [E2E tests](./e2e-testing.md)
-- `npm run build` will output a Webpack-bundled example module to `dist` (or `nix build -L .#ctl-example-bundle-web` to build an example module using Nix into `./result/`)
+- `npm run webpack-serve` will start a Webpack development server at `localhost:4008`, which is required for [E2E tests](./e2e-testing.md)
+- `npm run {webpack|esbuild}-bundle` will output a bundled example module to `dist` (or `nix build -L .#ctl-example-bundle-web-{webpack|esbuild}` to build an example module using Nix into `./result/`)
 
 By default, Webpack will build a [Purescript module](../examples/ByUrl.purs) that serves multiple example `Contract`s depending on URL (see [here](./e2e-testing.md#serving-the-contract-to-be-tested)). You can point Webpack to another Purescript entrypoint by changing the `ps-bundle` variable in the Makefile or in the `main` argument in the flake's `packages.ctl-examples-bundle-web`.
 
-You will also need a light wallet extension pre-configured.
+You will also need a light wallet extension pre-configured to run a `Contract`.
 
-**Note**: The `BROWSER_RUNTIME` environment variable must be set to `1` in order to build/bundle the project properly for the browser (e.g. `BROWSER_RUNTIME=1 webpack ...`). For Node environments, leave this variable unset or set it to `0`.
+**Note**: The `BROWSER_RUNTIME` environment variable must be set to `1` in order to build/bundle the project properly for the browser (e.g. `BROWSER_RUNTIME=1 webpack ...`). For Node environments, leave this variable unset.
 
-**Note**: The `KUPO_HOST` environment variable must be set the base URL of the Kupo service in order to successfully run the project for the browser (e.g. `KUPO_HOST=http://localhost:1442`), otherwise all requests to Kupo will fail.
+**Note**: The `KUPO_HOST` environment variable must be set to the base URL of the Kupo service in order to successfully serve the project for the browser (by default, `KUPO_HOST=http://localhost:1442`).
 
 ## Generating PS documentation
 

doc/e2e-testing.md

@@ -214,7 +214,7 @@ Note that the test closes successfully after the first successful `Contract` exe
 
 CTL offers a function to serve the `Contract`s to be tested with a router, that dispatches contracts and configuration parameters based on query part of the URL.
 
-It also builds a page with a table consisting of links to all possible examples with all possible environments, that looks like this:
+It also builds a page with drop-downs, allowing to select an example and an environments, that looks like this:
 
 ![Headless browser test suite - served examples](./images/e2e.png)
 

doc/importing-scripts.md

@@ -4,7 +4,8 @@
 <!-- START doctoc generated TOC please keep comment here to allow auto update -->
 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
 
-- [Exporting scripts from Plutus](#exporting-scripts-from-plutus)
+- [Exporting scripts from Plutus or Plutarch](#exporting-scripts-from-plutus-or-plutarch)
+  - [Using Plutonomy](#using-plutonomy)
 - [Importing serialized scripts](#importing-serialized-scripts)
 - [Serializing Plutus scripts](#serializing-plutus-scripts)
   - [PlutusTx](#plutustx)
@@ -13,9 +14,35 @@
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 
-## Exporting scripts from Plutus
+## Exporting scripts from Plutus or Plutarch
 
-See [the-plutus-scaffold](https://github.com/mlabs-haskell/the-plutus-scaffold)'s [exporter](https://github.com/mlabs-haskell/the-plutus-scaffold/tree/main/onchain/exporter) for a pipeline example.
+Usually projects use a Haskell binary called *exporter* that outputs a pre-compiler UPLC bundle into a file.
+
+The output file should be a Cardano envelope:
+
+```json
+{
+    "cborHex": "4e4d01000033222220051200120011",
+    "description": "always-succeeds",
+    "type": "PlutusScriptV2"
+}
+```
+
+- An example of a Plutus exporter can be found [here](https://github.com/Mr-Andersen/ctl-multisign-mre/blob/main/onchain/exporter/Main.hs).
+- For Plutarch, see [the-plutus-scaffold](https://github.com/mlabs-haskell/the-plutus-scaffold)'s [exporter](https://github.com/mlabs-haskell/the-plutus-scaffold/tree/main/onchain/exporter).
+
+
+### Using Plutonomy
+
+It makes sense to use [Plutonomy](https://github.com/well-typed/plutonomy) (an optimizer for UPLC) in the exporter:
+
+```haskell
+import Plutonomy (aggressiveOptimizerOptions, optimizeUPLCWith)
+
+script :: Script
+script = fromCompiledCode $
+  optimizeUPLCWith aggressiveOptimizerOptions $$(PlutusTx.compile [||policy||])
+```
 
 ## Importing serialized scripts
 

doc/key-management.md

@@ -11,7 +11,7 @@
 
 # Key Management
 
-The process of going from private key menmonic to a set of addresses that the wallet can use is well specified in [CIP-1852](https://cips.cardano.org/cip/CIP-1852/).
+The process of going from private key menmonic to a set of addresses that the wallet can use is well specified in [CIP-1852](https://cips.cardano.org/cip/CIP-1852).
 
 For an address to be built, the user must provide some entropy for private key derivation and a *derivation path*, that specifies a list of cryptographic operations to perform on the "parent" or "root" key to get the "child" private key that can be used for signing.