Wallet Backend builds and tests on Windows #703

KtorZ · 2019-09-13T07:36:44Z

Context

cardano-wallet ultimately results in two binaries as detailed in the root .cabal:

cardano-wallet-http-bridge
cardano-wallet-jormungandr

Both are command-lines and web server doing "wallet backend stuff", but working with different chain producers.

Decision

At the moment, our CI only produces binary artifacts for linux machines yet, most are using Windows and OSX machines so we ought to be able to produce executables for these targets too. Ideally, our test suites should also run on both platform to ensure that code produced works as expected on these platforms.

The focus is given on cardano-wallet-jormungandr for cardano-wallet-http-bridge will be removed soon enough and will not see the lights of production.

Acceptance Criteria

cardano-wallet-jormungandr must compile to Windows
cardano-wallet-core:test:unit must pass on Windows
cardano-wallet-cli:test:unit must pass on Windows
cardano-wallet-launcher:test:unit must pass on Windows
bech32:test:unit must pass on Windows
cardano-wallet-jormungandr:test:unit must pass on Windows
cardano-wallet-jormungandr:test:integration must pass on Windows
cardano-wallet-core:benchmark:db should pass on Windows

Development Plan

Get all tests working under nix-build for Linux.
Get cross-compiled tests passing under Wine.
Manually test .exe on a Windows 10 VM.

QA

Checking for windows builds and tests in CI

Builds for the master branch are shown in the cardano-wallet jobset.
Ensure that x86_64-pc-mingw32 jobs for the packages listed in the AC are green.
Check that the required job is green.
Check the Constituents of the required job. These should include x86_64-pc-mingw32 jobs for the packages listed in the AC.

Running tests on windows

Download the Windows testing bundle onto a Windows system.
Extract to zip archive to somewhere (e.g. Desktop).
Ensure old data in c:\cardano-wallet-jormungandr is removed.
Use the example batch files launch.bat, cw.bat, jm.bat.
Or you can invoke cardano-wallet-jormungandr.exe yourself.
Run all tests with tests.bat.
Or you can run test suites individually with their .exe files.

Building windows testing bundle locally

See Building Wiki for background info.
nix-build release.nix -A cardano-wallet-jormungandr-tests-win64 && ls ./result

Running tests with Wine

nix-env -f '<nixpkgs>' -iA pkgs.wineWowPackages.minimal  # Install Wine if you don't already have it.
nix-build release.nix -A x86_64-pc-mingw32.tests.cardano-wallet-core.unit.x86_64-linux  # Build a test suite
wine ./result/cardano-wallet-core-2019.11.7/unit.exe --match ...

The text was updated successfully, but these errors were encountered:

rvl · 2019-09-16T08:24:17Z

How to test cardano-wallet-jormungandr on Windows

If you don't already have it, set up Windows 10 in VirtualBox.
You can use the Powershell script and instructions at
iohk-ops/windows/iohk-setup.ps1.
Just delete everything in the script starting from where it installs GHC.

Also make sure you have the
Microsoft Visual C++ Redistributable for Visual Studio 2015
(or maybe it was 2017, I forget now).
Set up a shared folder between Linux and Windows.
Let's call it ~/iohk/win_shared or E:.
Under the cardano-wallet repo, run:
```
nix-build release.nix -A cardano-wallet-jormungandr-win64
cp -vf result/*.zip ~/iohk/win_shared
```
Alternatively, download the prebuilt distribution from Hydra:
cardano-wallet-jormungandr-win64 | zip download
On Windows, open E: in explorer, then right-click on the zip file
and select "Extract All...". Extract it to somewhere like
C:\cardano-wallet-jormungandr.
Open up two command prompt windows (Start→Run cmd.exe) and cd
to where you extracted the files.
In the first window, type jm [ENTER].
In the second window, type cw [ENTER].

717: Fix windows build r=craigem a=rvl Relates to #703 # Overview - I broke the cross build in 7208a54. This fixes it, and adds a necessary DLL to the packages. - In the course of writing instructions for testing the windows build, I made a script which packages all the dependencies and configs together in a zip file. This is easy to download off Hydra and quick to test. # Comments - [Link to Hydra build](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-717/cardano-wallet-jormungandr-win64/latest) - [zip download](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-717/cardano-wallet-jormungandr-win64/latest/download-by-type/file/binary-dist) Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

KtorZ · 2019-10-15T23:07:02Z

@rvl @Disasm

Can someone give a clear albeit concise status update on Windows builds?

In particular, answers to the following questions:

Can we build a Windows executable that runs, without crashing, the wallet server?
Can we run our unit tests on wine or Windows directly?
Can we run our integration scenarios on wine or Windows?
If it exists some build artifacts that are produced for Windows, where can they be found?
If any answer to the questions above is "no", what are the current blockers or items being worked on?

I've got the impression that many things are happening in background but I can't clearly see what because nothing seem to be really tracked and discussions are happening.. Somewhere else. There are PR pending with no descriptions, and it's very hard to follow.

Thank you very much 🙏

766: Run tests in nix release build r=rvl a=rvl Relates to #703 and #704. Base branch is #864 - this should be merged first. # Overview - [x] This enables test suites under the Nix build. - [x] Fixes test cases that used symlinks. - [x] `Cardano.Wallet.Api.TypesSpec` is trying to access a file `/build/core/../../specifications/api/swagger.yaml` which is outside of its package. - [x] Some tests in `cardano-wallet-jormungandr:test:integration` required `jcli`. - [x] Some tests in `cardano-wallet-jormungandr:test:integration` required a writable home directory. - [x] In LauncherSpec the test scripts used `/usr/bin/env`, which is not in the nix-build sandbox, so i made the tests not need external scripts. - [x] Adds macOS tests to the Hydra required jobs. [Hydra jobset](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-766) # Comments - Symlinks/references to files in other packages won't work with the nix build because when it builds it copies the package source tree to `/build` and builds from there. # Remaining failures These windows-specific failures can be fixed in the next PR. - [ ] `x86_64-pc-mingw32.tests.cardano-wallet-jormungandr.unit.x86_64-linux` expects UTF-8 encoding, but this is not the encoding that Wine is using. - [ ] Windows cross build got broken. We now require a fix for Haskell.nix. - [ ] `cardano-wallet-jormungandr.exe` would fail on windows at startup with ``` NUL: openFile: does not exist (No such file or directory) ``` Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

906: nix: Update windows jormungandr version and the instructions r=piotr-iohk a=rvl Relates to #703. # Overview Updates the instructions for `nix/jormungandr.nix` to include the windows binary release. Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

914: Fix text locale encoding issues on windows r=KtorZ a=rvl Relates to #703. # Overview - Forces the current windows console into UTF-8 mode. - UTF-8 is far from standard on windows. But it shouldn't be assumed on any platform. - This fixes strange characters appearing on windows and test failures with exceptions related to encoding. Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

@angerman

888: Update Haskell.nix r=KtorZ a=rvl Relates to #703. # Overview - [x] Updates the Haskell.nix version so that we can build for windows again. - [x] Keeps more or less the same structure as before - i.e. no IFD, no niv. I don't want too much change at once. - [x] Does however update nixpkgs from iohk fork of 18.09 → upstream 19.03. - [x] Provides a big bundle of scripts and tests that can be run on Windows. [Hydra jobset](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888) [Hercules jobset](https://hercules-ci.com/github/input-output-hk/cardano-wallet) [Link to download Windows testing bundle for PR #888](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-888/cardano-wallet-jormungandr-win64/latest) [Link to download Windows testing bundle (master branch)](https://hydra.iohk.io/job/Cardano/cardano-wallet/cardano-wallet-jormungandr-win64/latest) # Comments @angerman Please help - [x] Fix failure for missing `hsc2hs` - [x] Fix a previous mistake that affects cross builds (failing to cross-compile bash for some reason) - [x] Worked around an [evaluation error](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888#tabs-errors) because weeder and ghci are no longer available from `pkgs.haskellPackages`. - [ ] Evaluation is taking a rather long time (5+ hours) in Hydra. - [ ] Some of the Buildkite pipeline steps don't work with the updated Haskell.nix. # Next PRs - [ ] Fix failures in sqlite tests on Windows. - [ ] Fix failure in launch command on Windows. - [ ] Fix the UTF-8 text issues on Windows ⇒ #914. - [ ] Add code which tells the windows command prompt to interpret ANSI color codes. - [ ] Identify other test failures and fix. Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

@angerman

888: Update Haskell.nix r=KtorZ a=rvl Relates to #703. # Overview - [x] Updates the Haskell.nix version so that we can build for windows again. - [x] Keeps more or less the same structure as before - i.e. no IFD, no niv. I don't want too much change at once. - [x] Does however update nixpkgs from iohk fork of 18.09 → upstream 19.03. - [x] Provides a big bundle of scripts and tests that can be run on Windows. [Hydra jobset](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888) [Hercules jobset](https://hercules-ci.com/github/input-output-hk/cardano-wallet) [Link to download Windows testing bundle for PR #888](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-888/cardano-wallet-jormungandr-win64/latest) [Link to download Windows testing bundle (master branch)](https://hydra.iohk.io/job/Cardano/cardano-wallet/cardano-wallet-jormungandr-win64/latest) # Comments @angerman Please help - [x] Fix failure for missing `hsc2hs` - [x] Fix a previous mistake that affects cross builds (failing to cross-compile bash for some reason) - [x] Worked around an [evaluation error](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888#tabs-errors) because weeder and ghci are no longer available from `pkgs.haskellPackages`. - [ ] Evaluation is taking a rather long time (5+ hours) in Hydra. - [ ] Some of the Buildkite pipeline steps don't work with the updated Haskell.nix. # Next PRs - [ ] Fix failures in sqlite tests on Windows. - [ ] Fix failure in launch command on Windows. - [ ] Fix the UTF-8 text issues on Windows ⇒ #914. - [ ] Add code which tells the windows command prompt to interpret ANSI color codes. - [ ] Identify other test failures and fix. Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

@angerman

888: Update Haskell.nix r=KtorZ a=rvl Relates to #703. # Overview - [x] Updates the Haskell.nix version so that we can build for windows again. - [x] Keeps more or less the same structure as before - i.e. no IFD, no niv. I don't want too much change at once. - [x] Does however update nixpkgs from iohk fork of 18.09 → upstream 19.03. - [x] Provides a big bundle of scripts and tests that can be run on Windows. [Hydra jobset](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888) [Hercules jobset](https://hercules-ci.com/github/input-output-hk/cardano-wallet) [Link to download Windows testing bundle for PR #888](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-888/cardano-wallet-jormungandr-win64/latest) [Link to download Windows testing bundle (master branch)](https://hydra.iohk.io/job/Cardano/cardano-wallet/cardano-wallet-jormungandr-win64/latest) # Comments @angerman Please help - [x] Fix failure for missing `hsc2hs` - [x] Fix a previous mistake that affects cross builds (failing to cross-compile bash for some reason) - [x] Worked around an [evaluation error](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888#tabs-errors) because weeder and ghci are no longer available from `pkgs.haskellPackages`. - [ ] Evaluation is taking a rather long time (5+ hours) in Hydra. - [ ] Some of the Buildkite pipeline steps don't work with the updated Haskell.nix. - [ ] Tests for windows are built not not executed. The tests should be run under wine. # Next PRs - [ ] Fix failures in sqlite tests on Windows ⇒ #918. - [ ] Fix failure in launch command on Windows. - [ ] Fix the UTF-8 text issues on Windows ⇒ #914. - [ ] Add code which tells the windows command prompt to interpret ANSI color codes. - [ ] Identify other test failures and fix. 952: review golden tests for legacy transaction and generate witnesses using jcli r=KtorZ a=KtorZ # Issue Number  N/A # Overview  - [x] I have reviewed the legacy transaction golden tests, now fully using jcli to also generate the witnesses (this feature was missing when the golden tests were initially implemented). - [x] I have updated the comment on private key to reflect the real format expected by jcli # Comments  A bit disappointing to some extend since nothing really has changed.. It seems that we were already providing the right output :(  Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io> Co-authored-by: Michael Bishop <cleverca22@gmail.com> Co-authored-by: KtorZ <matthias.benkort@gmail.com>

@angerman

888: Update Haskell.nix r=KtorZ a=rvl Relates to #703. # Overview - [x] Updates the Haskell.nix version so that we can build for windows again. - [x] Keeps more or less the same structure as before - i.e. no IFD, no niv. I don't want too much change at once. - [x] Does however update nixpkgs from iohk fork of 18.09 → upstream 19.03. - [x] Provides a big bundle of scripts and tests that can be run on Windows. [Hydra jobset](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888) [Hercules jobset](https://hercules-ci.com/github/input-output-hk/cardano-wallet) [Link to download Windows testing bundle for PR #888](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-888/cardano-wallet-jormungandr-win64/latest) [Link to download Windows testing bundle (master branch)](https://hydra.iohk.io/job/Cardano/cardano-wallet/cardano-wallet-jormungandr-win64/latest) # Comments @angerman Please help - [x] Fix failure for missing `hsc2hs` - [x] Fix a previous mistake that affects cross builds (failing to cross-compile bash for some reason) - [x] Worked around an [evaluation error](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888#tabs-errors) because weeder and ghci are no longer available from `pkgs.haskellPackages`. - [ ] Evaluation is taking a rather long time (5+ hours) in Hydra. - [ ] Some of the Buildkite pipeline steps don't work with the updated Haskell.nix. - [ ] Tests for windows are built not not executed. The tests should be run under wine. # Next PRs - [ ] Fix failures in sqlite tests on Windows ⇒ #918. - [ ] Fix failure in launch command on Windows. - [ ] Fix the UTF-8 text issues on Windows ⇒ #914. - [ ] Add code which tells the windows command prompt to interpret ANSI color codes. - [ ] Identify other test failures and fix. Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io> Co-authored-by: Michael Bishop <cleverca22@gmail.com>

918: Fix Sqlite DB on Windows r=rvl a=rvl Relates to #703. Based on the branch of #888. # Overview The wallet failed with: NUL: openFile: does not exist (No such file or directory) This is due to runMigrationSilent which uses the silently library: http://hackage.haskell.org/package/silently-1.2.5.1/docs/src/System.IO.Silently.html#line-27 To fix, we swap `runMigrationQuiet` for `runMigrationSilent` on Windows. Co-authored-by: KtorZ <matthias.benkort@gmail.com> Co-authored-by: iohk-bors[bot] <43231472+iohk-bors[bot]@users.noreply.github.com> Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

918: Fix Sqlite DB on Windows r=rvl a=rvl Relates to #703. Based on the branch of #888. # Overview The wallet failed with: NUL: openFile: does not exist (No such file or directory) This is due to runMigrationSilent which uses the silently library: http://hackage.haskell.org/package/silently-1.2.5.1/docs/src/System.IO.Silently.html#line-27 To fix, we swap `runMigrationQuiet` for `runMigrationSilent` on Windows. Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

958: nix: Add necessary test data files to windows testing bundle r=rvl a=rvl Relates to #703. # Overview - Includes test data in the zip file so that tests can pass. - Fixes `launch.bat` - Updates the Haskell.nix revision so that it's on the master branch, not a PR branch. # Comments Testing procedure is in #703. Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

957: cli: ANSI colour for windows 10, and easier utf-8 setup r=rvl a=rvl Relates to #703. # Overview IOHK logging framework uses ANSI control codes for colouring its output (instead of the Win32-specific method). Windows console does not interpret ANSI colour codes unless told to (even then, it's only supported under win 10). - Enables ANSI colour at startup ([System.Console.ANSI](http://hackage.haskell.org/package/ansi-terminal-0.10.1/docs/System-Console-ANSI.html#v:hSupportsANSIWithoutEmulation)). - There is a better way of setting UTF-8 on Windows - [code-page package](http://hackage.haskell.org/package/code-page-0.2/docs/System-IO-CodePage.html#v:withCP65001). Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

959: LauncherSpec: Let it work on Windows r=rvl a=rvl Relates to #703. # Overview - LauncherSpec used posix commands for testing. - This substitutes equivalent commands when running on windows. Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

@angerman

971: tests: Fix ServerSpec and SqliteSpec on windows r=KtorZ a=rvl Relates to #703. # Overview Windows unit test fixes. - Let ServerSpec error path tests pass. - Close temp file so that SqliteSpec passes. # Comments Cross-compiled tests are not yet automatically run under Haskell.nix with Wine (cc @angerman @hamishmack). So am testing with: ``` nix-env -f '<nixpkgs>' -iA pkgs.wineWowPackages.minimal wine $(nix-build release.nix -A x86_64-pc-mingw32.tests.cardano-wallet-core.unit.x86_64-linux)/cardano-wallet-core-2019.11.6/unit.exe ``` Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io> Co-authored-by: KtorZ <matthias.benkort@gmail.com>

981: Make API Golden tests filenames compatible with Windows r=rvl a=KtorZ # Issue Number  #703 # Overview  - [x] I have revised api golden test filenames generation to only use alpha-num characters - [x] I have re-generated all api golden tests # Comments  ``` $ ls AddressAmountTestnet.json ApiNetworkTip.json ApiTimeReference.json ApiTWalletId.json ByronWalletPostData.json ApiAddressTestnet.json ApiStakePool.json ApiTPassphraseencryption.json ApiTWalletName.json PostTransactionDataTestnet.json ApiBlockReference.json ApiStakePoolMetrics.json ApiTransactionTestnet.json ApiTWalletPassphraseInfo.json PostTransactionFeeDataTestnet.json ApiByronWallet.json ApiTAddressPoolGap.json ApiTSyncProgress.json ApiTxId.json WalletPostData.json ApiByronWalletMigrationInfo.json ApiTAddressProxyNetworkDiscriminantTestnet.json ApiTTxStatus.json ApiUtxoStatistics.json WalletPutData.json ApiFee.json ApiTDirection.json ApiTWalletBalance.json ApiWallet.json WalletPutPassphraseData.json ApiNetworkInformation.json ApiTHashTx.json ApiTWalletDelegationApiTPoolId.json ApiWalletPassphrase.json ```  Co-authored-by: KtorZ <matthias.benkort@gmail.com>

975: Fix bench-db on windows r=KtorZ a=rvl Relates to #703. # Overview The database benchmark was failing with: ``` cardano-wallet-core-2019.11.6-bench-db.exe: C:\users\rodney\Temp\benf0f5.db: DeleteFile "\\\\?\\C:\\users\\rodney\\Temp\\benf0f5.db": permission denied (Sharing violation.) ``` And: ``` db.exe: <stdout>: commitBuffer: invalid argument (Invalid argument) ``` It works on windows but not under wine (encoding problem). 980: Move delegation certificate declaration in a separate table r=KtorZ a=KtorZ # Issue Number  #913 # Overview  - [x] I have moved tracking of delegation "certificate" into a separate SQLite table - [x] I have adjusted `readWalletMetadata` to now pull the delegation status from the right table - [x] I have added a `putDelegationCertificate` function to insert a new row in the database when new certificates are discovered - [x] I have extended the state-machine tests accordingly, with proper tagging # Comments  ``` Cardano.Wallet.DB.Sqlite Sqlite State machine tests Sequential +++ OK, passed 400 tests: 65.8% UnsuccessfulReadTxHistory 57.8% SuccessfulReadCheckpoint 57.2% CreateThenList 55.0% TxUnsortedInputs 54.8% TxUnsortedOutputs 49.0% CreateWalletTwice 32.5% ReadTxHistoryAfterDelete 28.5% PutCheckpointTwice 27.8% UnsuccessfulReadCheckpoint 24.8% ReadMetaAfterPutCert 22.5% CreateThreeWallets 18.5% RemovePendingTxTwice 17.5% RolledBackOnce 14.2% SuccessfulReadPrivateKey 13.2% SuccessfulReadTxHistory 12.0% RemoveWalletTwice Parallel # PENDING: No reason given Sqlite State machine (RndState) Sequential state machine tests +++ OK, passed 400 tests: 65.8% UnsuccessfulReadTxHistory 57.8% SuccessfulReadCheckpoint 57.2% CreateThenList 55.0% TxUnsortedInputs 54.8% TxUnsortedOutputs 49.0% CreateWalletTwice 32.5% ReadTxHistoryAfterDelete 28.5% PutCheckpointTwice 27.8% UnsuccessfulReadCheckpoint 24.8% ReadMetaAfterPutCert 22.5% CreateThreeWallets 18.5% RemovePendingTxTwice 17.5% RolledBackOnce 14.2% SuccessfulReadPrivateKey 13.2% SuccessfulReadTxHistory 12.0% RemoveWalletTwice ```  Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io> Co-authored-by: KtorZ <matthias.benkort@gmail.com>

974: tests: Fix Network.Wai.Middleware.LoggingSpec on Windows r=KtorZ a=rvl Relates to #703. # Overview ``` test/unit/Network/Wai/Middleware/LoggingSpec.hs:109:5: 9) Network.Wai.Middleware.Logging, Logging Middleware, GET, 200, no query uncaught exception: IOException of type UserError user error (Expected exactly 3 log entries but got 1: [LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "[GET] /get"}]) To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/GET, 200, no query/" test/unit/Network/Wai/Middleware/LoggingSpec.hs:117:5: 10) Network.Wai.Middleware.Logging, Logging Middleware, GET, 200, with query uncaught exception: IOException of type UserError user error (Expected exactly 3 log entries but got 5: [LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "200 OK in 0s"},LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Debug, privacy = Public}, loContent = LogMessage "14"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "[GET] /get?query=patate"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "200 OK in 0s"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Debug, privacy = Public}, loContent = LogMessage "14"}]) To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/GET, 200, with query/" test/unit/Network/Wai/Middleware/LoggingSpec.hs:184:13: 11) Network.Wai.Middleware.Logging, Logging Middleware, different request ids Falsifiable (after 4 tests and 6 shrinks): NumberOfRequests 1 expected: 1 but got: 48 To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/different request ids/" test/unit/Network/Wai/Middleware/LoggingSpec.hs:198:13: 12) Network.Wai.Middleware.Logging, Logging Middleware, correct time measures Falsifiable (after 1 test): (NumberOfRequests 5,RandomIndex 1) expected: 1 but got: 0 To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/correct time measures/" Randomized with seed 534946986 Finished in 629.7711 seconds 915 examples, 12 failures, 1 pending C:\Users\win\cw> ``` - This "fixes" the tests on windows. - "different request ids" test still fails # Comments It seems like iohk-monitoring writes its logs asynchronously. Or there is some other race condition with the TVar containing log messages. Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

974: tests: Fix Network.Wai.Middleware.LoggingSpec on Windows r=rvl a=rvl Relates to #703. # Overview ``` test/unit/Network/Wai/Middleware/LoggingSpec.hs:109:5: 9) Network.Wai.Middleware.Logging, Logging Middleware, GET, 200, no query uncaught exception: IOException of type UserError user error (Expected exactly 3 log entries but got 1: [LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "[GET] /get"}]) To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/GET, 200, no query/" test/unit/Network/Wai/Middleware/LoggingSpec.hs:117:5: 10) Network.Wai.Middleware.Logging, Logging Middleware, GET, 200, with query uncaught exception: IOException of type UserError user error (Expected exactly 3 log entries but got 5: [LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "200 OK in 0s"},LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Debug, privacy = Public}, loContent = LogMessage "14"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "[GET] /get?query=patate"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "200 OK in 0s"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Debug, privacy = Public}, loContent = LogMessage "14"}]) To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/GET, 200, with query/" test/unit/Network/Wai/Middleware/LoggingSpec.hs:184:13: 11) Network.Wai.Middleware.Logging, Logging Middleware, different request ids Falsifiable (after 4 tests and 6 shrinks): NumberOfRequests 1 expected: 1 but got: 48 To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/different request ids/" test/unit/Network/Wai/Middleware/LoggingSpec.hs:198:13: 12) Network.Wai.Middleware.Logging, Logging Middleware, correct time measures Falsifiable (after 1 test): (NumberOfRequests 5,RandomIndex 1) expected: 1 but got: 0 To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/correct time measures/" Randomized with seed 534946986 Finished in 629.7711 seconds 915 examples, 12 failures, 1 pending C:\Users\win\cw> ``` - This "fixes" the tests on windows. - "different request ids" test still fails # Comments It seems like iohk-monitoring writes its logs asynchronously. Or there is some other race condition with the TVar containing log messages. Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

986: LauncherSpec: Check that all launched processes do exit r=KtorZ a=rvl Relates to #703. # Overview It appears that the `jormungandr` node backend process was not getting cleaned up on Windows and that actions which are supposed to run concurrently with the backend were not running. These extra tests check that the [`withCreateProcess`](http://hackage.haskell.org/package/process-1.6.6.0/docs/System-Process.html#v:withCreateProcess) function takes care of terminating the process -- if it is given a chance (i.e. if the process is not killed with -9, or "End Task" on Windows). It also checks that the concurrent actions run while the backend process is running, and that the backend process is terminated when the other action completes. Finally, it implements a workaround for the unwanted behaviour of the `process` library on Windows where `waitForProcess` seems to block all concurrent async actions in the thread. - [x] Adds a test to `cardano-wallet-launcher:test:unit` for process clean up. - [x] Adjusts commands used so that the tests can be run under Wine. - [x] Adds an assertion to check that the process is killed if the action does not complete. - [x] Adds an assertion to check that the the process is killed promptly if the action completes. - [x] Fixes async blocking issue on Windows. ### Testing under Wine Use something like this: ``` wine $(nix-build release.nix -A x86_64-pc-mingw32.tests.cardano-wallet-launcher.unit.x86_64-linux -o launcher-unit-windows)/cardano-wallet-launcher-2019.11.7/unit.exe --match "Backend process" ``` Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

1016: Unit tests passing on both Windows and Wine r=KtorZ a=rvl Relates to #703. # Overview Fixes/disables some of the last tests that were failing on Windows for Microsoft reasons. 1017: Update stylish-haskell to latest (v0.9.4.4) r=KtorZ a=rvl Updates stylish-haskell to the latest. I mistakenly thought that the latest version in a Stackage LTS would be recent and downgraded it before. To test: ``` nix-shell --run "stylish-haskell --version" ``` Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

1015: nix: Make a separate testing bundle for windows r=KtorZ a=rvl Relates to #703. # Overview - Moves the test executables and test data out of the windows release zip and into a separate archive. # Comments - [release job](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-1015/cardano-wallet-jormungandr-win64/latest) - [testing bundle job](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-1015/cardano-wallet-jormungandr-tests-win64/latest) Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

1011: Can run Windows integration tests under Wine r=rvl a=rvl Relates to #703. # Overview - [ ] It's possible to execute `cardano-wallet-jormungandr:test:integration` under Wine. # Comments The tests run on Windows but there is something weird going on with Wine. ![2019-11-12-14-30-53-wine-integration-tests](https://user-images.githubusercontent.com/1019641/68679174-19e1db00-05ab-11ea-9c62-8a30e2807f77.png) To build and run the tests under wine, use: ``` wine $(nix-build release.nix -A x86_64-pc-mingw32.tests.cardano-wallet-jormungandr.integration.x86_64-linux -o integration-windows)/cardano-wallet-jormungandr-2019.11.7/integration.exe ``` 1028: Fix the initial sleep delay in the chain following r=KtorZ a=Anviking # Issue Number #1027 # Overview  - [x] I have changed the initial sleep delay from `0` to `1 s`. # Comments - Minor downside: we sleep for the initial delay on startup.   Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io> Co-authored-by: KtorZ <matthias.benkort@gmail.com> Co-authored-by: Johannes Lund <johannes.lund@iohk.io>

KtorZ · 2019-11-20T14:45:40Z

@rvl How are we doing with wine tests execution here? Wine doesn't work for me locally for a.. mysterious reason and as far as I can see, the Hydra job doesn't really work either.

rvl · 2019-11-21T05:47:26Z

We are waiting for execution of wine tests to be put back - input-output-hk/haskell.nix#316. Currently it builds the test suite for windows but does not run it, so it's always green in Hydra.

I don't know what version and configuration of wine you were using. But the version used in the nix build is capable of running the test suite and jormungandr.exe.

KtorZ · 2019-11-22T10:25:39Z

I am using the "default" one from aptitude on Ubuntu 18.04, so it's probably not the most recent version as usual ^^" ...
I'll try getting it installed through nix and see.

1109: Test.Utils.Paths: Fix the test data directory for Windows and Nix r=rvl a=rvl # Issue Number #703 # Overview When running tests under either Windows or Nix, we want the test data directory to be relative to the current directory, rather than an absolute path. # Comments Tests pass on Windows: - [x] `cardano-wallet-core-2019.11.18-test-unit.exe` - [x] `cardano-wallet-jormungandr-2019.11.18-test-unit.exe` - [ ] `cardano-wallet-jormungandr-2019.11.18-test-integration.exe` (_running_) - [x] `cardano-wallet-launcher-2019.11.18-test-unit.exe` - [x] `cardano-wallet-cli-2019.11.18-test-unit.exe` - [x] `bech32-2019.11.18-bech32-test.exe` - [x] `text-class-2019.11.18-test-unit.exe` Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

1109: Test.Utils.Paths: Fix the test data directory for Windows and Nix r=KtorZ a=rvl # Issue Number #703 # Overview When running tests under either Windows or Nix, we want the test data directory to be relative to the current directory, rather than an absolute path. # Comments Tests pass on Windows: - [x] `cardano-wallet-core-2019.11.18-test-unit.exe` - [x] `cardano-wallet-jormungandr-2019.11.18-test-unit.exe` - [ ] `cardano-wallet-jormungandr-2019.11.18-test-integration.exe` (_running_) - [x] `cardano-wallet-launcher-2019.11.18-test-unit.exe` - [x] `cardano-wallet-cli-2019.11.18-test-unit.exe` - [x] `bech32-2019.11.18-bech32-test.exe` - [x] `text-class-2019.11.18-test-unit.exe` Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>

piotr-iohk · 2019-12-11T13:06:03Z

@KtorZ KtorZ added this to QA in cardano-wallet 2 hours ago

Does it mean that tests are running on wine in CI?

KtorZ · 2019-12-11T13:13:54Z

Whoops. My bad, I messed up when setting up the GH board

KtorZ · 2020-01-10T09:31:39Z

closing this one. Tests are now running in hydra:

unit

integration

https://hydra.iohk.io/job/Cardano/cardano-wallet/x86_64-pc-mingw32.checks.cardano-wallet-jormungandr.integration.x86_64-linux

Some jobs are still failing but this is tracked by #1115

KtorZ assigned iohk-devops Sep 13, 2019

rvl mentioned this issue Sep 16, 2019

Fix windows build #717

Merged

rvl mentioned this issue Sep 29, 2019

Run tests in nix release build #766

Merged

10 tasks

KtorZ added this to the Usability & Compatibility milestone Oct 16, 2019

rvl assigned rvl and unassigned iohk-devops Oct 23, 2019

This was referenced Oct 23, 2019

Update Haskell.nix #888

Merged

nix: Update windows jormungandr version and the instructions #906

Merged

This was referenced Oct 25, 2019

Fix text locale encoding issues on windows #914

Merged

Fix Sqlite DB on Windows #918

Merged

This was referenced Nov 5, 2019

cli: ANSI colour for windows 10, and easier utf-8 setup #957

Merged

nix: Add necessary test data files to windows testing bundle #958

Merged

rvl mentioned this issue Nov 5, 2019

LauncherSpec: Let it work on Windows #959

Merged

rvl mentioned this issue Nov 7, 2019

tests: Fix ServerSpec and SqliteSpec on windows #971

Merged

This was referenced Nov 7, 2019

tests: Fix Network.Wai.Middleware.LoggingSpec on Windows #974

Merged

Fix bench-db on windows #975

Merged

KtorZ mentioned this issue Nov 7, 2019

Make API Golden tests filenames compatible with Windows #981

Merged

2 tasks

rvl mentioned this issue Nov 8, 2019

LauncherSpec: Check that all launched processes do exit #986

Merged

5 tasks

rvl mentioned this issue Nov 12, 2019

Can run Windows integration tests under Wine #1011

Merged

1 task

This was referenced Nov 12, 2019

nix: Make a separate testing bundle for windows #1015

Merged

Unit tests passing on both Windows and Wine #1016

Merged

rvl mentioned this issue Dec 3, 2019

Test.Utils.Paths: Fix the test data directory for Windows and Nix #1109

Merged

7 tasks

piotr-iohk mentioned this issue Dec 4, 2019

Faililing tests integration tests on Windows #1115

Closed

piotr-iohk mentioned this issue Jan 8, 2020

Use database temp dir for wallet db in integration tests instead of in-memory DB #1240

Closed

1 task

KtorZ closed this as completed Jan 10, 2020

piotr-iohk mentioned this issue Jan 20, 2020

Windows tests are failing on hydra #1283

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wallet Backend builds and tests on Windows #703

Wallet Backend builds and tests on Windows #703

KtorZ commented Sep 13, 2019 •

edited

Loading

rvl commented Sep 16, 2019 •

edited

Loading

KtorZ commented Oct 15, 2019 •

edited

Loading

KtorZ commented Nov 20, 2019

rvl commented Nov 21, 2019

KtorZ commented Nov 22, 2019

piotr-iohk commented Dec 11, 2019

KtorZ commented Dec 11, 2019

KtorZ commented Jan 10, 2020

Wallet Backend builds and tests on Windows #703

Wallet Backend builds and tests on Windows #703

Comments

KtorZ commented Sep 13, 2019 • edited Loading

Context

Decision

Acceptance Criteria

Development Plan

QA

Checking for windows builds and tests in CI

Running tests on windows

Building windows testing bundle locally

Running tests with Wine

rvl commented Sep 16, 2019 • edited Loading

How to test cardano-wallet-jormungandr on Windows

KtorZ commented Oct 15, 2019 • edited Loading

KtorZ commented Nov 20, 2019

rvl commented Nov 21, 2019

KtorZ commented Nov 22, 2019

piotr-iohk commented Dec 11, 2019

KtorZ commented Dec 11, 2019

KtorZ commented Jan 10, 2020

unit

integration

KtorZ commented Sep 13, 2019 •

edited

Loading

rvl commented Sep 16, 2019 •

edited

Loading

KtorZ commented Oct 15, 2019 •

edited

Loading