Moved Script Evaluation Test to GHA #5095

zeme-iohk · 2023-01-31T07:57:23Z

Added ./.github/workflows/script-evaluation-test.yml
Some renames and "cosmetics" to the other workflow files
Partly tested locally using act and a fork of the repo.
Must be pushed to master for proper e2e test

zeme-iohk · 2023-01-31T08:17:04Z

.github/workflows/script-evaluation-test.yml

+        run: |
+          export EVENT_DUMP_DIR="$HOME/mainnet-script-dump-downloaded"
+          nix develop --no-warn-dirty --accept-flake-config --command cabal update 
+          nix develop --no-warn-dirty --accept-flake-config --command \


We need two calls to nix develop unfortunately, this won't work:

nix develop --no-warn-dirty --accept-flake-config --command \ cabal update && cabal v2-run ...

As
nix develop --command won't allow this syntax -> it breaks at &&

Alternatively, what about

nix run .#x86_64-linux.plutus.library.plutus-project-924.hsPkgs.plutus-ledger-api.components.exes.evaluation-test -- --num-threads=1

zeme-iohk · 2023-01-31T08:17:55Z

.github/workflows/script-evaluation-test.yml

+          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          AWS_DEFAULT_REGION: us-east-1
+          AWS_ENDPOINT_URL: https://s3.devx.iog.io


Can someone confirm that these two are correct? AWS_ENDPOINT_URL and AWS_DEFAULT_REGION
Maybe @Pacman99

zeme-iohk · 2023-01-31T08:18:28Z

.github/workflows/script-evaluation-test.yml

+name: Script Evaluation Test
+on:
+  schedule:
+    - cron: 30 3 * * * # 3:30am every day 


I chose at random, happy to change

michaelpj · 2023-01-31T12:09:18Z

.github/workflows/script-evaluation-test.yml

+        run: |
+          export LOCAL_DIR="$HOME/mainnet-script-dump-downloaded" 
+          nix develop --no-warn-dirty --accept-flake-config --command \
+            bash ./scripts/s3-sync-unzip.sh s3://plutus/mainnet-script-dump/ \*.event.bz2


Since we're running on a persistent machine, is there a way we can save this somewhere persistent and only sync the remainder?

I think the file $HOME/mainnet-script-dump-downloaded is going to persist across runs, no?

Oh I see. I also don't know if that's true. Maybe we should check with @Pacman99 .

I also don't know if that script does things incrementally, probably a question for @zliu41

michaelpj · 2023-01-31T12:25:06Z

.github/workflows/script-evaluation-test.yml

+        run: |
+          export EVENT_DUMP_DIR="$HOME/mainnet-script-dump-downloaded"
+          nix develop --no-warn-dirty --accept-flake-config --command cabal update 
+          nix develop --no-warn-dirty --accept-flake-config --command \


Alternatively, what about

nix run .#x86_64-linux.plutus.library.plutus-project-924.hsPkgs.plutus-ledger-api.components.exes.evaluation-test -- --num-threads=1

michaelpj · 2023-01-31T12:26:46Z

build/buildkite/script-evaluation-test.yml

-#     TODO(std) check that this works 
-#     nix develop --command 'cabal update && EVENT_DUMP_DIR=$HOME/mainnet-script-dump-downloaded cabal v2-run plutus-ledger-api:evaluation-test -- --num-threads=1'
-    concurrency: 1
-    concurrency_group: "plutus-script-evaluation"


Is this important? In fact, shouldn't we have a concurrency group for all things that use the benchmarking machine? or is that handled by having the benchmarking machine only accept one job at once?

yeah I think it's important, we can also cancel previous jobs actually. see my update.

michaelpj · 2023-01-31T15:03:43Z

.github/workflows/benchmark.yml

@@ -3,6 +3,10 @@ on:
  issue_comment:
    types: [created]

+concurrency:
+  group: plutus-benchmark
+  cancel-in-progress: true


That seems wrong. If people make benchmark requests on multiple PRs, we want them to queue up and all get executed.

Another reason why it's worth writing explanatory comments for stuff like this!

ok, will update

michaelpj · 2023-01-31T15:04:18Z

.github/workflows/benchmark.yml

@@ -3,6 +3,10 @@ on:
  issue_comment:
    types: [created]

+concurrency:
+  group: plutus-benchmark
+  cancel-in-progress: true


Another reason why it's worth writing explanatory comments for stuff like this!

michaelpj · 2023-01-31T15:04:48Z

.github/workflows/script-evaluation-test.yml

@@ -4,6 +4,10 @@ on:
  schedule:
    - cron: 30 3 * * * # 3:30am every day 

+concurrency:
+  group: script-evaluation-test
+  cancel-in-progress: true


This seems okay... we shouldn't ever end up with multiple of these in flight? Worth a comment.

michaelpj · 2023-01-31T15:05:11Z

.github/workflows/script-evaluation-test.yml

-          nix develop --no-warn-dirty --accept-flake-config --command \
-            cabal v2-run plutus-ledger-api:evaluation-test -- --num-threads=1
+          Alternatively, what about
+          nix run .#x86_64-linux.plutus.library.plutus-project-924.hsPkgs.plutus-ledger-api.components.exes.evaluation-test -- --num-threads=1


We should test this, I'm not 100% sure whether it will work.

tested locally

zeme-wana added 2 commits January 31, 2023 14:49

Moved Script Evaluation Test to GHA

c60718d

Cosmetics

78c42b1

zeme-iohk requested review from zliu41 and michaelpj January 31, 2023 08:11

zeme-iohk marked this pull request as ready for review January 31, 2023 08:15

zeme-iohk commented Jan 31, 2023

View reviewed changes

michaelpj approved these changes Jan 31, 2023

View reviewed changes

zeme-wana added 2 commits January 31, 2023 21:38

Applied requested changes

4669bc2

WIP

fcc5ac4

michaelpj reviewed Jan 31, 2023

View reviewed changes

WIP

dd93a97

michaelpj approved these changes Jan 31, 2023

View reviewed changes

zeme-wana added 2 commits January 31, 2023 22:07

WIP

78a2cee

WIP

ebca464

zeme-iohk merged commit 5e6bda0 into master Feb 1, 2023

zeme-iohk deleted the zeme-iohk/set branch February 1, 2023 06:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Moved Script Evaluation Test to GHA #5095

Moved Script Evaluation Test to GHA #5095

zeme-iohk commented Jan 31, 2023 •

edited

Loading

zeme-iohk Jan 31, 2023 •

edited

Loading

michaelpj Jan 31, 2023

zeme-iohk Jan 31, 2023

zeme-iohk Jan 31, 2023

michaelpj Jan 31, 2023

zeme-iohk Jan 31, 2023

michaelpj Jan 31, 2023

michaelpj Jan 31, 2023

michaelpj Jan 31, 2023

zeme-iohk Jan 31, 2023

michaelpj Jan 31, 2023

michaelpj Jan 31, 2023

zeme-iohk Jan 31, 2023

michaelpj Jan 31, 2023

michaelpj Jan 31, 2023

michaelpj Jan 31, 2023

zeme-iohk Jan 31, 2023

Moved Script Evaluation Test to GHA #5095

Moved Script Evaluation Test to GHA #5095

Conversation

zeme-iohk commented Jan 31, 2023 • edited Loading

zeme-iohk Jan 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zeme-iohk commented Jan 31, 2023 •

edited

Loading

zeme-iohk Jan 31, 2023 •

edited

Loading