Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(custom-checks): fix broken package.json sort verification #2356

Closed
petermetz opened this issue Mar 29, 2023 · 2 comments · Fixed by #2434
Closed

ci(custom-checks): fix broken package.json sort verification #2356

petermetz opened this issue Mar 29, 2023 · 2 comments · Fixed by #2434
Assignees
Labels
Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) P3 Priority 3: Medium Tests Anything related to tests be that automatic or manual, integration or unit, etc.

Comments

@petermetz
Copy link
Contributor

Description

As a maintainer I want to have an automated CI job (that actually works) that checks for package.json files being sorted so that I don't have to manually be on the lookout for non-sorted dependencies and the like.

Previous attempt at making this happen: https://github.com/hyperledger/cacti/issues/1341

A related discussion I had with @outSH about this: https://github.com/hyperledger/cacti/pull/2310#discussion_r1152057132

Acceptance Criteria

  1. The custom checks fail in the CI if any package.json files are found which were previously unsorted.
  2. When the custom checks are executed locally, the ensure sort check can still be used to perform the sorting so that contributors
@petermetz petermetz self-assigned this Mar 29, 2023
@petermetz petermetz added Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) Tests Anything related to tests be that automatic or manual, integration or unit, etc. P3 Priority 3: Medium labels Apr 6, 2023
rwat17 added a commit to rwat17/cactus that referenced this issue May 23, 2023
     - Remove not working check-package-json-sort.ts script.
     - Add ci action to check for unsorted package.json files.

Closes: hyperledger-cacti#2356
Signed-off-by: Tomasz Awramski <tomasz.awramski@fujitsu.com>
@rwat17
Copy link
Contributor

rwat17 commented May 23, 2023

@petermetz
I have done some work towards this issue. Please review if it suits your requirements.

Found error in usage of sort-package in scripts introduced in this issue https://github.com/hyperledger/cacti/issues/1341.
Module API accepts only JSON objects https://www.npmjs.com/package/sort-package-json. But here https://github.com/hyperledger/cacti/blob/3e6d908cb4cf732ef057a30df3e8267381f3f806/tools/custom-checks/check-package-json-sort.ts#L51 only paths to the files are passed.

I did simple CI action to check if package.json files in /examples and /packages are sorted. If not, it prints short script to run in cactus root folder to sort remaining packages.

rwat17 added a commit to rwat17/cactus that referenced this issue May 23, 2023
     - Remove not working check-package-json-sort.ts script.
     - Add ci action to check for unsorted package.json files.

Closes: hyperledger-cacti#2356
Signed-off-by: Tomasz Awramski <tomasz.awramski@fujitsu.com>
rwat17 added a commit to rwat17/cactus that referenced this issue Jun 6, 2023
    - Remove not working check-package-json-sort.ts script.
    - Add ci action to check for unsorted package.json files.

Closes: hyperledger-cacti#2356
Signed-off-by: Tomasz Awramski <tomasz.awramski@fujitsu.com>
@petermetz
Copy link
Contributor Author

@petermetz I have done some work towards this issue. Please review if it suits your requirements.

Found error in usage of sort-package in scripts introduced in this issue #1341. Module API accepts only JSON objects https://www.npmjs.com/package/sort-package-json. But here

https://github.com/hyperledger/cacti/blob/3e6d908cb4cf732ef057a30df3e8267381f3f806/tools/custom-checks/check-package-json-sort.ts#L51

@rwat17 Thank you very much!

My preferred way would be to solve it via the existing NodeJS script instead of going down a level to writing shell because of these questions that popped into my head:

Questions:

  1. Did you verify the shell script to also be working for Mac users? We've had lots of bugs in the past where tiny little implementation details made things broken on either Linux or mac and also Windows back when our build was still supported on it (and we should - in my opinion - eventually get the build working again on Windows/PowerShell as well)
  2. Can the shell script be changed/extended (relatively easily) later to pull package path patterns (globs) from the root folder's lerna.json file? This is a future change we'll need to change because of the cactus -> cacti rename all of these hardcoded globs will start failing soon. I'm thinking maybe with some clever jq magic it can be made so that the shell script parses the lerna.json file for the globs and then iterates through them but I'm a little worried that by that point the shell code will be hard to read/understand for beginners.
  3. Is there an easy way to plug in the shell script into the existing set of "custom checks" that get executed when we run yarn run custom-checks? The only way I can think of is that the custom-checks script starts spawning other processes to run the shell script, not necessarily bad but it does increase complexity 'coz now we have to write logic to parse and interpret the stdout/stderr of that process in addition to just running the check itself.

If all of these are OK in your opinion then I'd be fine with the shell script instead of the NodeJS script, just let me know your thoughts!

rwat17 added a commit to rwat17/cactus that referenced this issue Jul 12, 2023
     - Remove not working check-package-json-sort.ts script.
     - Add ci action to check for unsorted package.json files.

Closes: hyperledger-cacti#2356
Signed-off-by: Tomasz Awramski <tomasz.awramski@fujitsu.com
rwat17 added a commit to rwat17/cactus that referenced this issue Aug 22, 2023
     - Remove not working check-package-json-sort.ts script.
     - Add ci action to check for unsorted package.json files.

Closes: hyperledger-cacti#2356
Signed-off-by: Tomasz Awramski <tomasz.awramski@fujitsu.com
rwat17 added a commit to rwat17/cactus that referenced this issue Aug 22, 2023
     - Remove not working check-package-json-sort.ts script.
     - Add ci action to check for unsorted package.json files.

Closes: hyperledger-cacti#2356
Signed-off-by: Tomasz Awramski <tomasz.awramski@fujitsu.com
petermetz pushed a commit to rwat17/cactus that referenced this issue Aug 25, 2023
- Remove not working check-package-json-sort.ts script.
- Add ci action to check for unsorted package.json files.

Peter's changes:
1. Changed the commit message in a way that it won't show up in the
release notes among the bug fixes (the section that is meant for production
bug-fixes not tooling bug-fixes)
2. Fixed typos in the .ts scripts
3. Applied the prettier/eslint formatting, deleted unused variables
4. Added a convenience script to the root package.json that runs the
package.json sorter
5. Fixed the sorter script so that it appends a newline character after
the sorted JSON string when writing back to the file. This is needed
because when you run `yarn install` or `npm install` they both like to
close the package.json file's with an empty line last so we have to mimic
that in order to avoid spamming all future PRs with these line changes.
One problem that this might still have is that on different operating
systems the newline character might be different and it also depends on
the git configuration in effect (you can configure Windows' git to use
Unix line endings for example). So it's not trivial how to fix it but
we'll cross that bridge when we get to it.

Closes: hyperledger-cacti#2356

Co-authored-by: Peter Somogyvari <peter.somogyvari@accenture.com>

Signed-off-by: Tomasz Awramski <tomasz.awramski@fujitsu.com
Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
petermetz pushed a commit that referenced this issue Aug 25, 2023
- Remove not working check-package-json-sort.ts script.
- Add ci action to check for unsorted package.json files.

Peter's changes:
1. Changed the commit message in a way that it won't show up in the
release notes among the bug fixes (the section that is meant for production
bug-fixes not tooling bug-fixes)
2. Fixed typos in the .ts scripts
3. Applied the prettier/eslint formatting, deleted unused variables
4. Added a convenience script to the root package.json that runs the
package.json sorter
5. Fixed the sorter script so that it appends a newline character after
the sorted JSON string when writing back to the file. This is needed
because when you run `yarn install` or `npm install` they both like to
close the package.json file's with an empty line last so we have to mimic
that in order to avoid spamming all future PRs with these line changes.
One problem that this might still have is that on different operating
systems the newline character might be different and it also depends on
the git configuration in effect (you can configure Windows' git to use
Unix line endings for example). So it's not trivial how to fix it but
we'll cross that bridge when we get to it.

Closes: #2356

Co-authored-by: Peter Somogyvari <peter.somogyvari@accenture.com>

Signed-off-by: Tomasz Awramski <tomasz.awramski@fujitsu.com
Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
sandeepnRES pushed a commit to sandeepnRES/cacti that referenced this issue Dec 21, 2023
- Remove not working check-package-json-sort.ts script.
- Add ci action to check for unsorted package.json files.

Peter's changes:
1. Changed the commit message in a way that it won't show up in the
release notes among the bug fixes (the section that is meant for production
bug-fixes not tooling bug-fixes)
2. Fixed typos in the .ts scripts
3. Applied the prettier/eslint formatting, deleted unused variables
4. Added a convenience script to the root package.json that runs the
package.json sorter
5. Fixed the sorter script so that it appends a newline character after
the sorted JSON string when writing back to the file. This is needed
because when you run `yarn install` or `npm install` they both like to
close the package.json file's with an empty line last so we have to mimic
that in order to avoid spamming all future PRs with these line changes.
One problem that this might still have is that on different operating
systems the newline character might be different and it also depends on
the git configuration in effect (you can configure Windows' git to use
Unix line endings for example). So it's not trivial how to fix it but
we'll cross that bridge when we get to it.

Closes: hyperledger-cacti#2356

Co-authored-by: Peter Somogyvari <peter.somogyvari@accenture.com>

Signed-off-by: Tomasz Awramski <tomasz.awramski@fujitsu.com
Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Flaky-Test-Automation Issues related to test stability (which is a long running issue that can never fully be solved) P3 Priority 3: Medium Tests Anything related to tests be that automatic or manual, integration or unit, etc.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants