Errors in GitLab CI Pipeline Setup for Specmatic Integration #1098

gurubabujampala · 2024-05-08T08:00:17Z

gurubabujampala
May 8, 2024

"I have created a specmatic.json file within the consumer service and updated with central contract repository git URL. However, during the project build in the GitLab CI pipeline, I encountered an authentication error stating 'Authentication is required but no CredentialsProvider has been registered'.

I attempted to provide the Git credentials through the CICD variables using the format https://$USERNAME:$PASSWORD@gitlab.com/git/central-contract.git. Despite this, I am still receive the same error, and it appears that these CICD variables are not being substituted during the build execution.

I had a looked into similar discussion here #900 but I can't go in this direction.

Any recommendation approach (documentation) from specmatic team, to get it done in Gitlab CI pipeline?

Sample specmatic.json for reference

{
  "sources": [
    {
      "provider": "git",
      "repository": "https://gitlab.com/git/specmatic/central-contract.git",
      "branch": "modified-specs",
      "test": [
        "customer/customer.yaml"
      ],
      "stub": [
        "order/order.yaml"
      ]
    }
  ]
}

Thanks!

UPDATE (by @harikrishnan83): Fixed link to related discussion

Answered by harikrishnan83

May 12, 2024

hello @gurubabujampala, Specmatic now substitutes env variables that are part of the Git repo URL that you mention in specmatic.json. This is part of the latest release 1.3.18. Here is the link to the documentation. Hope this helps. Thanks.

View full answer

harikrishnan83 · 2024-05-08T11:07:51Z

harikrishnan83
May 8, 2024
Maintainer

Hello @gurubabujampala , thanks for reaching out.

An access token needs to be setup and passed to Specmatic so that it can then clone your Central Contract Repo while running in CI. Here is the documentation on how we achieve this on Github actions. Also here is a complete workflow YAML file for your reference - workflow file.

We will get back to you shortly with similar instructions for Gitlab.

1 reply

gurubabujampala May 9, 2024
Author

Hi @harikrishnan83 , thanks for your response. I tried by setting the PERSONAL_ACCESS_TOKEN in Gitlab variables and specmatic is able to read it. But unfortunately it didn't solve my issue. It's not giving me the authentication error this time instead it's giving authentication not supported.

Error Log:

Loading config file ./specmatic.json
Looking for a contract repo checkout at: /builds/git/specmatic/customer/.specmatic/repos/central-contract
Contract repo does not exist.
Cloning https://gitlab.com/git/specmatic/central-contract.git into .specmatic/repos
Resetting /builds/git/specmatic/customer/.specmatic/repos/central-contract
**Using personal access token from environment variable
Using personal access token from environment variable**
Cloning: https://gitlab.com/git/specmatic/central-contract.git -> /builds/git/specmatic/customer/.specmatic/repos/central-contract
24665 [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 6.051 s <<< FAILURE! -- in com.customer.contract.ContractTest
24672 [ERROR] com.customer.contract.ContractTest -- Time elapsed: 6.051 s <<< ERROR!
org.eclipse.jgit.api.errors.TransportException: **https://gitlab.com/git/specmatic/central-contract.git:** **authentication not supported**
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:249)
	at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:319)
	at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:189)
	at in.specmatic.core.git.GitOperations.jgitClone(GitOperations.kt:107)
	at in.specmatic.core.git.GitOperations.clone(GitOperations.kt:56)
	at in.specmatic.core.git.GitOperations.clone(GitOperations.kt:32)
	at in.specmatic.core.utilities.GitRepo.cloneRepoAndCheckoutBranch(GitRepo.kt:110)
	at in.specmatic.core.utilities.GitRepo.loadContracts(GitRepo.kt:66)
	at in.specmatic.core.utilities.Utilities.contractFilePathsFrom(Utilities.kt:284)
	at in.specmatic.core.utilities.Utilities.contractStubPaths(Utilities.kt:255)
	at in.specmatic.stub.API.createStub(api.kt:39)
	at com.customer.contract.ContractTest.setup(ContractTest.java:36)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

harikrishnan83 · 2024-05-12T18:46:40Z

harikrishnan83
May 12, 2024
Maintainer

hello @gurubabujampala, Specmatic now substitutes env variables that are part of the Git repo URL that you mention in specmatic.json. This is part of the latest release 1.3.18. Here is the link to the documentation. Hope this helps. Thanks.

4 replies

gurubabujampala May 13, 2024
Author

Hi @harikrishnan83 , thanks for very quick fix on this. I tried the approach and it was able to connect Git repo. But unfortunately I am getting below issue in the pipeline.

Repo URL: https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/contract-testing/central-contract-repo.git/

Error Log:
26706 [ERROR] com.customer.contract.ContractTest -- Time elapsed: 7.312 s <<< ERROR!
ContractException(errorMessage=Error running git check-ignore .specmatic: Cannot run program "git" (in directory "/builds/git/specmatic/customer/."): error=2, No such file or directory, breadCrumb=, exceptionCause=java.io.IOException: Cannot run program "git" (in directory "/builds/git/specmatic/customer/."): error=2, No such file or directory, scenario=null, isCycle=false)
at in.specmatic.core.utilities.ExternalCommand.executeAsSeparateProcess(ExternalCommand.kt:39)
at in.specmatic.core.git.SystemGit.executeCommandWithWorkingDirectory(SystemGit.kt:36)
at in.specmatic.core.git.SystemGit.execute(SystemGit.kt:24)
at in.specmatic.core.git.SystemGit.checkIgnore(SystemGit.kt:79)
at in.specmatic.core.utilities.GitRepo.isSpecmaticFolderIgnored(GitRepo.kt:104)
at in.specmatic.core.utilities.GitRepo.ensureThatSpecmaticFolderIsIgnored(GitRepo.kt:120)
at in.specmatic.core.utilities.GitRepo.cloneRepoAndCheckoutBranch(GitRepo.kt:115)
at in.specmatic.core.utilities.GitRepo.loadContracts(GitRepo.kt:66)
at in.specmatic.core.utilities.Utilities.contractFilePathsFrom(Utilities.kt:284)
at in.specmatic.core.utilities.Utilities.contractStubPaths(Utilities.kt:255)
at in.specmatic.stub.API.createStub(api.kt:39)
at com.customer.contract.ContractTest.setup(ContractTest.java:36)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.io.IOException: Cannot run program "git" (in directory "/builds/git/specmatic/customer/."): error=2, No such file or directory
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
at in.specmatic.core.utilities.ExternalCommand.executeAsSeparateProcess(ExternalCommand.kt:29)
... 13 more
Caused by: java.io.IOException: error=2, No such file or directory
at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
at java.base/java.lang.ProcessImpl.(ProcessImpl.java:314)
at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
... 15 more
26791 [INFO]
26791 [INFO] Results:
26791 [INFO]
26791 [ERROR] Errors:
26792 [ERROR] ContractTest.setup:36 » Contract Error running git check-ignore .specmatic: Cannot run program "git" (in directory "/builds/git/specmatic/customer/."): error=2, No such file or directory

Note: It looks unable to find the .specmatic to do the git clone. But not sure why it's looking under the /build/git/specmatic/customer folder instead of /build/git/specmatic/customer/.specmatic folder.

harikrishnan83 May 13, 2024
Maintainer

hello @gurubabujampala , good to know you were able to connect to Git repo with this approach.

This error indicates that your build machine does not have Git command installed on it (Please correct me if I am wrong). Specmatic needs Git command to be available on the command line of the build machine in order to check if the specifications are the latest etc.

Here is an example of we have Git on our build machine.

image: openjdk:17-alpine

stages:
  - test

test:
  stage: test
  script:
    - apk add --no-cache git # install git
    # Run you regular build (mvn, gradle, etc.) and test here

Please let me know if you have any questions on this. Thanks.

gurubabujampala May 13, 2024
Author

Hi @harikrishnan83, yes I tried this approach and was able to execute git commands in the pipeline.

mvn-build:
  stage: build
  before_script:
    - apk add --no-cache git

Thank you very much

harikrishnan83 May 14, 2024
Maintainer

@gurubabujampala that is great. Thanks for letting us know. I am marking this response as the answer to this discussion. If you have any more questions on this please feel free to add to this discussion.

prateekkarmakar · 2024-10-30T08:09:27Z

prateekkarmakar
Oct 30, 2024

Hi @harikrishnan83, we are trying to use the bamboo pipeline for build we have set up the git lab access token in bamboo variables. Once we start to build we are getting the below error that token is not set. We are using the same token that is used for accessing the repo manually its working in that case.

build 30-Oct-2024 07:38:07 Loading config file ./specmatic.json
build 30-Oct-2024 07:38:07 Looking for a contract repo checkout at: /opt/bamboo_agent/xml-data/build-dir/QECOEITAF-P2OCI4-JOB1/.specmatic/repos/central-contract
build 30-Oct-2024 07:38:07 Contract repo does not exist.
build 30-Oct-2024 07:38:07 Cloning https://gitlab-ci-token:${bamboo.CI_JOB_TOKEN}@gitlab.tools.telstra.com/telstra/git/GQGIIG/specmatic/central-contract.git into .specmatic/repos
build 30-Oct-2024 07:38:07 Resetting /opt/bamboo_agent/xml-data/build-dir/QECOEITAF-P2OCI4-JOB1/.specmatic/repos/central-contract
build 30-Oct-2024 07:38:07 bamboo.CI_JOB_TOKEN in https://gitlab-ci-token:${bamboo.CI_JOB_TOKEN}@gitlab.tools.telstra.com/telstra/git/GQGIIG/specmatic/central-contract.git resembles an environment variable, but skipping evaluation since value for the same is not set.
build 30-Oct-2024 07:38:07 [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.277 s <<< FAILURE! - in com.telstra.p2o.customerinfo.contracts.ContractTest
build 30-Oct-2024 07:38:07 [ERROR] com.telstra.p2o.customerinfo.contracts.ContractTest Time elapsed: 0.277 s <<< ERROR!
build 30-Oct-2024 07:38:07 in.specmatic.core.git.NonZeroExitError:
build 30-Oct-2024 07:38:07 Error executing git clone https://gitlab-ci-token:${bamboo.CI_JOB_TOKEN}@gitlab.tools.telstra.com/telstra/git/GQGIIG/specmatic/central-contract.git /opt/bamboo_agent/xml-data/build-dir/QECOEITAF-P2OCI4-JOB1/.specmatic/repos/central-contract
build 30-Oct-2024 07:38:07 ERRORS: Cloning into '/opt/bamboo_agent/xml-data/build-dir/QECOEITAF-P2OCI4-JOB1/.specmatic/repos/central-contract'...
build 30-Oct-2024 07:38:07 remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://gitlab.tools.telstra.com/help/topics/git/troubleshooting_git#error-on-git-fetch-http-basic-access-denied
build 30-Oct-2024 07:38:07 fatal: Authentication failed for 'https://gitlab.tools.telstra.com/telstra/git/GQGIIG/specmatic/central-contract.git/'
build 30-Oct-2024 07:38:07

Do we need to follow some different steps to get the token while in bamboo pipeline?

2 replies

harikrishnan83 Nov 20, 2024
Maintainer

@prateekkarmakar sorry I missed this message as it was part of an answered question and this is more about Github Actions. Can you please create a separate discussion for Bamboo CI pipelines please?

harikrishnan83 Nov 20, 2024
Maintainer

@prateekkarmakar I see this error in the logs you have shared.

build 30-Oct-2024 07:38:07 bamboo.CI_JOB_TOKEN in [https://gitlab-ci-token:${bamboo.CI_JOB_TOKEN}@gitlab.tools.telstra.com/telstra/git/GQGIIG/specmatic/central-contract.git](https://gitlab-ci-token:$%7Bbamboo.CI_JOB_TOKEN%7D@gitlab.tools.telstra.com/telstra/git/GQGIIG/specmatic/central-contract.git) resembles an environment variable, but skipping evaluation since value for the same is not set.

Ideally you should extract the bamboo.CI_JOB_TOKEN into an env variable like BAMBOO_CI_JOB_TOKEN so that it can be picked by Specmatic. Please refer to this documentation and let me know if you have questions. Thanks.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Errors in GitLab CI Pipeline Setup for Specmatic Integration #1098

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments 7 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Errors in GitLab CI Pipeline Setup for Specmatic Integration #1098

gurubabujampala May 8, 2024

Replies: 3 comments · 7 replies

harikrishnan83 May 8, 2024 Maintainer

gurubabujampala May 9, 2024 Author

harikrishnan83 May 12, 2024 Maintainer

gurubabujampala May 13, 2024 Author

harikrishnan83 May 13, 2024 Maintainer

gurubabujampala May 13, 2024 Author

harikrishnan83 May 14, 2024 Maintainer

prateekkarmakar Oct 30, 2024

harikrishnan83 Nov 20, 2024 Maintainer

harikrishnan83 Nov 20, 2024 Maintainer

gurubabujampala
May 8, 2024

Replies: 3 comments 7 replies

harikrishnan83
May 8, 2024
Maintainer

gurubabujampala May 9, 2024
Author

harikrishnan83
May 12, 2024
Maintainer

gurubabujampala May 13, 2024
Author

harikrishnan83 May 13, 2024
Maintainer

gurubabujampala May 13, 2024
Author

harikrishnan83 May 14, 2024
Maintainer

prateekkarmakar
Oct 30, 2024

harikrishnan83 Nov 20, 2024
Maintainer

harikrishnan83 Nov 20, 2024
Maintainer