/bin/sh: 5: source: not found

[Timple]

Our images in the bitbucket-pipelines have bash as default runner. Can this be adhered?

This does not only affect the source command, which we could easily replace by .. But also environment variables that should be present are not there.

[ktomk]

Good question. /bin/sh has the design decision as we expect every Docker container to have it. This is albeit we're aware that Atlassian Bitbucket Pipelines supports both /bin/sh and /bin/bash (and IIRC prefers /bin/bash if available).

A couple of questions for clarification:

• Can you share the path of bash in your scenario within the container?
• Is only the script and after-script execution affected by it? (Copying files, caches and artifacts work?)
• Can you provide an exemplary pipleines YAML w/ such a script to test with?

We can then figure something out how to better support the bash use-case I'm pretty sure.

The second issue with the environment variables that are not present is less clear to me, perhaps if an exemplary script could show this as well would be helpful to write more.

[Timple]

We're using ubuntu dockers as base images (but with bash as SHELL).

• So path is /bin/bash

• Yes, no other issues

• This should do it:

FROM ubuntu:latest
SHELL ["/bin/bash", "-c"] 

docker build -t ubuntu-bash .

image: ubuntu-bash
pipelines:
  default:
    - step:
        name: Build and test
        script:
          - source /etc/os-release
          - echo "$PRETTY_NAME"

[ktomk]

Thanks. Is the PRETTY_NAME parameter non-null in the good case and null/undefined in the reported issue?

[Timple]

Yes, because the content of /etc/os-release is:

$ cat /etc/os-release 
PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
...

(I put the wrong file to source in the example, copy past error. Fixed that now, so please modify in your trials as well)

[ktomk]

Please find a temporary topic branch issue-17-bash in action. Build phar files (choose any) will be kept for two days so they can be used for early user testing.

I target release version 0.0.66 for the new bash runner feature.

[Timple]

Thank you for taking the time to fix this!
This did fix the example I posted. 🙂

Unfortunately it still doesn't work for our full system. So I've put together a new example :/

FROM ubuntu:latest
RUN echo "HELLO=YOU" > ~/.bashrc
SHELL ["/bin/bash", "-c"]

image: ubuntu-bash
pipelines:
  default:
    - step:
        name: Build and test
        script:
          - echo "$HELLO"
          - source /etc/os-release
          - echo "$PRETTY_NAME"

Now the outcome is the same as from the docker command:

$ docker run ubuntu-bash echo "$HELLO"
<nothing>

$ ~/Downloads/pipelines.phar 
+++ step #1

    name...........: "Build and test"
    effective-image: ubuntu-bash
    container......: pipelines-1.Build-and-test.default.test
    container-id...: 7e400d87b793

+++ copying files into container...


+ echo "$HELLO"


+ source /etc/os-release

+ echo "$PRETTY_NAME"
Ubuntu 22.04 LTS

However, inside the docker:

$ docker run -it ubuntu-bash
root@10dc7d3e3e46:/# echo $HELLO
YOU

So from within the docker all variables declared in the .bashrc file are present. Looks like the pipelines run inside the dockerfile. This might be a completely different concept than what this project is doing?

[ktomk]

docker run -it on (effectively) ubuntu:latest has CMD ["bash"] and loads ~/.bashrc before dropping into the shell prompt.

pipelines runs ubuntu:latest detached with an interactive shell (but /bin/sh) to keep it running. Then it executes the step script via docker exec in the now running pipeline container (this deviates from Atlassian Bitbucket Pipelines and allows keeping the container).

This might be a completely different concept than what this project is doing?

Not sure I understand. From your feedback I read that you'd like to have the environment configured by ~/.bashrc to be loaded. I'll do some tests with that.

You can also inspect what pipelines does by executing with -v (verbose), it shows the docker commands. Additionally with --keep you can keep the container running and then figure out if with docker exec ... $ID/$NAME ... you can already come close to what you're looking for. This would also help me to better understand I guess.

[ktomk]

@Timple Did some tests for ~/.bashrc and I think it improves on your use-case:

It now seems valid to me that, when using /bin/bash as script runner, to execute (source as in source - not dot - bashism) it if it exists at the very beginning of the step scripts.

Rationale is the history of the Atlassian Bitbucket Pipelines Plugin. Before February 2017, scripts were run in an interactive shell. The Atlassian Bitbucket Pipelines Plugin continued "to execute the .bashrc file as if run in an interactive non-login shell but it" then behaved "as a non-interactive shell" (ref).

shell: [interactive,     non-login] | ~>                                          #  February 2017 [ref]
    ~> [non-interactive, non-login]

    +  if [ -f ~/.bashrc ]; then . ~/.bashrc; fi                                  #  from: ref: GNU Bash: [Invoked as an interactive non-login shell]
    +' if [ -f ~/.bashrc ]; then source ~/.bashrc; fi                             #  source (bashism) asserts /bin/bash (over /bin/sh)
    +" test "$0" = "/bin/bash" && if [ -f ~/.bashrc ]; then source ~/.bashrc; fi  #  test for bash runner (/bin/bash)

ref: GNU Bash: Invoked as an interactive non-login shell

---

Please continue to find a temporary topic branch issue-17-bash in action. Build phar files (choose any) will be kept for two days so they can be used for early user testing.

I continue to target release version 0.0.66 for the new bash runner feature.

[ktomk]

@Timple if .bashrc is executed, would it then match all of your expectations? Because then I'd say from my end this ready and I'll prepare the next release.

[Timple]

Awesome, this works! Thank you.

One item left (unrelated so I'll open up a new issue!

[ktomk]

Released in 0.0.66.

[Timple]

Rationale is the history of the Atlassian Bitbucket Pipelines Plugin. Before February 2017, scripts were run in an interactive shell. The Atlassian Bitbucket Pipelines Plugin continued "to execute the .bashrc file as if run in an interactive non-login shell but it" then behaved "as a non-interactive shell" (ref).

I only now read this comment (was on phone yesterday). You did an awesome job investiging!