support newlines in env files

[vikstrous]

Environment variable values can have new lines in them and docker-compose.yml already supports that because yml does:

test:
    image: alpine
    environment:
        a: "b\nc"
    command: sh -c "echo \"$$a\""

However, the env file syntax doesn't support this. It looks like it doesn't do anything more than split on newlines and the first = sign.

compose/compose/config/environment.py

Line 16 in 86b5ed1

def split_env(env):

One way would be to allow quoted strings in the env file. This is not backwards compatible because anyone who already has a quoted string in their env file would see unexpected behaviour. Hopefully someone else has a better idea?

[sandrodz]

Why is this closed?

[notok]

@sandrodz shin- just closed duplicate issue of this. This issue is still open.

I also think it's convenient if env_file supports newline.
I found similar issue on docker's --env-file option.

moby/moby#12997

[grimabe]

+1

[kachkaev]

This feature will be really useful in cases when env variables may be quite long. E.g. in an email forwarder.

I was trying different variations of the following in my .env, but with no luck so far.

SMF_CONFIG="\
a@example.com:real.mailbox@gmail.com;\
b@example.com:real.mailbox@gmail.com;\
c@example.com:real.mailbox@gmail.com;\
d@example.com:real.mailbox@gmail.com;\
e@example.com:real.mailbox@gmail.com"

docker-compose config only picks characters from the first line. An ideal scenario would be an option to do something like this:

DOMAIN=example.com
REAL_MAILBOX=real.mailbox@gmail.com
SMF_CONFIG="\
a@${DOMAIN}:${REAL_MAILBOX};\
b@${DOMAIN}:${REAL_MAILBOX};\
c@${DOMAIN}:${REAL_MAILBOX};\
d@${DOMAIN}:${REAL_MAILBOX};\
e@${DOMAIN}:${REAL_MAILBOX}"

[einarf]

EDIT: This works with normal environment variables, so it's weird that it doesn't work with .env files as well.

Example:

export TEST="This is line 1
This is line 2
This is line 3"

Compose file

version: '2'
services:
  test:
    image: debian:jessie
    command: echo "$TEST"
    environment:
      - TEST=${TEST}

docker-compose up will print out

test_1  | This is line 1
test_1  | This is line 2
test_1  | This is line 3
test_1 exited with code 0

[vikstrous]

Yaeh... If you are wrapping compose to do this, you could also just generate the yaml with another tool.

[tmornini]

Would love to see multi-line .env file support!

[aanand]

We can only support multi-line values in env files if they're also supported in Engine (i.e. docker run --env-file), as env files must be interpreted identically by both tools.

Currently, newlines don't seem to be supported in Engine. The latest discussion I can find seems to be here: moby/moby#12997

[vikstrous]

It should be possible to hack around it, right?

see moby/moby#12997 (comment)

[likewei92]

The hack for those who are lazy: moby/moby#12997 (comment)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it had not recent activity during the stale period.

[lonix1]

In a large config base, this is a real pain point.

Does someone have a (almost) 2020 workaround for this?

(Also, there is this proposal for a fix.)

[tmornini]

I, too, would love to see this resolved.

[johannespfeiffer]

Small workaround, if my.env looks like this:

# my.env

PUBLIC_KEY="-----BEGIN PUBLIC KEY-----
MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAI4g6LvM
mJ2qlXmpoKWEuQlM8jUO1MTA/ztPA8iu8CAwEAAQ==
-----END PUBLIC KEY-----"

VAR1=test
VAR2=test
VAR3=test
# ...

Then just source the my.env in the ENTRYPOINT of the Dockerfile:

ADD my.env

ENTRYPOINT ["sh", "-c", "set -a; . my.env; set +a; /your/entrypoint.sh"]

If you want to be able to overwrite the envs via environment in docker-compose.yml you can use:

ENTRYPOINT ["sh", "-c", "env > custom.env; set -a; . my.env; . custom.env; set +a; /your/entrypoint.sh"]

[sshishov]

Please reopen and fix...

[henrhoi]

According to dotenv's documentation (link), which is used to parse the envfiles within Docker Compose, one should use double quotations with new-lines (\n) to achieve new lines after parsing.

Example:

PRIVATE_KEY="-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSU72AwEHBG0wawIBAQQgf+zcD+dC95xJH2YIGtZxg6cBEtZYzeMHZIdAtGarkfahRANCAAReRj+7DNK3rfudJGSo2k9mUBhmz2zqE3d4sfpkIGUizc1s8UXYmnhgezq6fceV9o6HG1I6BdVWi5cfbwTB17Qf\n-----END PRIVATE KEY-----"

[wichopy]

According to dotenv's documentation (link), which is used to parse the envfiles within Docker Compose, one should use double quotations with new-lines (\n) to achieve new lines after parsing.

Example:

PRIVATE_KEY="-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSU72AwEHBG0wawIBAQQgf+zcD+dC95xJH2YIGtZxg6cBEtZYzeMHZIdAtGarkfahRANCAAReRj+7DNK3rfudJGSo2k9mUBhmz2zqE3d4sfpkIGUizc1s8UXYmnhgezq6fceV9o6HG1I6BdVWi5cfbwTB17Qf\n-----END PRIVATE KEY-----"

When i try this, i get a double backslash

-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSU72AwEHBG0wawIBAQQgf+zcD+dC95xJH2YIGtZxg6cBEtZYzeMHZIdAtGarkfahRANCAAReRj+7DNK3rfudJGSo2k9mUBhmz2zqE3d4sfpkIGUizc1s8UXYmnhgezq6fceV9o6HG1I6BdVWi5cfbwTB17Qf\n-----END PRIVATE KEY-----

[ap-wtioit]

@wichopy we use it like this:

..._PUBLIC="-----BEGIN PUBLIC KEY-----
The
Public
Content
Lines
Of
Our
Key
-----END PUBLIC KEY-----"