Using vc dev .env has higher priority than .env.local

[gabrielalmeida]

Bug report

Describe the bug

Doc states the following:

In general only one .env.local file is needed. However, sometimes you might want to add some defaults for the development (next dev) or production (next start) environment.

Next.js allows you to set defaults in .env (all environments), .env.development (development environment), and .env.production (production environment).

.env.local always overrides the defaults set.

Creating .env and .env.local setting the same TEST env variable and console.log shows that .env is actually overriding .env.local

To Reproduce

https://github.com/gabrielalmeida/nextjs-env-error-issue-17338

Expected behavior

.env.local values should be printed instead of .env "default" values

When running now dev, it actually does print the following wrong loading order for .env* files:

❯ now dev
Now CLI 20.0.0 dev (beta) — https://vercel.com/feedback
info  - Loaded env from /Users/gbr/Dev/env-override-error-repro/.env.local
info  - Loaded env from /Users/gbr/Dev/env-override-error-repro/.env
ready - started server on http://localhost:3000
event - compiled successfully
event - build page: /
wait  - compiling...
event - compiled successfully
.env .env. >>>>>>>>>>>>>> outputs .env instead of .env.local when console.log(process.env.NEXT_PUBLIC_TEST and process.env.TEST)

System information

• OS: macOS
• Version of Next.js: 9.5.3
• Version of Node.js: v12.16.3

Additional context

The same unexpected behavior is present when using now dev or publishing to Vercel.

[arthurjdam]

Judging by its implementation in load-env-config.ts I think it might be the docs that are wrong here.

Changing the order will break existing builds relying on these stacked config files, and would make for an awkward upgrade process as a result.

[lfades]

@gabrielalmeida Can you try reproducing that behavior in this example: https://github.com/vercel/next.js/tree/canary/examples/environment-variables 🙏

[gabrielalmeida]

Even though the implementation at load-env-config.ts seems misleading at first because of the reverse order:

const dotenvFiles = [
    `.env.${mode}.local`,
    // Don't include `.env.local` for `test` environment
    // since normally you expect tests to produce the same
    // results for everyone
    mode !== 'test' && `.env.local`,
    `.env.${mode}`,
    '.env',
  ]

It's actually matching the expected behavior of the docs, because it's only applying a new value if it was not present earlier on a higher priority .env:

for (const key of Object.keys(result.parsed || {})) {
  if (
    typeof parsed[key] === 'undefined' &&  <---- Will only override if not defined by anyone earlier on the loop
    typeof origEnv[key] === 'undefined'  
  ) {
    parsed[key] = result.parsed?.[key]!
  }
}

After acknowledging this, I've realized that the issue was elsewhere. Digging further, I've noticed that I've been starting my dev environment mostly using Vercel CLI directly, by issuing a vc dev (actually mostly now dev due to muscle memory).

The Vercel CLI will automatically look for a .env and bypass the next Next's (no pun intended) .envs, causing this mess:

$ vc dev --debug
> [debug] [2020-09-25T01:17:33.818Z] GET /
> [debug] [2020-09-25T01:17:33.818Z] GET /_next/webpack-hmr?page=/
> [debug] [2020-09-25T01:17:33.820Z] Locating files /Users/gbr/Dev/Whats/Meta/env-override-error-repro: 143639.895ms
> [debug] [2020-09-25T01:17:33.823Z] Using local env: /Users/gbr/Dev/env-override-error-repro/.env <<<<<<<<
> [debug] [2020-09-25T01:17:33.823Z] Proxying to frontend dev server: http://localhost:61110
.env .env <----- console.log unexpected output

So the pesky gotcha is, use next dev instead of vc dev to avoid the .env conventions collision between next and vercel.

I'd close this if there was a way to avoid this error when publishing via Vercel CLI, because as soon you type vc to create a deployment, the same unexpected load order of .envs will happen and it seems there is no way to bypass this while using the CLI.

Should I close this one and reference it on a new issue at vercel?

[lfades]

@gabrielalmeida We can have it open here, this is something we would most likely fix with: #17152 - And tracking this issue is important. Thank you!.

[gabrielalmeida]

Cool, that PR seems to be the way to go! Just a heads up that this is not exclusive to vc dev, deploying any Next app with vc will lead to the same load order error.

I haven't tested though if checking a .env to VCS (which is by new Next's .env conventions now an advocated practice) while using Github Integration with Vercel to deploy will also end up using .env for deployments and lead to the same error.

[KevinDanikowski]

I'd like to add to this, any dotenv set ups that require .env being overridden no longer work. The last working version I had was 9.3.3 and upgrading to 9.5.3 broke this. The package dotenv-flow, for example, is a package that just this and no longer works.

my temp solution for those who need it
Until this is fixed is renaming my .env file to .env.temp and writing my own dotenv-flow file:

const fs = require('fs')
const dotenv = require('dotenv')
const dotenvExpand = require('dotenv-expand')

const APP_ENV = process.env.APP_ENV
const appDirectory = fs.realpathSync(process.cwd())

// The order of loading files is:
// 1. Local overrides (.env.local)
// 2. Env specific (ie. .env.production)
// 3. Default (.env)
const dotenvFiles = [
  APP_ENV !== 'test' && `${appDirectory}/.env.local`,
  `${appDirectory}/.env.${APP_ENV}`,
  `${appDirectory}/.env.temp`,
].filter(Boolean)

dotenvFiles.forEach(dotenvFile => {
  if (fs.existsSync(dotenvFile)) {
    dotenvExpand(
      dotenv.config({
        path: dotenvFile,
      })
    )
  }
})

console.log('host', process.env.HOST_NAME)

[efriandika]

Previously I use v10.0.2. and it works as expected:

.env.local always overrides the defaults set.

But after I upgrade to v10.0.8, it does not work anymore.
It seems like .env is actually overriding .env.local

[efriandika]

I just tried v10.0.7, it works too,
So, it does not work "again" starting from v10.0.8

[rtrembecky]

hi, what's the status here? I'm experiencing this problem on 10.2.0. tried to upgrade to 10.2.2 and still a problem.
my issue: .env overriding .env.local.
my current workaround: use .env.development instead of .env. then .env.local overrides .env.development