[FEATURE]: Support referencing files in parent directories

[eysi09]

Feature Request

UPDATE: See this comment for a solid workaround.

Background / Motivation

One of Garden's main features is to allow users to distribute their Garden configuration across the repo and co-locate it with the source code. This is particularly useful for large monorepos and ensures users don't need to stuff everything into a massive config file (although they can if they want to).

A common pattern which is repeated throughout our examples and docs is to structure your Garden project like this:

.
├── api
│   ├── Dockerfile
│   ├── src
│   └── garden.yml
├── db
│   └── garden.yml
├── web
│   ├── Dockerfile
│   ├── src
│   └── garden.yml
└── project.garden.yml

However, it is very common for teams to store manifests, Dockerfiles, shared files and more at the root and currently you can't reference files in parent directories in Garden config.

To make a concrete example, let's imagine the project above has a manifests directory at the root that contains the Kubernetes manifests for the api and web components.

Typically you would include the manifests with in respective Deploy action via the files field like so:

kind: Deploy
type: kubernetes
name: api
spec: 
  files: [deployment.yaml, service.yaml, ingress.yaml]

But for this example we run into trouble because the manifests are in a parent directory and the following does not work:

spec:
  files: [../manifests/deployment.yaml, ../manifests/service.yaml, ../manifests/ingress.yaml]

The current workaround is to hoist the relevant Garden config to the root of the project and if needed use the include and source.path fields to set the build context.

That's why for a lot of "real world" projects we end up seeing a project structure that looks more like this:

.
├── api
│   ├── Dockerfile
│   └── src
├── web
│   ├── Dockerfile
│   └── src
├── api.garden.yml
├── db.garden.yml
├── web.garden.yml
└── project.garden.yml

A similar problem arises if you e.g. have Garden templates that you'd like to store in a common templates directory.

What should the user be able to do?

To structure their monorepos in a manner that works best for them and their project, without having Garden impose constraints on either where the Garden config lives or how the rest of the project is structured.

This will make it easier for users to adopt Garden for existing projects without changing the structure of the project—which is something users should be able to do—and without having to dump all the config at the root which doesn't make it very distributed.

One hard constraint that we can't lift is that the files need to belong to the same git repo and perhaps the same Garden project.

Current workaround

To reference things like manifest files or local Helm charts you need hoist your Garden config to the root. If the same config has a Build action you need to set the source.path field to the correct directory.

For library functions or config files that belong with the build context, you can work around this by using the build.copy directive or by using a base image that includes these files and your "main" image includes as a base layer.

EDIT: See this comment for a workaround

Other notes

Note that this has also been requested here:

• Allow specifying a different docker build context #853
• Allow relative paths outside of the garden.yaml root #1199

Both are closed as stale and I figured I'd open a new one since these are pretty dated and opened during the 0.12 days.

[eysi09]

Actually there's a pretty solid workaround I wasn't aware of when I first created this issue.

Garden supports parent directories when setting the action source path via the source.path field. This essentially allows you to set the source path for that action without actually moving the location of the config file.

To make a contrived example, let's say we have the following project structure:

.
├── api
│   ├── Dockerfile
│   ├── src
│   └── config
│       └── garden.yml
├── manifests
│   └── deployment.yml
└── project.garden.yml

The API component has a Garden config file nested under ./api/config. Let's assume that file defines a Build action that needs to reference the src in the parent dir and a Deploy action that needs to reference the manifests at the project root.

With the source.path field we can do that like so:

# In api/config/garden.yml
kind: Build
type: container
name: api
source:
  path: ../ # <--- Set the Build action source to the parent dir where the build context lives

---
kind: Deploy
type: kubernetes
name: api
source:
  path: ../../manifests # <--- Set the Deploy action source to the manifest dir
spec:
  files:
    - deployment.yaml # <--- Reference the manifest files relative to the source path set above

It may be slightly awkward to set the source path field as opposed to referencing the files relative to the actual config file location (i.e. [files: ../../manifests/deploy.yaml]) but it turns out that supporting that will be a massive undertaking so I think this workaround is definitely good enough for now.

So I'll go ahead and close this.