unable to use arbitrary files from non-package sources

[leeederek]

Background & motivation

Today, Kurtosis only supports the mounting of files to containers when those files are part of a Kurtosis package. This is painful for when I need to mount files like Grafana dashboard templates to my package for rendering the correct dashboard.

My workarounds today is to store my files in a child/sub package and maintain it. Then, in my main package, import in the child/sub package (via import_module())

Desired behaviour

A Kurtosis package author should be able to specify any local file path or URL to fetch data from & turn into a files artifact to use for the services inside their enclaves, without needing to have the desired data to be part of a Kurtosis package.

How important is this to you?

Painful; the lack of this feature makes using Kurtosis frictionful.

What area of the product does this pertain to?

CLI: the Command Line Interface

[barnabasbusa]

It would also be great if the URL specified would recognize if the url points to a file or directory structure, in case its a directory structure, it should create a files artifact off of that directory.

This would enable configuring custom ethereum devnets an ease.

[barnabasbusa]

Would like to add another comment here:

I have just made a pr to postgres package, but I had no write access to the directory.
In order to be able to locally test if my PR changes are working fine, I have to adjust where I import the postgres package from (which is not a problem)
postgres = import_module("github.com/barnabasbusa/postgres-package/main.star@patch-1")

However, I also have to change the kurtosis.yml for some reason. If I change that in the branch, then the PR will update also, where I would change the default kurtosis.yml (which is obviously not something I want to do). So the only way to make this work, is by changing the kurtosis.yaml to my own repo name, then once the local test is done, revert it, so the PR wouldn't request changes for this file.

There was an error interpreting Starlark code 
Evaluation error: An error occurred while loading the module 'github.com/barnabasbusa/postgres-package/main.star@patch-1'
        Caused by: Error occurred while validating /kurtosis-data/startosis-packages/barnabasbusa/postgres-package/kurtosis.yml
        Caused by: The package name in kurtosis.yml must match the location it is in. Package name is 'github.com/kurtosis-tech/postgres-package' and kurtosis.yml is found here: 'github.com/barnabasbusa/postgres-package'
        at [github.com/kurtosis-tech/ethereum-package/src/blobscan/blobscan_launcher.star:2:25]: <toplevel>
        at [github.com/kurtosis-tech/ethereum-package/main.star:24:25]: <toplevel>

Error encountered running Starlark code.

It would be very good if kurtosis.yaml file would not be required at all, and you could just import any module from anywhere (yolo mode).

[leeederek]

Hugobyte had this pain point too today. They wanted to upload a set of files from a directory.

The workaround can be:

1. Make the remote directory (of files that a user wants to import into their enclave) into a Kurtosis package using a kurtosis.yml
2. Within plan.run_sh() instruction, git clone the repository and turn those files into files artifacts inside the enclave