Error with Grate 1.7.0 - Read-only file system

[mbp]

Since the version 1.7.0 of Grate, we are getting errors when running it in our Docker environment.

Log file:

Looking in /tmp/tmphW0TFr.tmp for scripts to run.
An error occurred: Read-only file system : '/app/output'

We use AWS Fargate to run grate with readonlyrootFileSystem enabled for security reasons. It seems new grate version is writing to the root file system. Can this change be reverted?

This issue can also be reproduced if you run grate in a Docker with --read-only set, e.g.

docker run -t my-grate-migration-container --read-only

[erikbra]

I understand why this is happening. It's the internal grate migration that extracts embedded SQL from the DLL and writes them to a temporary folder before running an extra, internal grate migration on its own internal tables, to make sure they are up-to-date, before proceeding with the actual migration. This is due to #482 . I did think of it, but I thought it was an exotic use case to not have a writable temporary folder. Of course I was wrong 😛

I'll try to look into virtualising the fil system access, so that we are not dependent on a writable file system for this. In the mean time, I recommend rolling back to grate 1.6.2.

A related question, how do you handle grate's changeDropFolder/outputPath? Do you get a lot of warnings that we're unable to copy the scripts to the changeDropFolder?

[erikbra]

A follow-up question, @mbp, could you try to add a tmpfs volume on /tmp, and see if that solves you issue? https://docs.docker.com/storage/tmpfs/ or is that not possible in your situation?

[mbp]

@erikbra right now we have workaround with pinning the version to 1.6.2. So there is no urgent issue for us 👍

We do have writable tmp folder. It's just the root file system that is not writable (I suppose it's to prevent someone from modfying any application code)

We actually invoke Docker like so:

docker run -t my-grate-migration-container --read-only --tmpfs "/tmp"

And grate is invoked like so:

grate --dbt postgresql -ct 300 --silent -f /tmp/UpgradeFiles -o /tmp/grate -cs "XXX"

As you can see, we set output path to tmp folder. I am not sure what is meant by "changeDropFolder", but we don't get any warnings when running grate.

[misterjunio]

@erikbra was #510 supposed to have fixed this? Running Grate version 1.7.4 I still see a similar issue to @mbp. My log running on ECS Fargate with a read-only root FS shows me An error occurred: Read-only file system : '/tmp/system-commandline-sentinel-files'..

[erikbra]

Double-checking, @misterjunio and @mbp , does grate 1.8.0 solve this, or is this still not working?

[misterjunio]

@erikbra can't check it on my end because of #569 😅 I'll give it a check once v1.8.1 is out

[erikbra]

Makes sense :) I'll try to get 1.8.1 out the window soon :)

Re-opening this for now, so that we have it on the radar until we're certain it's resolved (or not)

[mbp]

Hi @erikbra , I don't have any issue with read-only file system. We use grate 1.8.0.

[erikbra]

Do you also run in a container with a read-only filesystem, @mbp ?

[mbp]

Yes I do @erikbra.

[erikbra]

Wonderful, thanks for the follow-up, @mbp . I'll close this one again then, and assume the issue is actually solved in 1.8.0. All joy and happy-yappiness, then 😄