feat: nginx config for request size and timeouts

[abuchanan-airbyte]

Set up some nginx config for request body size and timeouts.

Related issues:

• 413 when creating connection airbyte#45423
• [destination-postgres] timeout funnel is inverted causing obscure 504 errors at the ingress proxy because nothing in the pipeline logs the root cause to a log airbyte#45156
• Custom images all return 504 when trying to set up local connectors airbyte#43893
• https://github.com/airbytehq/airbyte-internal-issues/issues/9859

Unfortunately, I think this will only work on new installs or uninstall + reinstall. In my tests, I couldn't get this to update my existing ingress.

The request body size is fairly easy to test by creating a postgres source with lots of tables/columns:

after:

I'm not sure how to test the timeout setting yet.

[niveditabaliga-payroc]

Hi @abuchanan-airbyte. When will this be merged in?

[niveditabaliga-payroc]

@abuchanan-airbyte does this merge mean we can repull airbyte using abctl and have the fix?

[abuchanan-airbyte]

@niveditabaliga-payroc we'll cut a release soon that will include this change.

[niveditabaliga-payroc]

hi @abuchanan-airbyte . We downloaded the new version of abctl (v0.18.0 by running the curl command curl -LsfS https://get.airbyte.com | bash -), ran abctl local install and when we opened up Airbyte, we still faced the HTTP 413 error when setting up a connection to a source DB with over 100 tables. Are you able to guide us on why that would still happen?

[abuchanan-airbyte]

@niveditabaliga-payroc Perhaps you could join the Slack channel, so we can debug the issue https://docs.airbyte.com/community/getting-support

[japerry911]

@niveditabaliga-payroc

adding annotation to ingress-abctl namespace worked for me -

kubectl edit ingress ingress-abctl -n airbyte-abctl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig

then add under metadata (50m = 50 mb, if you need more add higher number, etc..)...

metadata:
  annotations:
     nginx.ingress.kubernetes.io/proxy-body-size: "50m" 

Should work momentarily.. Just a temporarily fix though, I don't know if this is best method, but works for me.

[niveditabaliga-payroc]

thank you for the input @japerry911 . I will remember this in case we run into this issue again. For the time being, the fix released the day before worked after we ran the following commands

1. curl -LsfS https://get.airbyte.com | bash -
2. abctl local uninstall
3. abctl local install