Question: Why use a custom buildpack?

[jmcarp]

I'm wondering why this app needs a custom buildpack. Cloud Foundry supports multiple buildpacks natively or with another buildpack, so if users need to build the frontend with node and run the backend with go, either of those options should work. Also, in my experience, it's simpler to build static assets in CI before pushing to CF, in which case we should be able to use the standard go buildpack. Am I missing something here?

[nwmac]

We did look at using multiple buildpacks. It may be possible to revisit this.

The backend code (go) requires a pre-build script to be run for the plugin mechanism we have. Each plugin has its own dependencies, so we run glide for each plugin that is enabled. The go code is then built. Currently the go build process is done via an npm script. The backend also uses an npm script to build. The go buildpack can not build the backend code as is. Since we have both frontend and backend code in the repo, the backend code is not in the top-level of the git repo - I didn't see a way to get the go buildpack to build from a different folder.

We also wanted to support different stacks - the multiple buildpack does not support system buildpacks, so we would have to reference the go and node buildpacks via URLs. Last time I checked, the go buildpack only support the cflinux stack.

I don't like pre-building the angular code in CI - its source just like the go code and should be built as part of the push.

I would love to get rid of the custom buildpack - if someone can demonstrate how to use multiple buildpacks in the place of the custom buildpack that would be great.

[aeijdenberg]

As a note, I'm find it incredibly difficult to effectively make backend changes due to the non-standard Go code layout.

Generally with Go you can refactor all day long, and once Visual Studio Code stops showing compiler messages, you tend to have a pretty good success rate with doing a cf push.

As it stands it's pretty much make some changes, try a cf push, wait 5 minutes to find the syntax error, try again.

Perhaps I'm missing some docs. I'll see if I can knock something together to improve the situation (docs or other change).

[aeijdenberg]

As follow-up, I did an experiment to see how difficult it might be to make the backend Go app be compilable in a more standard Go way, as this may make it easier to use standard buildpacks, and much easier to edit code (taking advantage of modern IDE features such as compilers warnings etc that require the IDE understand the layout of the project).

Along the way I did the following:

1. Moved Go source to the locations where they would normally be to match the way they are imported.
2. Removed the dynamic plugins, and instead assume that all need to be built (unclear if this assumption is safe - might be easier to make enabling of a feature a runtime rather than compile time decision).
3. Adjust the import paths to match the new location of the repository.
4. Remove use of Glide, and replace with a single Gopkg.toml that dep uses.

And that's pretty much it. I was able to write a script (that I've put in a Gist here: https://gist.github.com/aeijdenberg/2d8f6f473e567b834bede97bc557771f) which you can run to produce the same output, as it may be easier to follow than the resulting commit:

Experimental PR that shows the resulting commit is here: #2662

Note that other than compiling the app-core binary with standard Go, I did not try to adjust the existing JS based build scripts to use. Wanted to check-point here before going any further down this path.

I believe that having a more standard Go layout will make contributing changes to this project much easier, so I hope this helps demonstrate a possible path forward to that.

[nwmac]

@aeijdenberg Thanks for this - sorry for the delay in replying. We are looking at this as a priority. We had been considering go modules that are in go 1.11 coming this month.

[richard-cox]

@nwmac has answered the original question.

Additionally, there's been some general improvements to the backend and it should now not show compile errors in VSCode.

On a slightly related note, cf push can be speeded up by building the frontend locally via npm run prebuild-ui.