To build or test the builder, let's use a released version of the
builder. This allows to fix the builder in case something got broken
with a PR.

When testing, pulling the latest version makes the whole test kinda
pointless, as the last version currently published gets tested instead
of the just built one. Add a new argument so we can still reuse the same
builder action, but skip pulling to reuse the freshly built builder.