Update Buf actions

[hariso]

Description

Closes #1710.

• Updates the Buf jobs (uses the new action)
• Splits existing workflow into two separate ones:
  • one that validates changes in a pull request
  • one that pushes the changes into BSR (once they've been merged into our main branch)
• Uses a token (that we already have set up in a secret) to authenticate requests to Buf.

The validation currently fails, because we haven't updated buf.lock when we made changes to conduit-commons (part of the schema related work in 0.11). A workflow that regularly runs buf update is implemented in #1772.

Quick checks

• I have followed the Code Guidelines.
• There is no other pull request for the same update/change.
• I have written unit tests.
• I have made sure that the PR is of reasonable size and can be easily reviewed.

[hariso]

Looks like Buf released a new action that unifies all of these actions. I'll convert this PR to a draft.

[lovromazgon]

Also keep in mind that we need the token specifically when generating the proto code in validate-generated-files.yml because we're using remote code generation.

[hariso]

Also keep in mind that we need the token specifically when generating the proto code in validate-generated-files.yml because we're using remote code generation.

I was looking at that too, but interestingly, buf isn't complaining about the token there. I plan to take a closer at that (I mean, it doesn't hurt just giving it the token, but still).

[github-actions]

The latest Buf updates on your PR. Results from workflow buf / validate (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	✅ passed	✅ passed	❌ failed (2)	Aug 12, 2024, 8:22 PM

[github-actions]

The latest Buf updates on your PR. Results from workflow buf-validate / validate (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	✅ passed	✅ passed	⏩ skipped	Aug 14, 2024, 1:43 PM

[lovromazgon]

LGTM 💪 Please do the same in the processor SDK, connector protocol and conduit commons, I believe we are using buf there as well.