Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support for local webhooks #61

Closed
jrhbcn opened this issue May 21, 2022 · 19 comments · Fixed by #83
Closed

feat: Support for local webhooks #61

jrhbcn opened this issue May 21, 2022 · 19 comments · Fixed by #83
Labels
enhancement New feature or request

Comments

@jrhbcn
Copy link

jrhbcn commented May 21, 2022

Please consider support for local webhooks such as webhook tool. It would also be extra-useful to somehow configure argus to change an HTTP Query parameter so different services can call the same webhook with different parameters. Such as for instance: http://192.168.9.9/hooks/update-docker?container=argus.

@jrhbcn jrhbcn added the enhancement New feature or request label May 21, 2022
@samcro1967
Copy link

@JosephKav This could also be used for the Gitea/Drone use case. I was able to setup webhook and with Drone CLI kick off a build. Would mean users maintaining scripts for each webhook (rather than just calling the native platform webhooks directly from Argus), but does not look like a large effort.

If you implemented this, could probably cover a lot of uses cases where the platform has a CLI, API, etc.

@JosephKav
Copy link
Collaborator

JosephKav commented May 26, 2022

Did either of you try the existing webhook on adnanh/webhook? I just tested mine and it worked

config.yml

    webhook:
      test:
        type: github
        url: https://FQDN/redeploy
        secret: SECRET

hook.json

[
    {
        "id": "redeploy",
        "execute-command": "/opt/webhook/redeploy.sh",
        "command-working-directory": "/opt/webhook",
        "pass-arguments-to-command": [],
        "trigger-rule":
        {
            "and":
            [
                {
                    "match":
                    {
                        "type": "payload-hmac-sha256",
                        "secret": "SECRET",
                        "parameter":
                        {
                            "source": "header",
                            "name": "X-Hub-Signature-256"
                        }
                    }
                },
                {
                    "match":
                    {
                        "type": "value",
                        "value": "refs/heads/master",
                        "parameter":
                        {
                            "source": "payload",
                            "name": "ref"
                        }
                    }
                }
            ]
        }
    },
]
[384de8] incoming HTTP POST request from 127.0.0.1:42944
[384de8] redeploy got matched
[384de8] redeploy hook triggered successfully
[384de8] 200 | 0 B | 5.5275ms | FQDN | POST /redeploy
[384de8] executing /opt/webhook/redeploy.sh (/opt/webhook/redeploy.sh) with arguments ["/opt/webhook/redeploy.sh"] and environment [] using /opt/webhook as cwd

@samcro1967
Copy link

I did try it, but without any luck. I think you are missing a ] at the end of your example.

In your example SECRET is representative of an random api_key? I was trying to pass in container name as the secret and ultimately an argument for the script. Not quite understanding what the second match is for.

I can run this script locally ./webhook.sh argus and it will kick off a drone rebuild of the argus container.

echo '{"SECRET": "'$1'"}' | http --json POST http://localhost:9357/hooks/drone-build

@JosephKav
Copy link
Collaborator

I did try it, but without any luck. I think you are missing a ] at the end of your example.

Yup, fixed that now

In your example SECRET is representative of an random api_key? I was trying to pass in container name as the secret and ultimately an argument for the script. Not quite understanding what the second match is for.

SECRET is the SECRET for your webhook. You just need to give Argus the same secret as you have your adnanh/webhook hook set to require

This secret is used to create a hash signature which the receiver then checks matches with the secret they are looking for

So in my example,

        url: https://FQDN/redeploy
        secret: SECRET

sends the webhook to FQDN/deploy and hashes it with SECRET. adnanh/webhook is listening to FQDN/redeploy and picks up on the webhook and runs the script

@samcro1967
Copy link

Is there a way to pass an argument so that one script is required or will we need a separate script for each with the container name/app name hard coded in the script? Not the end of the world, but it is 1 script vs. 30+ at the moment.

@JosephKav
Copy link
Collaborator

JosephKav commented May 26, 2022

Is there a way to pass an argument so that one script is required or will we need a separate script for each with the container name/app name hard coded in the script? Not the end of the world, but it is 1 script vs. 30+ at the moment.

I believe we'd need support for custom headers in the webhook for this
e.g. my current redeploy from GitHub has

        "pass-arguments-to-command":
        [
            {
                "source": "payload",
                "name": "head_commit.message"
            },
            {
                "source": "payload",
                "name": "pusher.name"
            },
            {
                "source": "payload",
                "name": "head_commit.id"
            }
        ],

so when it receives a webhook from GitHub it does the following

[2e4feb] executing /opt/webhook/redeploy.sh (/opt/webhook/redeploy.sh) with arguments ["/opt/webhook/redeploy.sh" "fix(examples): `wiki.js` typo (#14)" "JosephKav" "4f251b0ca927a1e641f88bace397e4e57fc1142c"] and environment [] using /opt/webhook as cwd

This seems to run /opt/webhook/redeploy.sh $head_commit.message $pusher.name $head_commit.id

I could work on adding custom_headers to the existing webhook field to allow for this kind of support
e.g.

    webhook:
      name:
        url: https://FQDN/redeploy
        secret: SECRET
        custom_headers:
          - something:
              foo: bish
              another: bash
          - app: bosh

so in your case, you could have

custom_headers:
  - app: argus
"pass-arguments-to-command": [ { "source": "payload", "name": "name" } ],

to run the execute-command with argus as a param. You'd just change the app value in Argus when you want to run it with a different value

@samcro1967
Copy link

Would probably make sense longer term if the lift is not large. Maintaining multiple scripts is not the end of the world in my case. Each is only going to have 3 or 4 commands and most will all be the same. Could be complex I suppose depending on the use case people come up with.

I do not see a -test.webhook. Is there a way to test from the Argus CLI?

Using your config with a container name hardcoded in the webhook script, the webhook is sent, but it is not triggered because the trigger rules were not satisfied. My hooks.json is identical to yours except for paths and script name.

webhook service log

May 26 08:28:31 osuhickeys4 webhook[12554]: [webhook] 2022/05/26 08:28:31 Started POST /hooks/drone-build
May 26 08:28:31 osuhickeys4 webhook[12554]: [webhook] 2022/05/26 08:28:31 [685246] incoming HTTP request from 172.18.0.14:41090
May 26 08:28:31 osuhickeys4 webhook[12554]: [webhook] 2022/05/26 08:28:31 [685246] drone-build got matched
May 26 08:28:31 osuhickeys4 webhook[12554]: [webhook] 2022/05/26 08:28:31 [685246] drone-build got matched, but didn't get triggered because the trigger rules were not satisfied
May 26 08:28:31 osuhickeys4 webhook[12554]: [webhook] 2022/05/26 08:28:31 Completed 200 OK in 2.614037ms

config.xml global definition

webhook:
  WEBHOOK_DRONE_BUILD:
    type: github
    url: http://192.168.1.104:9357/hooks/drone-build
    secret: SECRET

config.xml service definition

    notify:
      GOTIFY_NOTIFICATION: {}
    webhook:
      WEBHOOK_DRONE_BUILD: {}
    deployed_version:

@JosephKav
Copy link
Collaborator

Would probably make sense longer term if the lift is not large. Maintaining multiple scripts is not the end of the world in my case. Each is only going to have 3 or 4 commands and most will all be the same. Could be complex I suppose depending on the use case people come up with.

It doesn't seem like it'd be very large. Only possible issue I can think of is getting Go to handle unmarshaling the config.yml. Ideally I'd want it to be able to handle multiple levels of keys, so something like

custom_headers:
  foo:
    a: 1
    b: 2
    another:
      test: 123
  app: name

Not sure if there'd be a use-case, but I'd like it. I think it should be pretty easy, but not the end of the world if it'll only allow key: value as that'll be what's wanted normally

I do not see a -test.webhook. Is there a way to test from the Argus CLI?

The only current way of testing would be to set the deployed_version lower and approve the webhooks. I can add a -test.webhook flag when I add support for custom_headers

Using your config with a container name hardcoded in the webhook script, the webhook is sent, but it is not triggered because the trigger rules were not satisfied. My hooks.json is identical to yours except for paths and script name.

Do you definitely have the same trigger-rule and the secret under that matches the secret you're giving Argus?

@samcro1967
Copy link

The secret is both argus config.xml and hooks.json is SECRET for ease of testing. ;-)

I was thinking since the webhook type in argus config.xml is github, that the second match below was being handled automatically by argus.

			{
				"match":
				{
					"type": "value",
					"value": "refs/heads/master",
					"parameter":
					{
						"source": "payload",
						"name": "ref"
					}
				}
			}

@JosephKav
Copy link
Collaborator

JosephKav commented May 26, 2022

I was thinking since the webhook type in argus config.xml is github, that the second match below was being handled automatically by argus.

so your trigger-rule matches this? It worked fine for me with the secrets as SECRET
(make sure you restarted the webhook service after changing the config!)

        "trigger-rule":
        {
            "and":
            [
                {
                    "match":
                    {
                        "type": "payload-hmac-sha256",
                        "secret": "SECRET",
                        "parameter":
                        {
                            "source": "header",
                            "name": "X-Hub-Signature-256"
                        }
                    }
                },
                {
                    "match":
                    {
                        "type": "value",
                        "value": "refs/heads/master",
                        "parameter":
                        {
                            "source": "payload",
                            "name": "ref"
                        }
                    }
                }
            ]
        }
    },

@samcro1967
Copy link

Yeah, I restarted the webhook service.

Her si my complete hooks.json

[
  {
    "id": "drone-build",
	"execute-command": "/home/user/Documents/Docker/webhooks/drone-build.sh",
    "command-working-directory": "/home/user/Documents/Docker/webhooks",
    "pass-arguments-to-command": [],
	"trigger-rule":
	{
		"and":
		[
			{
				"match":
				{
					"type": "payload-hmac-sha256",
					"secret": "SECRET",
					"parameter":
					{
						"source": "header",
						"name": "X-Hub-Signature-256"
					}
				}
			},
			{
				"match":
				{
					"type": "value",
					"value": "refs/heads/master",
					"parameter":
					{
						"source": "payload",
						"name": "ref"
					}
				}
			}
		]
	}
  },
]

@JosephKav
Copy link
Collaborator

That's very odd as that exact config works with

    webhook:
      test:
        type: github
        url: https://FQDN/hooks/drone-build
        secret: SECRET

for me

[5b86d6] incoming HTTP POST request from 127.0.0.1:48350
[5b86d6] drone-build got matched
[5b86d6] drone-build hook triggered successfully
[5b86d6] 200 | 0 B | 5.696056ms | FQDN | POST /hooks/drone-build
[5b86d6] error in exec: "/home/user/Documents/Docker/webhooks/drone-build.sh": stat /home/user/Documents/Docker/webhooks/drone-build.sh: no such file or directory

@samcro1967
Copy link

Hmmm....I just reverted my hooks.json back to my config and tested from Ubuntu CLI and I am able to trigger the webhook. So it seems like the service is working as expected.

Here is that config if it is of any value.

[
  {
    "id": "drone-build",
    "command-working-directory": "/home/osuhickeys/Documents/Docker/webhooks",
    "pass-arguments-to-command":
    [
      {
        "source": "payload",
        "name": "container"
      }
     ],
        "execute-command": "/home/osuhickeys/Documents/Docker/webhooks/drone-build.sh"
  }
]

drone-buils.sh looks like this

#!/bin/sh
export DRONE_SERVER=http://osu.drone:3070
export DRONE_TOKEN=<redctaed>
drone build create osuhickeys/$1

webhooks.sh looks like this:
echo '{"SECRET": "'$1'"}' | http --json POST http://localhost:9357/hooks/drone-build

Can then execute with ./webhooks.sh argus and a build kicks off in Drone for Argus.

@jrhbcn
Copy link
Author

jrhbcn commented May 26, 2022

In my case I am unable to make it work :( I have this in the argus config.yaml:

  Argus:
    type: github
    url: release-argus/argus
    web_url: https://github.com/release-argus/Argus/blob/master/CHANGELOG.md
    icon: https://github.com/release-argus/Argus/raw/master/web/ui/static/favicon.svg
    webhook:
      test:
        url: "http://192.168.9.9/hooks/update-test"
        secret: ARGUS
    deployed_version:
      url: http://192.168.9.9/hooks/version-deployed?name=argus
      json: version

And this in hooks.yaml for webhook config (I prefer yaml to json):

- id: version-deployed
  execute-command: "/root/webhook/version_deployed.sh"
  pass-arguments-to-command:
    - source: url
      name: name
  command-working-directory: "/root/docker"
  include-command-output-in-response: true
- id: update-test
  execute-command: "/root/webhook/update_test.sh"
  command-working-directory: "/root/docker"
  trigger-rule:
    and:
    - match:
        type: payload-hmac-sha256
        secret: ARGUS
        parameter:
          source: header
          name: X-Hub-Signature-256
    - match:
        type: value
        value: refs/heads/master
        parameter:
          source: payload
          name: ref

When I try to "resend webhooks" in argus, no update-test webhook is called, however, it seems to trigger all of the webhook-deployed ones I have.

It might be something about my config but I cannot see what it is now. The deployed_version calls through webhook work fine though.

@JosephKav
Copy link
Collaborator

And this in hooks.yaml for webhook config (I prefer yaml to json):

:O didn't realise it supported yaml! Converted mine straight away XD, <3 yaml!

Your config worked when I tried it though :/.
All I'm seeing that's different between working and not is that the ones that work with the payload-hmac-sha256 secret (only mine?) are using HTTPS. Could that be a requirement for webhooks? Using a secret, but doing it over HTTP does seem quite pointless as anyone listening could just send exactly what you did and trigger the hook again

@samcro1967
Copy link

So for fun (I mean who doesn't love certs), I changed webhook to use https. I get a different error in the webhook log when I execute the webhook from Argus, but I am guessing it is related to how I generated the self signed cert. Thought I would share in case it sheds any new light on the topic.

webhook log

[webhook] 2022/05/26 15:30:37 http: TLS handshake error from 172.18.0.14:41156: tls: oversized record received with length 21536

cert generation command

openssl req -new -x509 -sha256 -newkey rsa:2048 -nodes -keyout key.pem -days 9999 -out cert.pem`

@JosephKav
Copy link
Collaborator

So local webhooks do currently work, you just requested support for params, which in the PR, I've enabled with support for custom_headers. Is there anything else to this ticket, or can I close it once I merge and release this?

@jrhbcn
Copy link
Author

jrhbcn commented Jun 4, 2022

Not from my part! Thank you @JosephKav

@samcro1967
Copy link

Agreed. Good to close and thank you very much @JosephKav.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants