Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds new proxy tests and manual proxy tester #138071

Merged
merged 5 commits into from
Aug 16, 2022

Conversation

pmuellr
Copy link
Member

@pmuellr pmuellr commented Aug 4, 2022

Partially addresses #102935 (adds the test framework, but not all the tests since some fail)

Part of proxy meta issue #138319

Summary

All the code here is for testing, no production-level code is changed.

Adds new proxy tests that can test a variety of different proxy and target server configurations, however many of those tests are broken with our current proxy agents. As we fix those, we can update the test here to test those additional scenarios

Adds some documentation regarding proxy settings, for some conditions that we know to be true today using the current proxy support.

In addition to tests, the stand-alone forward proxy has been enhanced to use a better proxy server, proxy. This proxy may be useful in helping to diagnose proxy issues customers are running into. The existing proxy server http-proxy does not support HTTPS out of the box, and so any HTTPS testing with it is going to be a little sketchy. Using this stand-alone forward proxy, I was able to post to Slack through http/https proxies with and without auth, with proxyRequestUnauthorized set to false. Which shows the existing proxy agents do work in some environments.

@pmuellr
Copy link
Member Author

pmuellr commented Aug 4, 2022

buildkite test this

partially addresses elastic#102935

The new proxy tests added can test a variety of different proxy
and target server configurations, however many of those tests
are broken with our current proxy agents.  Hopefully to be fixed
by replacing with hpagent instead.  In the meantime, we wanted to
get the basic test framework in as well.

In addition to tests, the stand-alone forward proxy has been
enhanced to use a better proxy server, `proxy`.  The existing
proxy server `http-proxy` does not support HTTPS out of the box,
and so any HTTPS testing with it is going to be a little sketchy.

Using the stand-alone forward proxy, I was able to post to Slack
through http/https proxies with and without auth, with
proxyRequestUnauthorized set to false.  Which shows the existing
proxy agents do work in _some_ environments.
@pmuellr pmuellr added the docs label Aug 8, 2022
@pmuellr pmuellr added Feature:Actions Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Feature:Actions/Framework Issues related to the Actions Framework backport:prev-major Backport to the previous major branch and all later branches still in development labels Aug 8, 2022
@pmuellr pmuellr marked this pull request as ready for review August 8, 2022 21:21
@pmuellr pmuellr requested a review from a team as a code owner August 8, 2022 21:21
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

Copy link
Contributor

@ymao1 ymao1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

+
Proxies may be used to proxy http or https requests through a proxy using the http or https protocol. Kibana only uses proxies in "CONNECT" mode (sometimes referred to as "tunnelling" TCP mode, compared to HTTP mode). That is, Kibana will always make requests through a proxy using the HTTP `CONNECT` method.
+
If your proxy is using the https protocol (vs the http protocol), the setting `xpack.actions.ssl.proxyVerificationMode: none` will likely be needed, unless your proxy's certificates are signed using a publically available certificate authority.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If your proxy is using the https protocol (vs the http protocol), the setting `xpack.actions.ssl.proxyVerificationMode: none` will likely be needed, unless your proxy's certificates are signed using a publically available certificate authority.
If your proxy is using the https protocol (vs the http protocol), the setting `xpack.actions.ssl.proxyVerificationMode: none` will likely be needed, unless your proxy's certificates are signed using a publicly available certificate authority.

@@ -143,6 +143,22 @@ Specifies preconfigured connector IDs and configs. Default: {}.

`xpack.actions.proxyUrl` {ess-icon}::
Specifies the proxy URL to use, if using a proxy for actions. By default, no proxy is used.
+
Proxies may be used to proxy http or https requests through a proxy using the http or https protocol. Kibana only uses proxies in "CONNECT" mode (sometimes referred to as "tunnelling" TCP mode, compared to HTTP mode). That is, Kibana will always make requests through a proxy using the HTTP `CONNECT` method.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Proxies may be used to proxy http or https requests through a proxy using the http or https protocol. Kibana only uses proxies in "CONNECT" mode (sometimes referred to as "tunnelling" TCP mode, compared to HTTP mode). That is, Kibana will always make requests through a proxy using the HTTP `CONNECT` method.
Proxies may be used to proxy http or https requests through a proxy using the http or https protocol. Kibana only uses proxies in "CONNECT" mode (sometimes referred to as "tunneling" TCP mode, compared to HTTP mode). That is, Kibana will always make requests through a proxy using the HTTP `CONNECT` method.

There is currently no support for using basic authentication with a proxy (authentication for the proxy itself, not the URL being requested through the proxy).
+
To help diagnose problems using a proxy, you can use the `curl` command with options to use your proxy, and log debug information, with the following command, replacing the proxy and target URLs as appropriate. This will force the request to be made to the
proxy in tunnelling mode, and display some of the interaction between the client and the proxy.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
proxy in tunnelling mode, and display some of the interaction between the client and the proxy.
proxy in tunneling mode, and display some of the interaction between the client and the proxy.

Copy link
Contributor

@mikecote mikecote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM! Ran the tests locally and they passed too 👍

@pmuellr pmuellr added the release_note:skip Skip the PR/issue when compiling release notes label Aug 15, 2022
@pmuellr
Copy link
Member Author

pmuellr commented Aug 15, 2022

@elasticmachine merge upstream

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id before after diff
actions 20 25 +5

Total ESLint disabled count

id before after diff
actions 26 31 +5

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@pmuellr pmuellr merged commit 9631649 into elastic:main Aug 16, 2022
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Aug 16, 2022
The new proxy tests added can test a variety of different proxy
and target server configurations, however many of those tests
are broken with our current proxy agents.  Hopefully to be fixed
by replacing with hpagent instead.  In the meantime, we wanted to
get the basic test framework in as well.

In addition to tests, the stand-alone forward proxy has been
enhanced to use a better proxy server, `proxy`.  The existing
proxy server `http-proxy` does not support HTTPS out of the box,
and so any HTTPS testing with it is going to be a little sketchy.

Using the stand-alone forward proxy, I was able to post to Slack
through http/https proxies with and without auth, with
proxyRequestUnauthorized set to false.  Which shows the existing
proxy agents do work in _some_ environments.

(cherry picked from commit 9631649)
@kibanamachine
Copy link
Contributor

💔 Some backports could not be created

Status Branch Result
7.17 Backport failed because of merge conflicts
8.3 Backport failed because of merge conflicts
8.4

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 138071

Questions ?

Please refer to the Backport tool documentation

pmuellr added a commit to pmuellr/kibana that referenced this pull request Aug 16, 2022
The new proxy tests added can test a variety of different proxy
and target server configurations, however many of those tests
are broken with our current proxy agents.  Hopefully to be fixed
by replacing with hpagent instead.  In the meantime, we wanted to
get the basic test framework in as well.

In addition to tests, the stand-alone forward proxy has been
enhanced to use a better proxy server, `proxy`.  The existing
proxy server `http-proxy` does not support HTTPS out of the box,
and so any HTTPS testing with it is going to be a little sketchy.

Using the stand-alone forward proxy, I was able to post to Slack
through http/https proxies with and without auth, with
proxyRequestUnauthorized set to false.  Which shows the existing
proxy agents do work in _some_ environments.

(cherry picked from commit 9631649)

# Conflicts:
#	package.json
#	x-pack/plugins/actions/server/builtin_action_types/lib/axios_utils_connection.test.ts
#	yarn.lock
kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Aug 16, 2022
…c#138919)

The new proxy tests added can test a variety of different proxy
and target server configurations, however many of those tests
are broken with our current proxy agents.  Hopefully to be fixed
by replacing with hpagent instead.  In the meantime, we wanted to
get the basic test framework in as well.

In addition to tests, the stand-alone forward proxy has been
enhanced to use a better proxy server, `proxy`.  The existing
proxy server `http-proxy` does not support HTTPS out of the box,
and so any HTTPS testing with it is going to be a little sketchy.

Using the stand-alone forward proxy, I was able to post to Slack
through http/https proxies with and without auth, with
proxyRequestUnauthorized set to false.  Which shows the existing
proxy agents do work in _some_ environments.

(cherry picked from commit 9631649)

Co-authored-by: Patrick Mueller <patrick.mueller@elastic.co>
clintandrewhall pushed a commit to clintandrewhall/kibana that referenced this pull request Aug 16, 2022
The new proxy tests added can test a variety of different proxy
and target server configurations, however many of those tests
are broken with our current proxy agents.  Hopefully to be fixed
by replacing with hpagent instead.  In the meantime, we wanted to
get the basic test framework in as well.

In addition to tests, the stand-alone forward proxy has been
enhanced to use a better proxy server, `proxy`.  The existing
proxy server `http-proxy` does not support HTTPS out of the box,
and so any HTTPS testing with it is going to be a little sketchy.

Using the stand-alone forward proxy, I was able to post to Slack
through http/https proxies with and without auth, with
proxyRequestUnauthorized set to false.  Which shows the existing
proxy agents do work in _some_ environments.
kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Aug 18, 2022
…lastic#138920)

* Adds new proxy tests and manual proxy tester (elastic#138071)

The new proxy tests added can test a variety of different proxy
and target server configurations, however many of those tests
are broken with our current proxy agents.  Hopefully to be fixed
by replacing with hpagent instead.  In the meantime, we wanted to
get the basic test framework in as well.

In addition to tests, the stand-alone forward proxy has been
enhanced to use a better proxy server, `proxy`.  The existing
proxy server `http-proxy` does not support HTTPS out of the box,
and so any HTTPS testing with it is going to be a little sketchy.

Using the stand-alone forward proxy, I was able to post to Slack
through http/https proxies with and without auth, with
proxyRequestUnauthorized set to false.  Which shows the existing
proxy agents do work in _some_ environments.

(cherry picked from commit 9631649)

# Conflicts:
#	package.json
#	x-pack/plugins/actions/server/builtin_action_types/lib/axios_utils_connection.test.ts
#	yarn.lock

* fix backport changes

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Mpdreamz pushed a commit to Mpdreamz/kibana that referenced this pull request Sep 6, 2022
The new proxy tests added can test a variety of different proxy
and target server configurations, however many of those tests
are broken with our current proxy agents.  Hopefully to be fixed
by replacing with hpagent instead.  In the meantime, we wanted to
get the basic test framework in as well.

In addition to tests, the stand-alone forward proxy has been
enhanced to use a better proxy server, `proxy`.  The existing
proxy server `http-proxy` does not support HTTPS out of the box,
and so any HTTPS testing with it is going to be a little sketchy.

Using the stand-alone forward proxy, I was able to post to Slack
through http/https proxies with and without auth, with
proxyRequestUnauthorized set to false.  Which shows the existing
proxy agents do work in _some_ environments.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-major Backport to the previous major branch and all later branches still in development docs Feature:Actions/Framework Issues related to the Actions Framework Feature:Actions release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v7.17.6 v8.4.0 v8.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants