Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User params #33

Merged
merged 7 commits into from
Jun 13, 2018
Merged

User params #33

merged 7 commits into from
Jun 13, 2018

Conversation

hagen1778
Copy link
Contributor

This PR allows to create param_groups - named lists of URL params. If the group is attached to user in params option, proxy will send those params with every user's request.

The param_groups configuration is following:

# Optional list of GET params to send with each request
param_groups:
    # Group name, which may be passed into `params` option on the `user` level.
  - name: "cron-job"
    # List of key-value params to send
    params:
      - key: "max_memory_usage"
        value: "40000000000"

      - key: "max_bytes_before_external_group_by"
        value: "20000000000"

Caching is still available while using param_groups. Proxy will get a hash from the params list and use it as a part of cache.Key

@hagen1778 hagen1778 requested a review from valyala June 12, 2018 12:54
README.md Outdated
@@ -391,6 +391,29 @@ network_groups:
- name: "reporting-apps"
networks: ["10.10.10.0/24"]

# Optional list of GET params to send with each request
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

substitute this with the following text:

Optional lists of query params to send with each proxied request to ClickHouse.
These lists may be used for overriding ClickHouse settings on a per-user basis.

README.md Outdated
@@ -420,7 +443,6 @@ server:
# Certificates are automatically issued and renewed if this section
# is present.
# There is no need in cert_file and key_file if this section is present.
# Autocert requires application to listen on :80 port for certificate generation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this line has been removed?

Copy link
Contributor Author

@hagen1778 hagen1778 Jun 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was replaced with config/testdata/full.yml where this line was absent. Will add it in both places

README.md Outdated
@@ -467,6 +489,11 @@ users:
# By default responses aren't cached.
cache: "longterm"

# ParamGroup name to use
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Substitute this with:

An optional group of params to send to ClickHouse with each proxied request.
These params may be set in param_groups block.

By default no additional params are sent to ClickHouse.

config/config.go Outdated
@@ -464,7 +469,13 @@ func (ng *NetworkGroups) UnmarshalYAML(unmarshal func(interface{}) error) error
if err := unmarshal((*plain)(ng)); err != nil {
return err
}
return checkOverflow(ng.XXX, "network_groups")
if len(ng.Name) == 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, this change should be pushed in a separate commit if it is correct, since it is unrelated to the PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. Should be fixed in a separate PR

@@ -125,6 +160,7 @@ func TestLoadConfig(t *testing.T) {
MaxExecutionTime: Duration(time.Minute),
DenyHTTPS: true,
NetworksOrGroups: []string{"office", "1.2.3.0/24"},
Params: "cron-job",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either remove Params from this user or add a user without params in order to verify user config without params works as expected.

scope.go Outdated
// key is a hashed concatenation of the params list
key uint32

r map[string]string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not storing []config.Param instead? This way params will be added to proxied requests always in the same order.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, thx for pointing that out!

@hagen1778
Copy link
Contributor Author

@valyala thanks for reviewing!

Copy link
Contributor

@valyala valyala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you make sure the full config example in README.md is equivalent to full.yml in a follow-up commit without review?

@@ -488,6 +518,8 @@ users:
to_user: "default"
allowed_networks: ["office", "1.2.3.0/24"]

params: "cron-job"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, this line must be removed in order to match full.yaml

@valyala valyala merged commit 9182f5b into master Jun 13, 2018
@vfoucault vfoucault deleted the user-params branch September 27, 2021 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants