Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NET-9467, NET-9468] Add partitions + exportedServices funcs #1940

Merged
merged 11 commits into from
May 28, 2024

Conversation

nathancoleman
Copy link
Member

@nathancoleman nathancoleman commented May 21, 2024

Overview

This pull request adds two new functions to consul-template:

  • partitions, which lists all partitions in the local datacenter using the existing /v1/partitions API
  • exportedServices, which lists all exported services in a given partition using the existing /v1/exported-services API

Example

By combining these two functions, you can list the services imported into the local partition from all other partitions within the local datacenter.

Note

The following should probably be optimized with a {{ break }} to end the range early once $localPartition has been identified as a consumer of a given service; however, I haven't had a chance to test with that change in place yet, so I'm noting it here.

{{- $localPartition := "ap2" -}}

{{/* Loop through all partitions in the local datacenter */}}
{{- range $partition := partitions -}}

  {{/* Loop through exported services for partition, unless it's our own partition */}}
  {{- if not (eq $partition.Name $localPartition) -}}
    {{- range $svc := (exportedServices $partition.Name) -}}

      {{/* For each service, loop through partition consumers looking for our own partition */}}
      {{- range $consumer := .Consumers.Partitions -}}

        {{/* Print only when the service is exported to our own partition */}}
        {{- if eq $consumer $localPartition -}}
        {{- printf "Service %q exported from %q to %q\n" $svc.Service $partition.Name $consumer -}}
        {{- end -}}

      {{- end -}}
    {{- end -}}
  {{- end -}}
{{- end -}}

@nathancoleman nathancoleman changed the title [NET-9467] Add partitions func for listing partitions in the local datacenter [NET-9467, NET-9468] Add partitions + exportedServices funcs May 21, 2024
@nathancoleman nathancoleman requested a review from blake May 21, 2024 21:11
docs/templating-language.md Outdated Show resolved Hide resolved
@@ -121,7 +121,6 @@ func TestNewNVGetQuery(t *testing.T) {
assert.Equal(t, tc.exp, act)
})
}
fmt.Println("done")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was unnecessary

exitCh <- m.Run()
}()

exit := <-exitCh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this channel shenanigans doesn't actually do anything

@@ -43,6 +43,20 @@ func TestMain(m *testing.M) {
tb := &test.TestingTB{}
runTestConsul(tb)
clients := NewClientSet()

defer func() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved the defer here so we actually catch panics during setup, this won't catch panics during test runs, I spent some time trying to get to a working solution but it would require checking for running instances of consul/nomad/vault that did not previously get cleaned up before running

@nathancoleman nathancoleman marked this pull request as ready for review May 28, 2024 16:18
Copy link
Member

@jmurret jmurret left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, nothing blocking from my additional comments, but think you do need to address RB's comment about exported services implementing blocking queries. You'll want to remove that logic that does this select based on the sleep time.

dependency/consul_exported_services_test.go Outdated Show resolved Hide resolved

// ListPartitionsQuerySleepTime is the amount of time to sleep between
// queries, since the endpoint does not support blocking queries.
ListPartitionsQuerySleepTime = 15 * time.Second
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[NIT] wonder if we want to create just one LongPollingSleepTime set to 15 seconds for use across all of the non blocking query endpoints?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we kinda can, we do some re-assigning of these within tests and I'm not 100% on how that will play out if these tests run in parallel (since we could get a concurrent read and write) what I did though was move this to a DefaultNonBlockingQuerySleepTime constant and assign the individual sleep times to that const

dependency/consul_partitions.go Outdated Show resolved Hide resolved
dependency/consul_partitions.go Outdated Show resolved Hide resolved
dependency/consul_partitions_test.go Outdated Show resolved Hide resolved
@jm96441n jm96441n merged commit 14965b7 into main May 28, 2024
54 checks passed
@jm96441n jm96441n deleted the partitions-func branch May 28, 2024 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants