Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporterhelper] Make the re-enqueuing configurable on the queue sender #8987

Closed
dmitryax opened this issue Nov 23, 2023 · 1 comment
Closed

Comments

@dmitryax
Copy link
Member

dmitryax commented Nov 23, 2023

Currently, the exporter helper puts failed (non-permanent error) requests back to the beginning of the queue only if retry_on_failure and persistent queue are enabled. There are a few problems with that:

  1. There is no way to use the persistent queue and drop requests that the retry sender failed to deliver after several attempts
  2. There is no way to configure the persistent queue with re-enqueueing but with disabled exponential backoff retry.
  3. There is no way to configure re-enqueueing with the memory queue, which potentially still can be useful for someone who wants to make sure that requests are never dropped after reaching the queue as long as collector is up and running.
  4. The queue sender (sending_queue configuration section) behavior is affected by the retry sender (retry_on_failure configuration section).

Proposed solution is to add an option to the sending_queue section:

	// RequeueOnFailure indicates whether requests are requeued or dropped on non-permanent failures.
	// If true, a request failed with a non-permenent error will be put back in the queue and 
	// retried after the current queue is drained.
	// If false (by default), items will be dropped after any failed send operation.
	// Setting this option to true means that the request will never leave the exporter queue until it's 
	// successfully exported or failed with permanent error.
	// Keep in mind that this option is independent of the `retry_on_failure`, which consumes 
	// requests from the queue and apply blocking exponential backoff retry mechanism.
	RequeueOnFailure bool `mapstructure:"requeue_on_failure"`

Introducing this option also requires making sure that the re-enqueue mechanism works well with the memory queue shutdown, which currently can lead to data races. See #8124

@dmitryax
Copy link
Member Author

Not needed anymore. See #8382

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant