Skip to content
This repository was archived by the owner on Dec 14, 2022. It is now read-only.

[BUG] In the subscription mode, the first message is lost at the first startup. #370

Closed
jianyun8023 opened this issue Jul 4, 2021 · 2 comments · Fixed by #371
Closed
Labels

Comments

@jianyun8023
Copy link
Contributor

Describe the bug

If you use an existing subscription and start using the subscription mode of the connector, the first message will be lost.

To Reproduce
Steps to reproduce the behavior:

  1. Create a subscriber for the topic, write 1-100 messages to the topic,
  2. Let Consumer of pulsar-client use this subscriber's message.
  3. Write 1-100 messages to the topic again.
  4. Use the Pulsar-flink connector subscription mode to start, and the first consumed message is lost.

Expected behavior
With existing subscribers, the first message will not be lost.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context

When the connector submits the cursor to pulsar, the submitted cursor is the last message currently consumed. Pulsar admin defaults that the current message is the first message of the next consumption.

In the connector, the first message is lost by default, which is also the reason for the problem.
The root cause is that the cursor submitted by the connector does not comply with pulsar's rules, resulting in inconsistent behavior.

@jianyun8023
Copy link
Contributor Author

This problem is also the main reason for #367

@jianyun8023
Copy link
Contributor Author

When this problem is solved, there will be one less cursor submitted by pulsar-flink than expected. The subscription mode will consume one more message when using the connector that fixes the problem. Repeated consumption will only happen this time.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
1 participant