Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JetStream InterestPolicy inconsistent when a JS Consumer has blank (all) Subject Filter #2836

Closed
tbeets opened this issue Feb 1, 2022 · 0 comments
Closed

JetStream InterestPolicy inconsistent when a JS Consumer has blank (all) Subject Filter #2836

tbeets opened this issue Feb 1, 2022 · 0 comments

Comments

@tbeets
Copy link
Contributor

tbeets commented Feb 1, 2022

Defect

When a stream is configured as InterestPolicy, message delete behavior on stream publish is inconsistent if one of the consumers on the stream is configured with blank subject filter (interest in all messages of underlying stream) and there is at least one other consumer on the stream with an explicit (non-blank) subject filter.

If consumer with blank subject filter is the only consumer, the published message is retained and not deleted.

If there is a second consumer with non-blank subject filter that is not a match for published message, the message is immediately deleted on publish.

If there is a second consumer with non-blank subject filter that is a match for published message, the message is retained and also visible to the "all" consumer as well.

Versions of nats-server and affected client libraries used:

Server: 2.7.2-beta
CLI: 0.0.28

OS/Container environment:

Linux

Steps or code to reproduce the issue:

On publish behavior:

nats str add interest1 with interest policy, filter interest1.>
nats con add interest1 con1 with blank subject filter (all)
nats pub interest1.bobo hello note stream and con1 reflect 1 message
nats con add interest1 con2 with subject filter interest1.foo
nats pub interest1.bobo hello note stream reflects 1 deleted message and con1 never sees
nats pub interest1.foo hello note stream reflects 2 messages that both con1 and con2 see

Non-publish behavior (interest calculation at other times) as expected:

nats con next interest1 con2 --ack note stream still reflects 2 messages and con1 can still see 2
nats con rm interest1 con2 note stream still reflects 2 messages and con 1 can still see 2

Expected result:

A consumer with blank subject filter means interest in all messages of the underlying stream regarding of subject. JetStream should honor this interest on stream publish in all conditions.

Actual result:

On publish, JetStream either sees the interest of the "all" consumer or doesn't see the interest of the "all" consumer depending on whether there are other consumers with explicit subject filter defined on the JetStream.

Additional reproduction detail

Create stream

Stream interest1 was created

Information for Stream interest1 created 2022-02-01T13:00:18-08:00

Configuration:

             Subjects: interest1.>
     Acknowledgements: true
            Retention: File - Interest
             Replicas: 3
       Discard Policy: Old
     Duplicate Window: 2m0s
    Allows Msg Delete: true
         Allows Purge: true
       Allows Rollups: false
     Maximum Messages: unlimited
        Maximum Bytes: unlimited
          Maximum Age: unlimited
 Maximum Message Size: unlimited
    Maximum Consumers: unlimited


Cluster Information:

                 Name: cluster1
               Leader: vbox2
              Replica: vbox1, current, seen 0.00s ago
              Replica: vbox3, current, seen 0.00s ago

State:

             Messages: 0
                Bytes: 0 B
             FirstSeq: 0
              LastSeq: 0
     Active Consumers: 0

Create consumer with blank subject interest filter (all)

Information for Consumer interest1 > con1 created 2022-02-01T13:01:14-08:00

Configuration:

        Durable Name: con1
           Pull Mode: true
      Deliver Policy: All
          Ack Policy: Explicit
            Ack Wait: 30s
       Replay Policy: Instant
     Max Ack Pending: 20,000
   Max Waiting Pulls: 512

Cluster Information:

                Name: cluster1
              Leader: vbox1
             Replica: vbox2, current, seen 0.01s ago
             Replica: vbox3, current, seen 0.01s ago

State:

   Last Delivered Message: Consumer sequence: 0 Stream sequence: 0
     Acknowledgment floor: Consumer sequence: 0 Stream sequence: 0
         Outstanding Acks: 0 out of maximum 20,000
     Redelivered Messages: 0
     Unprocessed Messages: 0
            Waiting Pulls: 0 of maximum 512

Publish a message to stream and note expected behavior (there is outstanding interest and message retained)

$ nats pub interest1.bobo hello
13:03:47 Published 5 bytes to "interest1.bobo"
$ nats str info interest1
Information for Stream interest1 created 2022-02-01T13:00:18-08:00

Configuration:

             Subjects: interest1.>
     Acknowledgements: true
            Retention: File - Interest
             Replicas: 3
       Discard Policy: Old
     Duplicate Window: 2m0s
    Allows Msg Delete: true
         Allows Purge: true
       Allows Rollups: false
     Maximum Messages: unlimited
        Maximum Bytes: unlimited
          Maximum Age: unlimited
 Maximum Message Size: unlimited
    Maximum Consumers: unlimited


Cluster Information:

                 Name: cluster1
               Leader: vbox2
              Replica: vbox1, current, seen 0.51s ago
              Replica: vbox3, current, seen 0.02s ago

State:

             Messages: 1
                Bytes: 49 B
             FirstSeq: 1 @ 2022-02-01T21:03:46 UTC
              LastSeq: 1 @ 2022-02-01T21:03:46 UTC
     Active Consumers: 1

$ nats con info interest1 con1
Information for Consumer interest1 > con1 created 2022-02-01T13:01:14-08:00

Configuration:

        Durable Name: con1
           Pull Mode: true
      Deliver Policy: All
          Ack Policy: Explicit
            Ack Wait: 30s
       Replay Policy: Instant
     Max Ack Pending: 20,000
   Max Waiting Pulls: 512

Cluster Information:

                Name: cluster1
              Leader: vbox1
             Replica: vbox3, current, seen 0.00s ago
             Replica: vbox2, current, seen 0.00s ago

State:

   Last Delivered Message: Consumer sequence: 0 Stream sequence: 0
     Acknowledgment floor: Consumer sequence: 0 Stream sequence: 0
         Outstanding Acks: 0 out of maximum 20,000
     Redelivered Messages: 0
     Unprocessed Messages: 1
            Waiting Pulls: 0 of maximum 512

Create a second consumer with explicit subject filter interest1.foo

Information for Consumer interest1 > con2 created 2022-02-01T13:06:10-08:00

Configuration:

        Durable Name: con2
           Pull Mode: true
      Filter Subject: interest1.foo
      Deliver Policy: All
          Ack Policy: Explicit
            Ack Wait: 30s
       Replay Policy: Instant
     Max Ack Pending: 20,000
   Max Waiting Pulls: 512

Cluster Information:

                Name: cluster1
              Leader: vbox2
             Replica: vbox1, current, seen 0.01s ago
             Replica: vbox3, current, seen 0.01s ago

State:

   Last Delivered Message: Consumer sequence: 0 Stream sequence: 0
     Acknowledgment floor: Consumer sequence: 0 Stream sequence: 0
         Outstanding Acks: 0 out of maximum 20,000
     Redelivered Messages: 0
     Unprocessed Messages: 0
            Waiting Pulls: 0 of maximum 512

Publish another message to stream (but not a match to the second consumer's filter). Whoops.

$ nats pub interest1.bobo hello
13:07:13 Published 5 bytes to "interest1.bobo"
$ nats str info interest1
Information for Stream interest1 created 2022-02-01T13:00:18-08:00

Configuration:

             Subjects: interest1.>
     Acknowledgements: true
            Retention: File - Interest
             Replicas: 3
       Discard Policy: Old
     Duplicate Window: 2m0s
    Allows Msg Delete: true
         Allows Purge: true
       Allows Rollups: false
     Maximum Messages: unlimited
        Maximum Bytes: unlimited
          Maximum Age: unlimited
 Maximum Message Size: unlimited
    Maximum Consumers: unlimited


Cluster Information:

                 Name: cluster1
               Leader: vbox2
              Replica: vbox1, current, seen 0.11s ago
              Replica: vbox3, current, seen 0.11s ago

State:

             Messages: 1
                Bytes: 49 B
             FirstSeq: 1 @ 2022-02-01T21:03:46 UTC
              LastSeq: 2 @ 2022-02-01T21:07:12 UTC
     Deleted Messages: 1
     Active Consumers: 2

$ nats con info interest1 con1
Information for Consumer interest1 > con1 created 2022-02-01T13:01:14-08:00

Configuration:

        Durable Name: con1
           Pull Mode: true
      Deliver Policy: All
          Ack Policy: Explicit
            Ack Wait: 30s
       Replay Policy: Instant
     Max Ack Pending: 20,000
   Max Waiting Pulls: 512

Cluster Information:

                Name: cluster1
              Leader: vbox1
             Replica: vbox3, current, seen 0.37s ago
             Replica: vbox2, current, seen 0.36s ago

State:

   Last Delivered Message: Consumer sequence: 0 Stream sequence: 0
     Acknowledgment floor: Consumer sequence: 0 Stream sequence: 0
         Outstanding Acks: 0 out of maximum 20,000
     Redelivered Messages: 0
     Unprocessed Messages: 1
            Waiting Pulls: 0 of maximum 512

Publish another message to stream that is a match for second consumer's explicit subject filter

$ nats pub interest1.foo hello
13:08:29 Published 5 bytes to "interest1.foo"
$ nats str info interest1
Information for Stream interest1 created 2022-02-01T13:00:18-08:00

Configuration:

             Subjects: interest1.>
     Acknowledgements: true
            Retention: File - Interest
             Replicas: 3
       Discard Policy: Old
     Duplicate Window: 2m0s
    Allows Msg Delete: true
         Allows Purge: true
       Allows Rollups: false
     Maximum Messages: unlimited
        Maximum Bytes: unlimited
          Maximum Age: unlimited
 Maximum Message Size: unlimited
    Maximum Consumers: unlimited


Cluster Information:

                 Name: cluster1
               Leader: vbox2
              Replica: vbox1, current, seen 0.45s ago
              Replica: vbox3, current, seen 0.45s ago

State:

             Messages: 2
                Bytes: 97 B
             FirstSeq: 1 @ 2022-02-01T21:03:46 UTC
              LastSeq: 3 @ 2022-02-01T21:08:29 UTC
     Deleted Messages: 1
     Active Consumers: 2

$ nats con info interest1 con1
Information for Consumer interest1 > con1 created 2022-02-01T13:01:14-08:00

Configuration:

        Durable Name: con1
           Pull Mode: true
      Deliver Policy: All
          Ack Policy: Explicit
            Ack Wait: 30s
       Replay Policy: Instant
     Max Ack Pending: 20,000
   Max Waiting Pulls: 512

Cluster Information:

                Name: cluster1
              Leader: vbox1
             Replica: vbox2, current, seen 0.16s ago
             Replica: vbox3, current, seen 0.16s ago

State:

   Last Delivered Message: Consumer sequence: 0 Stream sequence: 0
     Acknowledgment floor: Consumer sequence: 0 Stream sequence: 0
         Outstanding Acks: 0 out of maximum 20,000
     Redelivered Messages: 0
     Unprocessed Messages: 2
            Waiting Pulls: 0 of maximum 512

$ nats con info interest1 con2
Information for Consumer interest1 > con2 created 2022-02-01T13:06:10-08:00

Configuration:

        Durable Name: con2
           Pull Mode: true
      Filter Subject: interest1.foo
      Deliver Policy: All
          Ack Policy: Explicit
            Ack Wait: 30s
       Replay Policy: Instant
     Max Ack Pending: 20,000
   Max Waiting Pulls: 512

Cluster Information:

                Name: cluster1
              Leader: vbox2
             Replica: vbox1, current, seen 0.11s ago
             Replica: vbox3, current, seen 0.11s ago

State:

   Last Delivered Message: Consumer sequence: 0 Stream sequence: 0
     Acknowledgment floor: Consumer sequence: 0 Stream sequence: 0
         Outstanding Acks: 0 out of maximum 20,000
     Redelivered Messages: 0
     Unprocessed Messages: 1
            Waiting Pulls: 0 of maximum 512
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bruth @tbeets