Skip to content
This repository has been archived by the owner on Jun 14, 2024. It is now read-only.

update: topic naming #579

Merged
merged 6 commits into from
Mar 13, 2023
Merged

update: topic naming #579

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
60c87a1
update: topic naming
kaiserd Mar 3, 2023
d4706fc
Update content/docs/rfcs/23/README.md
kaiserd Mar 3, 2023
f45d38c
Update content/docs/rfcs/23/README.md
kaiserd Mar 3, 2023
233fa61
update: 23/ more informational, 51/ normative
kaiserd Mar 9, 2023
95248e6
Update content/docs/rfcs/51/README.md
kaiserd Mar 9, 2023
cc1bd8b
Update content/docs/rfcs/51/README.md
kaiserd Mar 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 36 additions & 21 deletions content/docs/rfcs/23/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,47 +3,65 @@ slug: 23
title: 23/WAKU2-TOPICS
name: Waku v2 Topic Usage Recommendations
status: draft
category: Informational
editor: Oskar Thoren <oskar@status.im>
contributors:
- Hanno Cornelius <hanno@status.im>
- Daniel Kaiser <danielkaiser@status.im>
---

This document outlines recommended usage of topics in Waku v2.
This document outlines recommended usage of topic names in Waku v2.
In [10/WAKU2 spec](/spec/10) there are two types of topics:

- PubSub topics, used for routing
- Content topics, used for content-based filtering


## PubSub topics

PubSub topics are used for routing of messages, see [11/WAKU2-RELAY](/spec/11) spec for more details of how this routing works.

As written in [10/WAKU2 spec](/spec/10) there is a default PubSub topic:
[51/WAKU2-RELAY-SHARDING](/spec/51) specifies Relay sharding,
which allows sharding Waku Relay into a hierarchical set of shards.
This document (23/WAKU2-TOPICS) comprises recommendations for naming pubsub topics,
and acts as an informational source for *named sharding*.
Named sharding is one of the sharding strategies specified in [51/WAKU2-RELAY-SHARDING](/spec/51).

The Waku v2 default PubSub topic is:

`/waku/2/default-waku/proto`
`/waku/2/default-waku/`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we need to change the default named pubsub topic (this would require a finicky migration). We can just mention that for legacy reasons it has the /proto encoding trailer, but that we no longer support semantic interpretation of that suffix. In other words, it still adheres to the format /waku/2/{topic-name}, but for legacy reasons {topic-name} is default-waku/proto. The same goes for the recently introduced /waku/2/dev-waku/proto topic.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @jm-clius. For compatibility with the existing implementations, we should support both topics as the default topic:

  • /waku/2/default-waku/proto
  • /waku/2/default-waku

The same way Protocol labs does with the multiaddr protocol ipfs, which is an alternate form for p2p for backward compatibility reasons.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with keeping /waku/2/default-waku/proto
Having two default topics would be a third option.
If I understand correctly, @jm-clius suggestion is only use /waku/2/default-waku/proto as the default topic.
While /waku/2/default-waku would be a valid topic, it is is not declared as a default.


This indicates that

1) It relates to the Waku problem domain
1) It relates to the Waku protocol domain
2) version is 2
3) `default-waku` indicates that it is the default topic for exchanging WakuMessages
4) that the [data field](/spec/11/#protobuf-definition) in PubSub is serialized/encoded as protobuf as determined by WakuMessage

> *Note*: In previous versions of this document, the default topic was `/waku/2/default-waku/proto`.
The now deprecated `/proto` part indicated that the [data field](/spec/11/#protobuf-definition) in PubSub is serialized/encoded as protobuf.
The inspiration for this format was taken from
[Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages).
However, because the payload of messages transmitted over [11/WAKU2-RELAY](/spec/11) must be a [14/WAKU2-MESSAGE](/spec/14),
which specifies the wire format as protobuf,`/proto` is the only valid encoding.
This makes the `/proto` indication obsolete.
The encoding of the `payload` field of a Waku Message is indicated by the /{encoding} part of the content topic name.
Specifying an encoding is only significant for the actual payload/data field.
Waku preserves this option by allowing to specify an encoding for the WakuMessage payload field as part of the content topic name.

### PubSub topic format

PubSub topics SHOULD follow the following structure:

`/waku/2/{topic-name}/{encoding}`
`/waku/2/{topic-name}`

This namespaced structure makes things like compatibility and discoverability and automatic handling of new topics easier.
For example, if the encoding of the payload is changed, compression is introduced, etc.
This namespaced structure makes things like compatibility, discoverability, and automatic handling of new topics easier.
If applicable, it is RECOMMENDED to structure `{topic-name}` in a hierarchical way as well.
For example, `/waku/2/rs/0/2`, where `rs/0/2` is a hierarchically structured `{topic-name}`.

For more on this format of PubSub topics, see [Ethereum 2 P2P spec](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/p2p-interface.md#topics-and-messages) where inspiration for this format was taken from.

### Default PubSub topic

Unless there's a good reason, the default PubSub topic SHOULD be used for all protocols.
However, in certain situations other topics MAY be used.

Using a single PubSub topic ensures a connected network, as well some degree of metadata protection.
See [section on Anonymity/Unlinkability](/spec/10/#anonymity--unlinkability).
Expand All @@ -65,30 +83,25 @@ Let's say we have two different topics that are both experience heavy traffic bu
This can be segregated into:

```
/waku/2/status/proto
/waku/2/walletconnect/proto
/waku/2/status/
/waku/2/walletconnect/
```

This indicates that they are WakuMessages but for different domains completely.

### Topic sharding example

Topic sharding is currently not supported by default, but is planned for the future in order to deal with increased network traffic.
Here's an example of what this might look like:
The following is an example of named sharing, as specified in [51/WAKU2-RELAY-SHARDING](/spec/51).
kaiserd marked this conversation as resolved.
Show resolved Hide resolved

```
waku/2/waku-9_shard-0/proto
waku/2/waku-9_shard-0/
Copy link
Contributor

@alrevuelta alrevuelta Mar 7, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not waku/2/rs/9/0? like in the other example?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an example for named sharding, not static sharding.

...
waku/2/waku-9_shard-9/proto
waku/2/waku-9_shard-9/
```

This indicates explicitly that the network traffic has been partitioned into 10 buckets.

### Compression example

Not yet implemented, but would be easy to add with:

`/waku/2/default-waku/proto_snappy`
Besides named sharing, [51/WAKU2-RELAY-SHARDING](/spec/51) specifies two more sharing methods: static sharding and automatic sharding.
kaiserd marked this conversation as resolved.
Show resolved Hide resolved

## Content topics

Expand Down Expand Up @@ -184,3 +197,5 @@ Copyright and related rights waived via
8. [15/WAKU-BRIDGE](/spec/15)

9. [26/WAKU-PAYLOAD](/spec/26)

10. [51/WAKU2-RELAY-SHARDING](/spec/51)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing with the references.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same reason as above.

11 changes: 5 additions & 6 deletions content/docs/rfcs/51/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,15 @@ This document also covers discovery of topic shards.
# Named Sharding

*Named sharding* offers apps to freely choose pubsub topic names.
App protocols SHOULD follow the naming structure detailed in [23/WAKU2-TOPICS](/spec/23/).
It is RECOMMENDED for App protocols to follow the naming structure detailed in [23/WAKU2-TOPICS](/spec/23/).
With named sharding, managing discovery falls into the responsibility of apps.

The default Waku pubsub topic `/waku/2/default-waku/proto` can be seen as a named shard available to all app protocols.
The default Waku pubsub topic `/waku/2/default-waku` can be seen as a named shard available to all app protocols.
Copy link
Contributor

@LNSD LNSD Mar 3, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same comment as @jm-clius pointed out, for compatibility with the existing implementations, we should support both topics as the default topic:

  • /waku/2/default-waku/proto
  • /waku/2/default-waku

The same way Protocol labs does with the multiaddr protocol ipfs which is an alternate form for p2p for backward compatibility reasons.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(see above)


> *Note*: Future versions of this document are planned to give more guidance with respect to discovery via
[33/WAKU2-DISCV5](/spec/33/),
[DNS discovery](https://eips.ethereum.org/EIPS/eip-1459),
and inter-mesh discovery via gossipsub control messages (also using circuit relay).
It might make sense to deprecate [23/WAKU2-TOPICS](/spec/23/) as a separate spec and merge it here.

From an app protocol point of view, a subscription to a content topic `waku2/xxx` on a shard named /mesh/v1.1.1/xxx would look like:

Expand Down Expand Up @@ -88,15 +87,15 @@ This offers k-anonymity and better connectivity, but comes at a higher bandwidth

The name of the pubsub topic corresponding to a given static shard is specified as

`/waku/2/static-rshard/<shard_cluster_index>/<shard_number>`,
`/waku/2/rs/<shard_cluster_index>/<shard_number>`,

an example for the 2nd shard in the global shard cluster:

`/waku/2/static-rshard/0/2`.
`/waku/2/rs/0/2`.

> *Note*: Because *all* shards distribute payload defined in [14/WAKU2-MESSAGE](spec/14/) via [protocol buffers](https://developers.google.com/protocol-buffers/),
the pubsub topic name does not explicitly add `/proto` to indicate protocol buffer encoding.
We use `rshard` (as well as `rs` as the ENR key) to indicate these are relay shard clusters; further shard types might follow in the future.
We use `rs` to indicate these are *relay shard* clusters; further shard types might follow in the future.

From an app point of view, a subscription to a content topic `waku2/xxx` on a static shard would look like:

Expand Down