Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

store Multiaddrs as strings and don't use interfaces #62

Closed
wants to merge 5 commits into from

Conversation

Stebalien
Copy link
Member

  1. This saves one level of indirection, one distinct allocation, and a few bytes.
  2. This allows Multiaddrs to be used as map keys (we can use this to make our Peerstore more efficient).
  3. Removing the interface allows us to serialize these without copying.

This commit isn't as efficient as it could be. We could improve it slightly by using unsafe casts to temporarily treat strings as bytes. However I'm not sure this will improve performance much and, in general, this change will lead to fewer allocations overall.

@Stebalien
Copy link
Member Author

The tests are failing because this change breaks GX. We'd have to fix that concurrently with this PR.

@whyrusleeping thoughts?

This will be a real pain (maddr == nil is broken) but, it will save us a ton of byte slice allocations in the DHT (we send out a lot of multiaddrs and copy them every time).

An alternative is to define multiaddrs to be type Multiaddr []byte (allowing a cheep cast via []byte(Multiaddr)) but that wouldn't allow us to use them in map keys. On the other hand maddr == nil will still work.

Stebalien added a commit that referenced this pull request Dec 16, 2017
This allows us to send multiaddrs on the wire without copying (through the
`Bytes()` method). It also avoids one allocation/indirection per multiaddr.

This is an alternative (or stop-gap) to #62 as #62 is invasive. Unlike #62, this
PR doesn't break `Multiaddr == nil` or `maddr1.Equals(maddr2)`. However,
it *doesn't* allow multiaddrs to be used as map keys :(.
@whyrusleeping
Copy link
Member

@Stebalien why not keep the interface, and just do:

type multiaddr string

That way we get all the same benefits, plus the ability to have nil multiaddrs (making the upgrade path easier)

1. We can now efficiently use them as map keys.
2. We can now efficiently sort them (without copying).
3. We can (in theory) put them in protobufs without copying (yay).

This commit isn't as efficient as it could be. We could improve it slightly by
using unsafe casts to temporarily treat strings as bytes. However I'm not sure
this will improve performance much.
@Stebalien
Copy link
Member Author

Note: I'm pretty sure that'll still require an additional allocation/indirection but it should be fairly cheep.

@Kubuxu
Copy link
Member

Kubuxu commented Mar 27, 2018

I think Go is smart enough and allocates the type header in the same call as the string.

@Stebalien
Copy link
Member Author

I think Go is smart enough and allocates the type header in the same call as the string.

It's the string header I'm worried about. Strings are usually:

  • data ->
    • "data"
  • length
  • capacity

And interfaces are usually:

  • vtable ->
    • ...
  • data ->
    • ...

So, a string wrapped in an interface should be:

  • vtable ->
    • ...
  • data ->
    • data ->
      • "data"
    • length
    • capacity

From a bit of quick testing, it looks like this is correct.

@Kubuxu
Copy link
Member

Kubuxu commented Mar 27, 2018

From a bit of quick testing, it looks like this is correct.

correct as in "it adds allocation"?

@Stebalien
Copy link
Member Author

Correct as in it adds a level of indirection. Go may do something sneaky with the allocation but I kind of doubt it. Go could do something to avoid an extra object to GC (e.g., have some concept of ownership) but the complexity probably isn't worth it.

@Stebalien Stebalien closed this Jun 16, 2018
Stebalien added a commit that referenced this pull request May 20, 2020
….com/multiformats/go-multiaddr-0.2.0

Bump github.com/multiformats/go-multiaddr from 0.1.0 to 0.2.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants