2018 Q4 OKRs Planning

[daviddias]

Ref: ipfs/team-mgmt#698

• 2018 Q3 Retrospective Document. Please complete this ASAP.
• Previous OKR Planning for reference
• 2018 Q4 OKR Spreadsheet

@vasco-santos, @jacobheun et al, you know the drill :)

[jacobheun]

With the fairly short Q3 we had with all the great events in July, we have some OKRs that need to get carried over into Q4. Here's an outline of my thoughts for focus in Q4.

Q4 Objectives

Debug-ability and Stability

The State Machine updates for libp2p-switch and libp2p will be wrapping up soon. These updates will introduce some error standardization to libp2p, but the work needs to continue.

Standard Errors (from Q3)

Standardize connection errors and add them to the ipfs initiative, #1406
This must be done to make the implementation successful.

Implement in interface-connection modules

By leveraging standard errors for all modules implementing the connection interface, the libp2p-switch connection state machines will be better able to respond to the errors appropriately.
Fatal codes could terminate connections outright and not cause libp2p or switch to crash the node. Unstable connections could be reported up to libp2p and the connection manager to handle. The connection manager could make decisions on which connections to keep, and/or potentially add dialing backoffs for connections (network outage, a restarting peer, etc).

Network Visualization

Being able to visualize the network can also be a useful tool for finding issues with the network. As libp2p's Connectivity Magic gets better and better, it would also be really helpful for people to be able to see differences in the network with things like Delegated Routing, the DHT and Circuit Relay enabled/disabled. That could provide a much clearer understanding of the intent and use of these components.

Interop Testing (Global libp2p OKR)

Libp2p Daemon

In order for interoperability between the libp2p implementations to be easily updated by each team, we need to ensure that libp2p daemons exist for each of them. Initial work has started on the go daemon, and we need to get this going for js very soon.

Connectivity Magic

Complete DHT and IPNS work from Q3

• DHT interop with GO
• IPNS over Pubsub
• IPNS over DHT

Formalize NAT Spec

Aside from running out of time for NAT Traversal work, the other blocker for getting this completed is formalizing the NAT Spec, libp2p/specs#64, so that we can continue with the implementation of the NAT Manager. Completing the Spec will also help provide more guidance for any community members that might want to work on completing the implementation.

JS-IPFS Support for Delegated Routing

Refs support for the js-api

Right now, delegated routing is reliant on go-ipfs as js-ipfs doesn't currently support refs. It would be nice to see the refs implementation added to js-ipfs so we could get the API updated for Delegated Routing Support. This would enable users to better support their own ipfs infrastructures and rely less on the dht for browser/mobile nodes, and provide us with more opportunities to take load off the gateway nodes and distribute that throughout the network

Note: I have made some updates to the js-ipfs-api to support delegated routing. If refs support was added to js-ipfs, it would be a fairly low effort to finish the api support.

[vasco-santos]

As we had several events and holidays during the last quarter, several OKRs from the previous quarter were not finished and I think they should be a priority for the next quarter.

IPNS in JS Land

The main implementation of IPNS is already released and the IPNS over Pubsub is ready for review. With the work done in IPNS over Pubsub, the IPNS over DHT will not need a big extra effort. However, it needs the DHT to be ready first.

Awesome DHT Endeavour

Since the DHT is a really important part of libp2p, I consider that we should get it ready to be enabled by default in IPFS. For this, we also need to guarantee the full interoperability with go. Accordingly, we should have an OKR to have a fully interoperable DHT enabled by default in js-ipfs.

Note: There is currently any example for using the DHT in GO nor JS, and we should provide it as well.

References:

• js-ipfs#856
• interop#36

Analyse and Improve DHT Performance

As stated before, DHT is a core concept of libp2p and after we get it enabled by default and fully interoperable with go, we should start analyzing its behavior and performance. Accordingly, I think that the next step should be to test it with a large network and scale it.

Daemon and Interoperability

I think that this should be one of the main focus of the next quarter. Go team already started their implementation and once they have their own daemon and we have ours, we should start the interop tests, which I consider crucial right now. As rust-libp2p is also on its way, we have to guarantee the same behavior accross all implementation to get libp2p a first class project.

References:

• go-libp2p-daemon

Error handling

Providing a stable, reliable and performant libp2p is crucial for the developers' productivity, as well as to the community engagement. This way, I believe that this effort now will be important in the long run, since it will decrease considerably the time needed for diagnosing current and future problems.

Since the last quarter that we are having an effort to have error handling into a better consideration, but there is much to be done and we should refactor the errors sooner, at least for the most error prone modules.

Network visualization tool

This tool did not have any relevant progress last quarter, but it is an important aspect of debuggability for a complex network. We should have it as a OKR as well.

[ghost]

These seem like great goals to me. Let's make sure the js-libp2p goals ultimately land in the libp2p OKR sheet. As Juan was saying to me recently, the interface between projects should be the OKRs themselves. So, something like a fully interoperable DHT should be in libp2p OKRs, whereas interoperable DHT enabled by default in js-ipfs should be in js-ipfs OKRs.

One question: both the Go and JS teams are building independent libp2p daemons. Is this intentional? I can definitely understand it from a testing standpoint, but in terms of exposing APIs, language bindings, etc., to downstream users, it seems like it would be easier to maintain one daemon as the canonical one. Thoughts? / cc @diasdavid @jacobheun @raulk @bigs

[vasco-santos]

@mgoelzer in terms of exposing APIs to downstream users you are completely right!

After some thoughts, taking into consideration that go-libp2p is the reference implementation if we only have a daemon for it, we can test the interoperability from every single other implementation with it. For instance in IPFS land, it is possible to use an in proc node with js-ipfsd-ctl, without the need for a daemon. It is important pointing out that the programatic interface needs to be exactly the same, in order to be transparent in the interop tests which type of node we are dealing it.

However, if we intend to have a CLI for libp2p as well, daemons for both will ease the general "testing" interface, but will need a considerable higher time.

[jacobheun]

@mgoelzer we'll make sure to move these over to the sheet. We like to discuss them openly in PRs as it's much easier than a spreadsheet for discussion and also improves visibility and opportunity to contribute for community members.

In regards to the daemon, our highest priority on the JS side of things is interop testing. As you mentioned, having daemons for each implementation makes things much easier and increases the likelihood that each working group go/js/rust, will contribute to them. The current interop tests for IPFS only really get used by the JS team, because it's all in JS.
The daemon would also make it much easier to spin up go/js/rust nodes via IPTB for better testing of simulated networks.

I agree that we should focus language bindings and exposed APIs to go-libp2p, as the reference implementation, to minimize support needs and focus enhancement efforts.

[pgte]

My 2 cents having landed in IPFS-land 2 years ago, but I'm going to be quite opiniative and probably controversial here:

I think that supporting more than one daemon is somewhat wasteful.
Do we have interface and interop tests because we need a js-ipfs daemon (I think we don't) or, as Jacob put, it's the other way around? I think this chicken-and-egg problem colapses over itself, leaving js-ipfs as a JS embedded API and remote client. I think these two should be the focal point of js-ipfs.
=> Less codebase to maintain and bundle.

Again, I may not be seeing the overall picture here, so I'm surely biased.

[ghost]

@jacobheun Completely agree about better to discuss in Github issues. My job is just to make sure that these items eventually get logged in a Google sheet, since this is how PL as an organization wants to track goal across all projects that it sponsors. So, I believe we're in agreement here.

On the libp2p daemon stuff:

• My initial perspective is that daemons in every language makes sense for testing (IPTB), but there should be some canonical daemon for external consumers (java bindings, python bindings, etc).
• I didn't mean to say that the go libp2p daemon must be the canonical one, just that something should be canonical. I could see a case for a Node.js daemon being canonical, or Rust for that matter.
• I need to think a little more about @pgte 's comment. Pedro, when I say "a daemon" in this thread, I mean a libp2p daemon, i.e., totally uncoupled from IPFS. The reason I think we want this is because we have users who want to build on top of libp2p alone (like Ethereum 2.0 or Polkadot). Apologies if I'm misunderstanding your comment.

[jacobheun]

With our ultimate goal being to reliably test and verify that different implementations of libp2p are able to intercommunicate, and properly support the same feature sets (private networks, dht, transports, crypto, etc), how do we efficiently and effectively create these test suites?

One option, which is being proposed, is to have the daemons for that. This would allow for an agnostic approach to testing.

go-libp2p has its own motivations for creating their daemon.

I am completely open to other options for interop tests between the libp2p implementations, but we need to have it. We want libp2p to be useable and useful to users on its own, and I think interop testing is an important part of that.

Are there other recommendations or ideas on how to approach this?

[daviddias]

@jacobheun Completely agree about better to discuss in Github issues. My job is just to make sure that these items eventually get logged in a Google sheet, since this is how PL as an organization wants to track goal across all projects that it sponsors.

The metapoint to have in consideration here is that this isn't the first time this team does OKR planning in the open and then moves it to a spreadsheet. No need to restate what is already knowledge and habit of the team as it might be received (as it was) that perhaps there is a new way to do things. @mgoelzer, make sure to read ipfs/team-mgmt#698 as it describes how we've been functioning and its benefits in depth. Let me know if it is helpful.

... is to have the daemons for that. This would allow for an agnostic approach to testing.

Yes, we need to have a daemon or at least some RPC API that lets us orchestrate multiple libp2p nodes remotely. Bonus points if the APIs are the same so that we can reuse the same CLI/Client Libraries for both.

I think that supporting more than one daemon is somewhat wasteful. Do we have interface and interop tests because we need a js-ipfs daemon (I think we don't) or, as Jacob put, it's the other way around?

Think of it as IPLD and js-ipfs. IPLD has its own API that gets bubbled up to js-ipfs.

The libp2p Daemon RPC API can be literally what IPFS exposes to the users as well as the Networking API of IPFS https://github.com/ipfs/interface-ipfs-core/issues/285

Missing Objective: Massive testbed

We should not end 2018 without having a testbed for js-libp2p that spins thousands of nodes (a good goal would be to be at least 50000), both browser and Node.js ones, to test Peer and Content Routing, PubSub and overall robustness.

[jacobheun]

I've committed a first pass of the OKRs to the markdown file for review. I went ahead and added my thoughts on assignments as well, please feel free to recommend different assignments!

[jacobheun]

@vasco-santos I thought this might be a good one for you to take with your work on the dht and pubsub. It will definitely be blocked by the base testbed, which I am assigned to, but I thought it would also be good exposure for us to both get involved more heavily in this. The daemon/rpc work is also a blocked for the testbed.

[vasco-santos]

I strongly agree that those OKR's are extremely important for the next quarter and I would like to be involved with them as well.

[jacobheun]

We've had several recent questions around how to use certain modules (dht, circuit-relay, pnet), so I thought it would be a good idea to get this on the list

[vasco-santos]

Just added another item, @jacobheun added a quite complete list and I agree with all the items he added. Left just a few comments.

Just one note related to the assignments.
I feel that they are almost as I would expect, but as I have been working in interop tests (with IPFS) for IPNS and DHT, I think it could make sense for me to own the interop tests here as well. This way, as the interop tests and the testbed are blocked by the RPC API, you could focus first on the RPC API, and when it is ready, while you focus on the testbed and I could work on the interop tests.

We can swap one of my OKRs with that one. As I did not start the network visualization tool in the last quarter, you could focus on that while I would be working on the testbed for DHT/Pubsub.

@jacobheun How do you feel about this change?

[vasco-santos]

I agree! Even when I started creating a new suite of tests for DHT interop, I tried to find examples both on go and js, but there was any. However, I would like something more.

I think we should aim to have examples for the key modules, as well as have a general overview to the README of several modules, which are really incomplete. Something like Usage examples and proper documentation exist for the key modules of libp2p. What do you think?

[jacobheun]

Yes, I agree. Many of the recent questions that have come up could be answered by better/clearer answers to what the modules does and how configuration options affect that

[jacobheun]

@vasco-santos I think that switch makes sense and should help prevent us from getting blocked mid quarter. I'll update it!

[daviddias]

It would be good now to do a Timing Exercise (count the number of days that you effectively have in the quarter) and plan for 60% of it.

At the same time, add priorities for each of these KRs.

[vasco-santos]

There are 66 weekdays in this quarter. From those 66 days, 3 are public holidays in Portugal. Also removing the days from the team week and my scheduled holidays I will have 54 days of full availability and 32.4 days for 60% of my time.

[jacobheun]

For me: 66 total days, minus scheduled holidays and team week (53 days), for a dev capacity at 60% of 31.8 days.

This gives us just over 6weeks each of dev time.

I added priorities to the items, let me know your thoughts.

As it stands, with our availability and the priority order, I think realistically the network visualization tool, nat traversal spec and error handling won't be completed. I think error handling will improve for code we'll be working in, but additional efforts to improve existing code will take a back seat. I think these are good for stretch goals though.

[daviddias]

L12 and L13 seem very similar to me and they are a HUGE undertaking, perhaps the most important one (right after finalizing IPNS and DHT).

I recommend focusing hard on those 2 Objectives: a) large scale testing and b) finalize Interoperability through feature parity (word them as you see fit)

[vasco-santos]

We separated L12 and L13 into two different OKRs because we thought that it would be good to get both of us involved more heavily in this. And I would have the ones related to my on going work on the dht + pubsub. However, if you prefer to have a single one of us focusing on it, we can rearrange the assignments.

I completelly agree that those should be the most important objectives of this quarter, and we definetelly have to get them by the end of Q4.

[jacobheun]

I think it would be best to keep L12 and L13 separate. As they are massive efforts we're really going to need to pair on this, especially later in the quarter as dependencies are finished. My thoughts on the separation are that L12 is focused on base functionality and support of growth, whereas L13 is adding on features to the tests. Getting the core functionality done in L12 opens up L13 to start. L12 can then continue, with a focus on scaling to 50,000 nodes.

It would be great if we could take some time at team week to talk through this in more depth with the larger libp2p team and infra, to discuss what needs to happen for all of this to be completed by end of year. I'd love to have some clear milestones for the various teams before team week is over, to better set ourselves up to hit a testbed of that scale.

[daviddias]

@mgoelzer , it would be great if you could take on a KR to augment the team, namely: Onboard 3 new contributors to the js-libp2p team

[ghost]

@diasdavid Yes, happy to own that one.

[vasco-santos]

I will add that KR to this list @mgoelzer

[vasco-santos]

Reviewed the KR priorities @jacobheun

[vasco-santos]

I would consider this to be a P0, as it has several dependencies. What do you think @jacobheun ?

I agree with the other priorities.

[jacobheun]

Yes, I think you're right, as it's a blocker for some of the other P0s. I'll update.

[jacobheun]

Please try to provide any remaining feedback/questions/comments today. I will be moving this over to the Spreadsheet first thing tomorrow in preparation for our OKR meetings later this week.

[jacobheun]

All OKRs have been moved to the Q4 planning sheet. OKRs will be locked on October 4th, if you have any feedback before then, please feel free to comment here or on the spreadsheet.

[daviddias]

Let's make sure to give one last pass on the OKRs taking agoric-labs/PlaygroundVat#8 into account. There are multiple things there that are already part of the plan but there are also others that should be quick wins (Ed25519 keys and TCP Hole punching) that we can incorporate.

[jacobheun]

I've added a few line items to the sheet for discussion:

• the connection manager supports dial backoffs
• Add support for TCP Hole Punching
• Users can configure libp2p to use Ed25519 crypto keys instead of RSA

The biggest constraint right now is the availability @vasco-santos and I have. The crypto updates should be fairly quick, barring any surprises, so I think we could fit that in.
Hole Punching and connection manager updates will take a bit more effort, but are potentially accomplishable.

[vasco-santos]

Yes! I think the Ed25519 integration seems easy to achieve.
The other two are not straightforward for me and I am not sure of the time I would need for them.

[ghost]

The biggest constraint right now is the availability @vasco-santos and I have

One of my KRs is to onboard more JS engineers this quarter. So, help is on the way. If we feel stuff like TCP hole punching is beyond what we can manage this quarter, let's push it to the backlog (by opening issues and leaving them in Waffle's backlog). Worst thing we can do is just ignore or forget about Agoric's feedback, because it's really thoughtful and well-reasoned.

[jacobheun]

Just reposting this from line level comments for better visibility around completion assumptions.

As it stands, with our availability and the priority order, I think realistically the network visualization tool, nat traversal spec and error handling won't be completed. I think error handling will improve for code we'll be working in, but additional efforts to improve existing code will take a back seat. I think these are good for stretch goals though.