[docs] NLnet follow up questions (#846)

archive/nlnet/2022-next-generation-internet-zero.md

@@ -106,3 +106,47 @@ Finally, since the go-fed library project has become unmaintained, much of our e
 ## Attachments
 
 > Attachments: add any additional information about the project that may help us to gain more insight into the proposed effort, for instance a more detailed task description, a justification of costs or relevant endorsements. Attachments should only contain background information, please make sure that the proposal without attachments is self-contained and concise. Don't waste too much time on this. Really.
+
+## Follow-up Questions
+
+> You mention the difficulties of interoperating with various server and client softwares, and highlight that this also requires contributing to other projects’ implementations (e.g. of HTTP signatures). How much of your work do you estimate to be specific to developing GoToSocial, and how much to contributions to the broader ActivityPub ecosystem?
+
+I think the ratio so far has been about 90% GoToSocial, 10% looking through other projects, contributing code, and talking to other project developers. I think if we were to take over maintenance of go-fed, or fork the library (more on this below) this would probably change to something more like 70% GoToSocial, 30% broader ActivityPub contributions.
+
+> With the go-fed library you depend on being unmaintained (as you say), why do you plan to fork it rather than contributing to its maintenance? Have you discussed with other users like Owncast? Could you reflect on go-ap, which seems to be in a healthier state? Would it be possible to merge the functionality of these two, given some additional budget?
+
+When I started sketching out ideas for GoToSocial, one of the first steps was to look around for Go libraries that implemented ActivityPub, in order to avoid reinventing the wheel. At that time (~ January 2021), go-fed was the best option: well-documented, easily extensible, well-maintained, with an active Matrix channel, and contributed to (and used) by lots of people. By contrast, go-ap came across as more of a work-in-progress: it wasn't as well documented, didn't have any releases or tags (which is still the case), and didn't have as many contributors, nor was it used by as many people as go-fed.
+
+In the year and a half since then, this situation has changed somewhat. go-fed is no longer being actively maintained by its original developer (CJ), and the Matrix channel is no longer active either. By contrast, go-ap is still being actively developed, judging by the commit history on Github (https://github.com/go-ap/activitypub/commits/master) -- not to mention the fact that the Gitea project has chosen to use go-ap for its federation extensions.
+
+Nevertheless, even if GoToSocial was just started up today, I believe we would still pick go-fed over go-ap, mostly for the same reasons that motivated our initial choice: it's a more mature library with wider adoption that has been used to federate in production by several different successful projects, it has proper versioned releases and great docs, and by now it's had lots and lots of eyes on the code.
+
+Now that I have more experience with go-fed--and the ActivityPub ecosystem in general--I can also better understand why go-fed code is generated based on jsonld documents, which is something that confused me a bit at first: it makes writing custom extensions relatively easy, and you essentially get the proper jsonld schema document for your namespace for free, because you need to write it anyway in order to generate your Go code (see https://github.com/go-fed/activity/tree/master/astool). This is very useful for compatibility with ActivityPub implementations that require the jsonld namespace to be properly dereferenceable, like Friendica; if GoToSocial wants to add a gts namespace for custom extensions (more on this below), we can then host the jsonld document at the appropriate URL so that it can dereferenced by other AP implementations.
+
+That said, go-fed is not without its downsides, and go-ap has its advantages too. The main reason Gitea picked go-ap over go-fed is because of the extra overhead that go-fed adds to compiled binary sizes (something like 40MB if I remember rightly). This is less of a problem in go-ap, which doesn't add a lot of size to binaries, and can be considered a much lighter library in this regard. This is well explained here by Marius himself: https://lists.sr.ht/~mariusor/activitypub-go/%3Ca4bb4473-1379-453a-93aa-0c6aa8af49e4@dachary.org%3E#%3C20220512072502.mltep3wqjk2v5ste@slate%3E
+
+As for the reasoning behind forking go-fed rather than contributing to its maintenance, the short answer is that I haven't been able to get in touch with CJ for a long time, and the last time I submitted a PR to go-fed, it took a long time to get it reviewed + merged. I got the idea that CJ was unfortunately rather burned out on the project, so I didn't want to keep badgering him about it. Ultimately it doesn't really make much difference whether we fork or contribute to the existing project repo instead; it's more of a logistical issue than an ideological one. If we can reach CJ, and he's conducive to it, we'd happily work on the existing repo instead with his guidance!
+
+Re: Owncast -- Forking/maintaining go-fed is not something we've discussed with Owncast yet, though we are on good terms with Gabe. If we were to take over maintenance of go-fed, or indeed decide to proceed forwards with the fork instead, we would absolutely want to involve Owncast and other users like WriteFreely as contributors, to make sure that any changes we made wouldn't break their use-cases, and to incorporate any ideas they have about easing some of the existing pain points with go-fed. The goal is not to make go-fed hyper-specific to GoToSocial, but to keep it as a community library, in a healthy state, usable by the widest number of projects.
+
+Re: merging the functionality of go-fed and go-ap, given additional budget -- I believe this would take some more detailed investigation about exactly what the differences are between go-fed and go-ap in terms of code and ActivityPub API coverage, and what people find beneficial about each library compared to the other. My understanding is that while go-fed implements almost all of the ActivityPub vocabulary, go-ap focuses on a smaller subset of the vocabulary. I'm not sure at this point if additional budget for GoToSocial would be well spent trying to pull the two libraries together in one package, but I do think it might make for another interesting NLnet application in future.
+
+> Related to that, how much of the implementation work will be directly to go-fed (or its fork) itself, and (how) might this also help other activitypub-based server software?
+
+This is tricky to foresee, beyond the basic 70/30 estimate given in the answer above.
+
+In the short term, we would like to polish, gently extend, and fix bugs in the existing codebase, using the existing code generation paradigm. We also foresee some possible performance improvements to the library, which are relatively easy to implement (things like not parsing URLs back and forth quite so much). This sort of low-key, incremental stewardship of go-fed would already provide benefit to developers using the library.
+
+In the long run, some of the wishlist changes I've discussed with Kim (one of the other GoToSocial developers) would involve modifying the underlying code generation methods of go-fed, and using Go 1.18 generics in place of generating lots of structs; this might necessitate changes to some of the library's API, and would likely eventually lead to something more like a v2 of go-fed. This would provide a lot of benefit for AP developers using Go, since it would probably fix some of the binary size issues, and make understanding + maintaining go-fed code generation much easier. This amount of work, however, would probably require a full-time developer to work on go-fed for at least six months or so. This is beyond the scope of this NLnet funding application, but it would make for a very exciting application further down the line if Kim (or someone else) has time and energy for it!
+
+Given the scope of this current application, it's probably best to stick with the low-hanging fruit and make gentle incremental tweaks and improvements to go-fed either on the main repo or a fork, but we would love to see the library reworked in the future. Unfortunately, there are only so many hours in the day!
+
+> Regarding “Writing + maintaining our own extensions to the AP protocol”: what extensions would GoToSocial require? And how would this impact the interoperability with other software that has not implemented these extensions?
+
+The extensions required by GoToSocial primarily involve things like adding flags to ActivityPub Objects to indicate whether a post can be boosted, replied-to, or liked. Currently, setting such flags on posts is supported internally in GoToSocial, and it is enforced by accepting or refusing Activities coming in via the GtS federation API. However, this does not solve the problem of how to federate these preferences out to remote servers.
+
+In an issue on the Mastodon repository here -- https://github.com/mastodon/mastodon/issues/14762 -- Claire and trwnh propose a few different possibilities for how to handle no-reply posts over ActivityPub, using flags added to the posts (or the owner's profile) and federating those to other instances. This requires extending the AP spec to support these new flags. If GoToSocial were to implement this, in a GtS AP namespace, it would require us to extend the go-fed library to support + document the new types, and to host the schema for the namespace at something like eg https://gotosocial.org/ns#, similar to Mastodon's toot namespace at http://joinmastodon.org/ns# (unfortunately not online anymore), and the as namespace at https://www.w3.org/ns/activitystreams.
+
+Alternatively, it could be the case that Mastodon implements the extensions first in their toot namespace, in which case we will extend go-fed to implement these new extensions in the existing toot jsonld document.
+
+From an interoperability perspective, we would work to ensure that any extensions to AP on our part represent progressive enhancement rather than breaking changes that prevent different servers from talking with each other. That is, if a GoToSocial server with GtS specific extensions communicates with another server without the extensions, both servers should be able to process the interaction. To take the example above of non-replyable posts, if a GoToSocial server with these extensions sends a post to a server without these extensions, the GtS server will fall back to the existing best-effort behavior for preventing post replies, which is non-breaking. Conversely, if a GoToSocial server receives a post with extensions it cannot parse, it will simply read the message to the best of its ability using the basic as namespace, without dropping the message.