Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elixir storage backend for libgit2. #2

Closed
redrabbit opened this issue Nov 6, 2017 · 2 comments
Closed

Elixir storage backend for libgit2. #2

redrabbit opened this issue Nov 6, 2017 · 2 comments
Labels

Comments

@redrabbit
Copy link
Owner

redrabbit commented Nov 6, 2017

The libgit2 libraries provide a mechanism to plugin our own backend for storing Git data.

A very interesting way to investigate would be to have the libgit2 backend communicate with our application (by exchanging messages) by implementing an Erlang port driver.

I'm not sure at all that this is in any way performant. We will have to do a lot of benchmarks here!

@redrabbit redrabbit changed the title Storage backend for *libgit2*. Storage backend for libgit2. Nov 14, 2017
@redrabbit redrabbit changed the title Storage backend for libgit2. Elixir storage backend for libgit2 Dec 1, 2017
@redrabbit redrabbit changed the title Elixir storage backend for libgit2 Elixir storage backend for libgit2. Dec 1, 2017
redrabbit added a commit that referenced this issue Apr 16, 2019
redrabbit added a commit that referenced this issue Apr 16, 2019
See #2 for more details.
@redrabbit
Copy link
Owner Author

We got our first experimental PostgreSQL backend. Switching backends is easy as setting :git_storage config to :postgres and voilà.

Loading the PostgreSQL backend is provided by GitRekt.Git.repository_load/1:

{:ok, repo} = GitRekt.Git.repository_load({:postgres, repo_id, "postgresql://localhost/my_db"})

Git objects and references are stored in Postgres database configured for GitGud.DB. The two tables for storing Git data are called git_objects and git_references.

I could not notice any great performance improvements so far.

A direction I would like to go in future iterations is storing Git objects in separated SQL tables depending on their type (commit, tag, tree, blob, etc.). This way we might be able to optimise slow commands such as counting commits or querying history for a specific pathspec.

@redrabbit
Copy link
Owner Author

redrabbit commented Apr 16, 2019

Closing this because implementing a loopback (Elixir -> NIF -> Elixir) seems not feasible currently.

A while ago, I've stumbled across this presentation: Future Extensions to the Native Interface.
The presentation is from 2011 and nothing so far seems to be available in the Erlang ecosystem.

redrabbit added a commit that referenced this issue Apr 24, 2019
See #2 for more details.
redrabbit added a commit that referenced this issue Sep 13, 2021
When calling GitRekt.GitAgent from other nodes, NIF resources might be
garbage collected between two calls:

    {:ok, head} = GitRekt.GitAgent.head(agent)
    {:ok, commit} = GitRekt.GitAgent.peel(agent, head) #1
    {:ok, message} = GitRekt.GitAgent.commit_message(agent, message) #2

In the above example, between #1 and #2 the agent process might garbage
collect the commit. This could lead lead to a bad argument error in #2.

In order to counter this, GitRekt.GiAgent stores a set of borrowed NIF
resources.  This way the resources are reference counted and available
between consecutive calls. Note that the references are dropped once a
:DOWN message is received from a given client process.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant