Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bare compiler should not require a hex registry #958

Open
ericmj opened this issue Dec 3, 2015 · 19 comments
Open

Bare compiler should not require a hex registry #958

ericmj opened this issue Dec 3, 2015 · 19 comments
Labels
enhancement new behaviour or additional functionality must reproduce TODO item for maintainers to confirm a bug or behaviour

Comments

@ericmj
Copy link
Contributor

ericmj commented Dec 3, 2015

It complains until I run rebar3 update and then works.

@ferd
Copy link
Collaborator

ferd commented Dec 3, 2015

Would there be a problem to having the compiler fetch its own registry? The problem is possibly (as you pointed out on IRC) with plugins required in order to properly compile. In such cases, unless the user of the bare compiler were to fetch plugins and compile them on rebar3's behalf then somehow passed in the proper path, I don't think we could avoid the registry being there.

@ericmj
Copy link
Contributor Author

ericmj commented Dec 3, 2015

If the compiler automatically fetches the registry or if mix can pass the path of its own registry to rebar works for me.

@ericmj
Copy link
Contributor Author

ericmj commented Dec 3, 2015

Another option would be for mix to treat plugins as normal dependencies so we fetch. In this case they should be available when rebar runs and fetching the registry would not be necessary for rebar. I will try this out and let you know how it works out.

@gleber
Copy link
Contributor

gleber commented Dec 15, 2015

This also is relevant to making rebar3 useful for building stuff in hermetic environment (e.g. Debian effort to make packages reproducible and Nix/NixOS build processes). Currently I need to work around this by manually providing registry file, as seen here:
https://github.com/NixOS/nixpkgs/pull/11651/files#diff-177ea02978ffc72f964033695c5e023eR9

This registry is not really needed, since all dependencies of built project are already provided locally and, I believe, rebar has enough information from rebar.config to validate that deps are the same as requested.

@ericmj
Copy link
Contributor Author

ericmj commented Jan 4, 2016

Ping?

@tsloughter
Copy link
Collaborator

Pong. not really sure what to do here.

@gleber
Copy link
Contributor

gleber commented Feb 1, 2016

Currently rebar3 combines two things in one:

  1. compiler wrapper (traversing packages based on config file and building them correctly)
  2. dependency manager (e.g. downloading, locking, etc.)

I believe Eric means to not do any of (2) if everything mentioned in config files is available locally to be compiled.

@ericmj
Copy link
Contributor Author

ericmj commented Feb 1, 2016

With the bare compiler rebar assumes that the user have provided all dependencies and that they are correct so rebar should not need the registry to proceed.

@tsloughter
Copy link
Collaborator

Ah. hm. I suppose doing plugin install in app_discovery is wrong anyway and could be moved to a new provider. I'll look at doing that today.

But plugins then won't be installed or used which isn't right either, and I don' tknow how to resolve that.

@ericmj
Copy link
Contributor Author

ericmj commented Feb 1, 2016

If possible the way I would like plugins to be handled would be that I make sure the plugins rebar need are available in the code path the same way I do for dependencies. Would that make sense?

@tsloughter
Copy link
Collaborator

The problem there is rebar3 treats plugins differently from deps so it doesn't accept a dep as a plugin. As in, if you have a dep A at vsn X you may have a plugin that also has dep A but at vsn Y. So plugins are installed to _build/<profile>/plugins and not _build/<profile>/lib.

@tsloughter
Copy link
Collaborator

That is unless you pass a plugins_dir in the config that could set it to lib/.

@ericmj
Copy link
Contributor Author

ericmj commented Feb 1, 2016

If rebar handles plugins separately from deps there is no reason for mix to handle them because as far as I can tell they will never conflict with mix deps. So we can close this issue for my part.

Could we tell rebar where the registry is located so we don't have to make multiple requests to fetch it? Keep in mind that mix may call rebar multiple times to compile dependencies.

@tsloughter
Copy link
Collaborator

We don't use the original registry for package look up :(. It gets converted to a packages.idx ets table on disk. But still possible if you are ok with it writing packages.idx to the directory with the registry.

@ericmj
Copy link
Contributor Author

ericmj commented Feb 1, 2016

@tsloughter That would be perfectly fine.

@ferd
Copy link
Collaborator

ferd commented Jun 7, 2016

@ericmj is this still required? Trying to filter issues.

@ericmj
Copy link
Contributor Author

ericmj commented Jun 8, 2016

It's not implemented yet afaik but it's also not a hard requirement. It
works right now, this feature would just make builds faster.

On Tuesday, 7 June 2016, Fred Hebert notifications@github.com wrote:

@ericmj https://github.com/ericmj is this still required? Trying to
filter issues.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#958 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AATV2od4RHsWt8vbpAp56q_hUXWDnFmYks5qJNi7gaJpZM4GuCua
.

Eric Meadows-Jönsson

@ferd ferd added the enhancement new behaviour or additional functionality label Aug 27, 2016
gleber added a commit to gleber/rebar3 that referenced this issue Jun 19, 2017
Similarly to PATH env variable, this allows to pass paths to bare
compiler which do not fit nicely into a single wildcard structure.
Colon (":") is used as separator.

This provides more flexibility when rebar is run in offline/hermetic
environment, e.g. erlang#958 and erlang#1281.
gleber added a commit to gleber/rebar3 that referenced this issue Jun 19, 2017
Similarly to PATH env variable, this allows to pass paths to bare
compiler which do not fit nicely into a single wildcard structure.
Colon (":") is used as separator.

This provides more flexibility when rebar is run in offline/hermetic
environment, e.g. erlang#958 and erlang#1281.
gleber added a commit to gleber/rebar3 that referenced this issue Jun 20, 2017
Similarly to PATH env variable, this allows to pass paths to bare
compiler which do not fit nicely into a single wildcard structure.
Colon (":") is used as separator.

This provides more flexibility when rebar is run in offline/hermetic
environment, e.g. erlang#958 and erlang#1281.
@ferd ferd added the must reproduce TODO item for maintainers to confirm a bug or behaviour label Jun 8, 2019
@tsloughter
Copy link
Collaborator

Is this still the case?

@ferd
Copy link
Collaborator

ferd commented Mar 8, 2021

I think there was a way to do it with only the local registry, at least since we shipped the v2 of the hex API, since we now work from a partial index rather than a complete one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement new behaviour or additional functionality must reproduce TODO item for maintainers to confirm a bug or behaviour
Projects
None yet
Development

No branches or pull requests

4 participants