Add elixir

[mvkvc]

No description provided.

[josevalim]

Maybe use ex?

Suggested change

	"elixir",
	"ex",

[mvkvc]

Makes sense

[mvkvc]

Currently for the results file I checked everything was timing out even though when I wrote one program to a file and ran locally it worked. Working on fixing that and renaming to ex from elixir.

[arjunguha]

Awesome! Thanks for writing this. I am happy to merge when you say its ready. Two comments:

1. Do you have a GPU available to run generations? (If not, we can help)
2. Would you edit the evaluator to use this as a template: https://github.com/nuprl/MultiPL-E/blob/main/evaluation/src/eval_lua.py it uses a different library that is a little more robust. (Uses Linux sessions.) We should really refactor the older generic_eval evaluators to use it.

[mvkvc]

Thank you for taking the time to review.

1. Yes, I ran it with Llama7B on a cloud GPU and the generation seems to work fine. Will upload the results here.
2. Sure not a problem. Currently having issues where everything seems to timeout even though when I write the program to a string and run myself it works. Hopefully this fixes that.

[mvkvc]

Here are the generations and results: results.zip. Happy with the generation for now and will rerun with the refactored eval on a faster machine.

[mvkvc]

Sorry misclick on the close.

[arjunguha]

Wonderful! Happy to merge. :)

[arjunguha]

@mvkvc I'm trying to put out a release with Elixir support. I am getting this error:

/usr/bin/elixir: 247: exec: erl: not found

This is how Elixir is installed:

https://github.com/nuprl/MultiPL-E/blob/main/evaluation/Dockerfile#L61

[jonatanklosko]

The following installs Erlang 26.2.3 and Elixir 1.16.2:

# Erlang
RUN wget https://binaries2.erlang-solutions.com/ubuntu/pool/contrib/e/esl-erlang/esl-erlang_26.2.3-1~ubuntu~jammy_amd64.deb -O /tmp/erlang.deb && dpkg -i /tmp/erlang.deb

# Elixir
RUN wget https://binaries2.erlang-solutions.com/ubuntu/pool/contrib/e/elixir/elixir_1.16.2_1_otp_26.2.3~ubuntu~jammy_all.deb -O /tmp/elixir.deb && dpkg -i /tmp/elixir.deb

[jonatanklosko]

Just for reference, here are instructions for installing the latest versions (Erlang 27.0 and Elixir 1.17.2):

# Install Erlang
RUN mkdir -p /erlang && \
  cd /erlang && \
  wget -nv -O erlang.tar.gz https://builds.hex.pm/builds/otp/ubuntu-22.04/OTP-27.0.tar.gz && \
  tar xzf erlang.tar.gz --strip-components=1 && \
  ./Install -minimal "$(pwd)"

# Erlang runtime dependencies, see https://github.com/hexpm/bob/blob/4fe43eb9853bb95dbfe276957bd7d3f931a451b3/priv/scripts/docker/erlang-ubuntu-jammy.dockerfile
RUN apt-get update && \
  apt-get -y --no-install-recommends install \
    ca-certificates \
    libodbc1 \
    libssl3 \
    libsctp1

# Install Elixir
RUN mkdir -p /elixir && \
  cd /elixir && \
  wget -nv -O elixir.zip https://builds.hex.pm/builds/elixir/v1.17.2-otp-27.zip && \
  unzip -q elixir.zip

ENV PATH="/erlang/bin:/elixir/bin:${PATH}"
ENV LANG=C.UTF-8

These are more verbose, but allow for installing latest versions, because the upstream builds are automated.

[josevalim]

I have deleted my comment because @jonatanklosko's recommendations are better. :)

[mvkvc]

Thank you @jonatanklosko @josevalim!