TODOs

[PedroRegisPOAR]

NYLUG Presents: Sneaking in Nix - Building Production Containers with Nix +
Optimising Docker Layers for Better Caching with Nix
+
Developing Python with Poetry & Poetry2nix: Reproducible flexible Python environments

• test all this packages https://pythonwheels.com/ and https://github.com/meshy/pythonwheels and https://github.com/hugovk/top-pypi-packages
• replicate this using nix + flakes https://github.com/tfc/nix_cmake_example
• Show some real world Wheel stats meshy/pythonwheels#23
• document it in some place Poesia pythonista com Poetry
• https://stackoverflow.com/questions/59390658/not-able-to-install-pendulum-on-python3-7-alpine-docker-image
• https://github.com/wlyra/covid19
• https://github.com/taizan-hokuto/pytchat and https://towardsdatascience.com/how-to-scrape-youtube-comments-with-python-61ff197115d4

Use the same poetry version by all team members. Apparently poetry does not version... poetry ;
From: https://stackoverflow.com/a/65627679

Replicate this Pillow is missing libX11 on WSL

About musl + position independent executables PIE: Small, Simple, and Secure: Alpine Linux under the Microscope

About ruby and gems lock: Long Shipit! Presents: How Shopify Uses Nix, short Shipit! Presents: How Shopify Uses Nix

Breaking changes in system dependencies. Packages built with Nix depend only on other packages managed by Nix, not on system libraries, so our packages are now independent of the host OS; we can deploy the same package to Ubuntu 18.04 and 20.04.
https://rzetterberg.github.io/kubernetes-nixos.html

poetry add -vvv git+https://github.com/matemax/test_poetry_submodule.git
python-poetry/poetry#1828

Manage dependencies of git submodules with poetry

Anaconda was messing the PATH environment variable: How do I deal with certificates using cURL while trying to access an HTTPS url?

Make as example:

• https://stackoverflow.com/questions/1614059/how-to-make-python-speak
• https://realpython.com/python-speech-recognition/
• https://www.geeksforgeeks.org/convert-text-speech-python/
• https://pypi.org/project/pytchat/

TODO: document it https://stackoverflow.com/a/58092909

preferWheel = true;
From: nix-community/poetry2nix#42 (comment)

export LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib/:$LD_LIBRARY_PATH

From: nix-community/poetry2nix#46

Or you may uncompress the archive and use setup.py directly with either pip or python
TODO: build an OCI image with python and try make it install a unpacked .whl
From: https://stackoverflow.com/a/36014474, other user with the same mindset python-poetry/poetry#3514 (comment)

TODO: https://discourse.nixos.org/t/preparing-a-nix-flake-for-a-python-program-migra-using-poetry/12030/9

TODO:

• PEP 615 https://www.python.org/dev/peps/pep-0615/, tzdata, from: https://github.com/docker-library/python/blob/2ce7a368bcc861fdf1cd7920a16b1845ffee8b4f/3.9/alpine3.14/Dockerfile#L21-L22
• https://github.com/docker-library/python/blob/2ce7a368bcc861fdf1cd7920a16b1845ffee8b4f/3.9/buster/slim/Dockerfile#L21-L22

Why netbase is only in the debian image? How is the python (especially the python3Minimal) from nixpkgs dealing with this tzdata dependency?

https://search.nixos.org/packages?channel=21.05&show=tzdata&from=0&size=50&sort=relevance&query=tzdata

Init poetry project

Bloated:

nix \
profile \
install \
nixpkgs#auditwheel \
nixpkgs#binutils.out \
nixpkgs#glibc.bin \
nixpkgs#git \
nixpkgs#patchelf \
nixpkgs#poetry \
nixpkgs#python3Full \
nixpkgs#python3Packages.wheel \
nixpkgs#python3Packages.wheel-filename \
nixpkgs#python3Packages.wheel-inspect \
nixpkgs#twine

python --version \
&& mkdir -p "$HOME"/test \
&& cd "$HOME"/test


# Should run only once?
poetry config virtualenvs.create true
poetry config virtualenvs.in-project true
poetry config virtualenvs.path .

poetry init --no-interaction \
&& poetry lock \
&& poetry show --tree \
&& poetry add flask \
&& poetry lock \
&& poetry show --tree 

cat <<WRAP > "$HOME"/test/flake.nix
{
  description = "A usefull description";

  inputs.nixpkgs.url = "github:NixOS/nixpkgs";
  inputs.flake-utils.url = "github:numtide/flake-utils";
  inputs.poetry2nix-src.url = "github:nix-community/poetry2nix";

  outputs = { self, nixpkgs, flake-utils, poetry2nix-src }:
    flake-utils.lib.eachDefaultSystem (system:
    let
        pkgsAllowUnfree = import nixpkgs {
          system = "x86_64-linux";
          config = { allowUnfree = true; };
        };

        pkgs = import nixpkgs { inherit system; overlays = [ poetry2nix-src.overlay ]; };

        config = {
          projectDir = ./.;
        };
    in
    {

      devShell = pkgsAllowUnfree.mkShell {

        buildInputs = with pkgsAllowUnfree; [
                       poetry
                       (pkgsAllowUnfree.poetry2nix.mkPoetryEnv config)
                     ];

          shellHook = ''
            unset SOURCE_DATE_EPOCH
          '';
        };
  });
}
WRAP

nix \
flake \
lock \
--override-input nixpkgs github:NixOS/nixpkgs/ea4c80b39be4c09702b0cb3b42eab59e2ba4f24b


nix \
flake \
lock \
--override-input nixpkgs github:NixOS/nixpkgs/12bdeb01ff9e2d3917e6a44037ed7df6e6c3df9d
# nix flake lock


git config --global user.email "you@example.com"
git config --global user.name "Your Name"


git init \
&& git add . \
&& git commit -m 'First nix flake commit'

nix develop --command python -c 'import flask'

nix shell nixpkgs#bashInteractive
nix profile install nixpkgs#coreutils
nix profile install nixpkgs#poetry
nix profile install nixpkgs#python3

python --version \
&& mkdir -p "$HOME"/test \
&& cd "$HOME"/test \
&& poetry init --no-interaction --dev-dependency numpy \
&& poetry lock \
&& poetry show --tree \
&& poetry add pillow \
&& poetry lock

cat <<WRAP > "$HOME"/test/flake.nix
{
  description = "A usefull description";

  inputs.nixpkgs.url = "github:NixOS/nixpkgs";
  inputs.flake-utils.url = "github:numtide/flake-utils";
  inputs.poetry2nix-src.url = "github:nix-community/poetry2nix";

  outputs = { self, nixpkgs, flake-utils, poetry2nix-src }:
    flake-utils.lib.eachDefaultSystem (system:
    let
        pkgsAllowUnfree = import nixpkgs {
          system = "x86_64-linux";
          config = { allowUnfree = true; };
        };

        pkgs = import nixpkgs { inherit system; overlays = [ poetry2nix-src.overlay ]; };

        config = {
          projectDir = ./.;
        };
    in
    {

      devShell = pkgsAllowUnfree.mkShell {

        buildInputs = with pkgsAllowUnfree; [
                       poetry
                       (pkgsAllowUnfree.poetry2nix.mkPoetryEnv config)
                     ];

          shellHook = ''
            unset SOURCE_DATE_EPOCH
          '';
        };
  });
}
WRAP

nix flake lock
nix develop --command python -c 'import numpy'

https://github.com/numpy/numpy/blob/76930e7d0c22e227c9ff9249a90a6254c5a6b547/doc/HOWTO_RELEASE.rst.txt#make-sure-current-branch-builds-a-package-correctly

TODO:

• https://peps.python.org/pep-0730/
• https://stackoverflow.com/questions/66849112/how-do-i-cross-compile-a-rust-application-from-macos-x86-to-macos-silicon
• https://stackoverflow.com/questions/77847389/cross-compiling-in-rust-requires-both-host-and-target-architecture-dependencies

[PedroRegisPOAR]

TODO:

• Python says it cannot find SetupTools NixOS/nixpkgs#25984 (comment)
• https://www.reddit.com/r/NixOS/comments/i8ct8o/developing_python_with_poetry_poetry2nix/
• https://www.reddit.com/r/NixOS/comments/p6nc8x/python_projects_nightmare/

nix \
profile \
install \
nixpkgs#python3Full \
nixpkgs#python3Full.pkgs.pip \
nixpkgs#python3Full.pkgs.virtualenv \
nixpkgs#python3Full.pkgs.setuptools \
nixpkgs#python3Full.pkgs.wheel \
nixpkgs#python3Full.pkgs.venvShellHook \
&& nix store gc

nix \
shell \
--impure \
--expr \
'
  (
    with builtins.getFlake "github:NixOS/nixpkgs/d639b2dfacdb3464faa11936a8c751ea3ff57775";
    with legacyPackages.${builtins.currentSystem};
      python3.withPackages (p: with p; [ 
        pip
        virtualenv
        setuptools
        wheel
        venvShellHook
        numpy 
      ])
  )
' \
--command \
python3 -c 'import numpy as np; np.show_config(); print(np.__version__)'

nix \
shell \
nixpkgs#{python37,python38,python39,python310}

python3.7 --version
python3.8 --version
python3.9 --version
python3.10 --version

[PedroRegisPOAR]

TODO: how to test it?
platform_machine targetMachine

A test for manylinux1:
Refs.:

• https://github.com/digital-asset/daml/tree/05e691f55852fbb207f0e43cf23bb95b95866ba3/dev-env/tests/manylinux1
• Add support for PEP 513 "manylinux1" wheels NixOS/nixpkgs#71935 (comment)

_manylinux.py has been removed
Refs.:

• python: remove _manylinux.py NixOS/nixpkgs#75763

nix \
shell \
--impure \
--expr \
'(
  with builtins.getFlake "github:NixOS/nixpkgs/d639b2dfacdb3464faa11936a8c751ea3ff57775";
  with legacyPackages.${builtins.currentSystem};
  python3.withPackages (p: with p; [ numpy ])
)' \
--command \
python3 -c 'import numpy as np; np.show_config(); print(np.__version__)'

Refs.:

• https://www.reddit.com/r/NixOS/comments/p6nc8x/python_projects_nightmare/

{ nix \
shell \
--impure \
--expr \
'(
  with builtins.getFlake "github:NixOS/nixpkgs/d639b2dfacdb3464faa11936a8c751ea3ff57775";
  with legacyPackages.${builtins.currentSystem};
  python3.withPackages (p: with p; [ packaging ])
)' \
--command \
python \
-c \
"from packaging import tags; print('\n'.join([str(t) for t in tags.sys_tags()]))" } | curl -F 'f:1=<-' ix.io

Refs.:

• https://packaging.pypa.io/en/latest/index.html
• BUG: Error installing NumPy via Pip on macOS Big Sur with python from HomeBrew numpy/numpy#17784 (comment)

TODO: read all this

• https://discourse.nixos.org/t/mach-nix-create-python-environments-quick-and-easy/6858/76