Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Portable Windows Version #1907

Open
RoyiAvital opened this issue Sep 7, 2022 · 28 comments
Open

Portable Windows Version #1907

RoyiAvital opened this issue Sep 7, 2022 · 28 comments
Labels
what::documentation where::macOS macOS-specific issues where::windows Windows-specific issues

Comments

@RoyiAvital
Copy link

Though documentation says that the deployment model of micromamba is based on prefix and everything contained within it some things doesn't work like that.

For instance:

  1. On installation of Notebook / Jupyter micromamba creates a start menu shortcut called Jupyter Notebook (Anaconda).
  2. Somehow the initialization of the shell (shell init) changed the PATH variable of the computer globally and not only for the specific session.

It would be great to have a portable mode which means everything is located in the same folder and there is a script (PS / CMD) to configure the session for work.

There is also an issue that when switching between environments repeatedly suddenly cause micromamba to say the shell isn't initialized and activate can not be used. It requires opening a new CMD window to continue to work.

@jonashaag
Copy link
Contributor

I’m not sure where your start menu entry is coming from but it’s not from Micromamba.

You can use Micromamba without shell integration which should make it “portable”.

For your last issue, if you reproduce this can you please give instructions how to reproduce?

@RoyiAvital
Copy link
Author

Could you elaborate on:

You can use Micromamba without shell integration which should make it “portable”.

What's the minimum things needed to be defined as variable for micromamba to work on cmd and powershell?
I thought one must run the activate batch file. It would be great to clarify this more on documentation.

Regarding the issue, I don't do anything special, just switch between environments.

@jonashaag
Copy link
Contributor

Can you send a video/screenshots/text output or what you do to reproduce the problem and what’s the error message?

Re: micromamaba without integration, you can just always use micromamba.exe -n yourenv instead of activate (and micromamba -n yourenv run to run any programs in the environment, eg Python).

@RoyiAvital
Copy link
Author

Hi, I will try to replicate and send something.

But I want to focus on the portable mode.

What does micromamba requires to be defined as env variables in order to work?
When I say work it means to use a certain folder as its base (Where it installs the environments).

What does exactly the shell init phase do?

@RoyiAvital
Copy link
Author

@jonashaag, Any chance to have the information?
I'd be happy to understand and then put it in the Wiki of the project (Or documentation).

@jonashaag
Copy link
Contributor

This is my initialize block for fish:

# >>> mamba initialize >>>
# !! Contents within this block are managed by 'mamba init' !!
set -gx MAMBA_EXE "/opt/homebrew/bin/micromamba"
set -gx MAMBA_ROOT_PREFIX "/Users/j/micromamba"
eval "/opt/homebrew/bin/micromamba" shell hook --shell fish --prefix "/Users/j/micromamba" | source
# <<< mamba initialize <<<

Have a look what micromamba shell hook does do

@pavelzw
Copy link
Member

pavelzw commented Sep 21, 2022

micromamba shell init does multiple things which you can read in the source code:

  1. It initializes the root_prefix, it creates a file <root_prefix>/etc/profile.d/micromamba.sh (Linux + macOS) which contains the necessary shell hooks. On PowerShell, it creates two files, <root_prefix>/condabin/mamba_hook.ps1 as well as <root_prefix>/condabin/Mamba.psm1. For cmd.exe, it creates some other files too.
  2. It modifies the rcfiles. It appends a mamba initialize block to your rc file on macOS + Linux. For PowerShell, it creates a file in Documents/PowerShell/Profile1.ps1 (something like that at least, I didn't check the exact name) which acts like the rc file for PowerShell. For cmd.exe, it adds a string to the registry autostart that says to execute the micromamba.bat from 1. (you can find it in Software\Microsoft\Command Processor in the Windows registry)
  3. On Windows, it asks to enable long path support.

@pavelzw
Copy link
Member

pavelzw commented Sep 21, 2022

Only for macOS/Linux

For me, micromamba already suggests what to do when I want to use it in a portable manner:

root@7085ba60af4b:/# ~/.local/bin/micromamba create -n testenv

                                           __
          __  ______ ___  ____ _____ ___  / /_  ____ _
         / / / / __ `__ \/ __ `/ __ `__ \/ __ \/ __ `/
        / /_/ / / / / / / /_/ / / / / / / /_/ / /_/ /
       / .___/_/ /_/ /_/\__,_/_/ /_/ /_/_.___/\__,_/
      /_/

warning  libmamba 'root_prefix' set with default value: /root/micromamba
Empty environment created at prefix: /root/micromamba/envs/testenv
root@7085ba60af4b:/# ~/.local/bin/micromamba activate testenv

'micromamba' is running as a subprocess and can't modify the parent shell.
Thus you must initialize your shell before using activate and deactivate.

    To initialize the current bash shell, run:
        $ eval "$(micromamba shell hook --shell=bash)"
    and then activate or deactivate with:
        $ micromamba activate

To automatically initialize all future (bash) shells, run:
    $ micromamba shell init --shell=bash --prefix=~/micromamba

Supported shells are {bash, zsh, xonsh, cmd.exe, powershell, fish}.

critical libmamba Shell not initialized

Like Jonas said, executing eval "$(micromamba shell hook --shell=bash)" before you run your micromamba commands should work, at least on Linux/macOS.

@pavelzw
Copy link
Member

pavelzw commented Sep 21, 2022

Btw if you want to revert the changes by micromamba shell init, you can execute micromamba shell deinit. This should revert all changes except for the long part support thing.

@RoyiAvital
Copy link
Author

OK, Is there a way to skip step (2)?
I want to create a portable distribution so it can't be based on a profile file on the system path?

@RoyiAvital
Copy link
Author

@pavelzw , In the process you described, what's the difference between shell init and shell hook?
Is it just the difference between doing step 2 or not?

@jonashaag
Copy link
Contributor

jonashaag commented Dec 10, 2022

You can think of shell init as something that makes shell hook permanent by adding it to the rcfile. I think shell hook does none of the steps 1–3.

@RoyiAvital
Copy link
Author

The problem is that now I created a cmd file to have a portable environment which runs:

set MAMBA_ROOT_PREFIX=%~dp0
set PATH=MAMBA_ROOT_PREFIX;%PATH%

cd /d %~dp0
micromamba.exe shell hook -s cmd.exe -p %MAMBA_ROOT_PREFIX% -v

I thought this will give a portable environment.
Yet when I run micromamba activate I get:

'micromamba' is running as a subprocess and can't modify the parent shell.
Thus you must initialize your shell before using activate and deactivate.

To automatically initialize all future (cmd.exe) shells, run:
    $ micromamba shell init --shell=cmd.exe --prefix=~/micromamba

Supported shells are {bash, zsh, csh, xonsh, cmd.exe, powershell, fish}.

critical libmamba Shell not initialized

The idea is I don't want to do init, as I don't want anything to be configured outside of MAMBA_ROOT_PREFIX.
How can I achieve this?

@jonashaag
Copy link
Contributor

Not sure about cmd.exe but for other shells this message appears when micromamba refers to an exe file and not a shell command (which the hook code creates). You need to execute/eval the output of shell hook.

@RoyiAvital
Copy link
Author

RoyiAvital commented Dec 10, 2022

OK, I think I found how to create a portable launcher:

:: Mamba Settings
set MAMBA_ROOT_PREFIX=%~dp0

cd /d %~dp0

if exist "%MAMBA_ROOT_PREFIX%\Scripts\" (
:: The initialization has happened once -> Go to activation
  goto start
)

micromamba.exe shell hook -s cmd.exe -p %MAMBA_ROOT_PREFIX% -v

:start
:: Shell hook was created, just activate the environment
cd /d %~dp0Scripts
activate
cd /d %~dp0

What's needed in order to make it work is using the activate script.
One note, which should be documented, in this case one can not use micromamba.exe activate <EnvName>. The exe must not appear. It should be micromamba activate <EnvName>.

I'm happy as now I have a completely portable Python based on Conda (MicroMamba) on Windows.

@stonebig, This might be the basis for MicroMamba based WinPython.

@jonashaag
Copy link
Contributor

@RoyiAvital how is this different to the "normal" cmd.exe activation? I have never used cmd.exe activation so I really don't know. I'm just wondering if we could modify the normal activation to be more portable.

@RoyiAvital
Copy link
Author

@jonashaag , I might be missing something, but what do you mean by the "normal" activation?
When you do the init you use the registry according to #1907 (comment). So it is not portable by definition.

If you mean that micromamba.exe shell hook -s cmd.exe -p %MAMBA_ROOT_PREFIX% -v should do the activation, then I didn't see it.

@jonashaag
Copy link
Contributor

I see, then how do you use your launcher?

@RoyiAvital
Copy link
Author

RoyiAvital commented Dec 12, 2022

OK, My launcher looks like that:

:: Mamba Settings
set MAMBA_ROOT_PREFIX=%~dp0

:: Go to the `%MAMBA_ROOT_PREFIX%` folder
cd /d %~dp0

:: Assuming `micromamba.exe` is in `%MAMBA_ROOT_PREFIX%`
if not exist "micromamba.exe" goto :FAIL

:: TODO: Check this!
if exist "%MAMBA_ROOT_PREFIX%\Scripts\" (
:: The initialization has happened once -> Go to activation
  goto :ACTIVATE
)

micromamba.exe shell hook -s cmd.exe -p %MAMBA_ROOT_PREFIX% -v

:ACTIVATE
:: Shell hook was created, just activate the environment
:: call Scripts\activate.bat
:: The `activate.bat` script from the folder where `micromamba.exe` resides.  
:: It will generate the error: 'micromamba' is running as a sub process and can't modify the parent shell`.
:: Thus one must initialize the shell before using `activate` and `deactivate`.
cd /d %~dp0Scripts
call activate.bat
:: Go back to the root folder
cd /d %~dp0
goto :DONE

:DONE
ECHO Finished setting Conda (MicroMamba) environment!
goto :eof

:FAIL
ECHO Could not find micromamba.exe, failed setting the environment!
goto :eof

Basically you can just double click it.
I also add a simple profile to Windows Terminal:

"commandline": "%SystemRoot%\\System32\\cmd.exe /k <PathToLauncher>MambaPython.cmd",

So I have an environment ready in my Windows Terminal.

I have issue with .conda / .ipython / .matplotlib which I don't know how to set the path manually.
They behave like on Linux while the strength in Windows, in my opinion, is the easiness to create portable environments.
So, in my actual script, which is extreme, I also set APPDATA, HOMEPATH and USERPROFILE to sub folders within MAMBA_ROOT_PREFIX. Then I have a real portable environment that works the same on any computer I use (For instance, I can put it in my OneDrive folder and have it synced on any machine I own).

@geocine
Copy link

geocine commented Jan 7, 2023

This is what I did on my Windows terminal avoiding creating a batch file

{
    "commandline": "%windir%\\System32\\cmd.exe \"/K\"set MAMBA_ROOT_PREFIX=.\\root & root\\condabin\\micromamba.bat activate myenv",
    "guid": "{123e4567-e89b-12d3-a456-426655440023}",
    "icon": "%USERPROFILE%\\miniconda3\\Menu\\Iconleak-Atrous-Console.ico",
    "name": "Micromamba"
}

@RoyiAvital
Copy link
Author

@jonashaag, Anyway to make conda aliased as micromamba? Could you add this to the activate script?

@jonashaag
Copy link
Contributor

The command line interfaces are incompatible so I don't recommend that

@RoyiAvital
Copy link
Author

OK, so at least micromamba -> mamba.

@jonashaag
Copy link
Contributor

Those are incompatible as well.

@RoyiAvital
Copy link
Author

When I run micromamba.exe shell hook -s cmd.exe -p <PathToMambaPrefix> -v I get:

info libmamba No AutoRun key detected.
Setting cmd.exe AUTORUN to: "\condabin\mamba_hook.bat"
Windows long-path support enabled.

How does it set cmd.exe to autorun the bat script?
I looked at the AutoRun registry key (See https://superuser.com/questions/144347) yet it is not set.
Yet when I open new cmd session indeed it works. How do you do that?

@RoyiAvital
Copy link
Author

According to what I found at #2712 (comment):

  1. AutoRun is set at Computer\HKEY_CURRENT_USER\SOFTWARE\Microsoft\Command Processor.
  2. It seems to use \condabin\mamba_hook.bat so it might be that my script should be updated with that.

@JohanMabille
Copy link
Member

JohanMabille commented Dec 5, 2024

This has been refactored in mamba 2. Also notice that mamba 2 has no python code anymore, it is a dynamically linked version of micromamba, meaning it has exactly the same commands and options. You can therefore alias micromamba to mamba. Can you check that mamba 2 works as expected for you?

@RoyiAvital
Copy link
Author

I am using micromamba with version 2.0.4 and it seems to work well.

@jjerphan jjerphan added the where::macOS macOS-specific issues label Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
what::documentation where::macOS macOS-specific issues where::windows Windows-specific issues
Projects
None yet
Development

No branches or pull requests

6 participants