Getting stuck at "Setting up the language model..." / Windows / local model

[Tom1827]

Describe the bug

On Windows, trying to get a local model to start, I'm getting stuck every time at the "Setting up the language model..." stage.

I've tried several times, by removing open-interpreter, ooba, and the ooba folder in C:\Users<user>\AppData\Local\ooba and then reinstalling and redownloading Mistral 7B. It has got stuck each time at a different amount of progress. The attempt shown in the screenshot has now been running for about two hours.

Reproduce

• Install per the instructions
• Run interpreter (i.e. for the first time)
• Press enter to use local model
• Wait for the model to download

...then it grinds to a halt at some point as it is setting up the language model

Expected behavior

To not get stuck, and finish setting up the language model? :)

Screenshots

Open Interpreter version

0.1.10

Python version

3.10.2

Operating System name and version

Windows 11 Pro

Additional context

Ryzen 7 5800X, 32GB RAM, NVidia 3060Ti 8GB

[bmwhite1611]

Yeah I waited like half a day and then closed the window. Now I run interpreter --local and it just says "Getting Started..." forever. I wish I could have gone back to Llama, but they turned it off for some reason, or else I just can't figure out how to go back to a larger model like that... should never have uninstalled it...

[sebvannistel]

yea same issue here. they still have not fixed it. been waiting for over 2 weeks for a fix #636

[thisdp]

It scans ports from 2000 to 10000 at the speed of 1 port pre second using single thread, we need at least 8000 secs. And under llm.py -> class llm -> init function, there are at least 3 scans....
This sucks:

while True:
 self.port = random.randint(2000, 10000)
 open_ports = get_open_ports(2000, 10000)
 if self.port not in open_ports:
  break

[thisdp]

And even there's even a bug in path management.
llm.py line 45, under windows, there should be "\" not "/"

[Notnaton]

Hi, @thisdp while the PRs are being merged, you can try my fork that fixes Ooba for windows. It fixes both the paths and port scanning, and some other stuff.
https://github.com/Notnaton/ooba
pip install git+https://github.com/Notnaton/ooba.git@windows --force-reinstall

[ericrallen]

Hey there, folks!

Now that LM Studio is the underlying local model provider, I think we can close this one.

Please reopen it if you still experience the issue in the latest version of Open Interpreter.