How do I access GPT4all as a local server?

[ngkkv]

I've checked the box to enable local server.

I've tried pinging it in cmd.exe 127.0.0.1:4891
or my ipv4 address:4891 but no no avail.

There are responses to both ip address if I removed the ":4891"

[mmarquezs]

Just so you are aware, there is no such thing as pinging a port. You can check if a port is open with telnet and other ways but it's not a ping. If you meant this second thing, ok. But if you did like "ping 127.0.0.1:4891" it's normal that it didn't work.

That said away, I have same issue where enabling that check, doesn't do anything, the API is not available in that port, no response.
Also it seems with the new page/documentation some documents about this feature are gone.

[cosmic-snow]

The API is there, but it's only available on localhost. See my reply below.

Regarding documentation. Yes, some things were reworked/are being reorganised. Old pages are still in the repository and on the website, but not linked.

[ngkkv]

Thanks for your explanation. I learnt something new today. How did you check if the API was available in that port?

…

On Thu, Jul 18, 2024, 15:15 Marc Marquez Santamaria < ***@***.***> wrote: Just so you are aware, there is no such thing as pinging a port. You can check if a port is open with telnet and other ways but it's not a ping. If you meant this second thing, ok. But if you did like "ping 127.0.0.1:4891" it's normal that it didn't work. That said away, I have same issue where enabling that check, doesn't do anything, the API is not available in that port, no response. Also it seems with the new page/documentation some documents about this feature are gone. — Reply to this email directly, view it on GitHub <#2689 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BJXAS6JTXNJ4Z7TBOVNJGCTZM5TRXAVCNFSM6AAAAABLB32522VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMBYGEYDANA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[mmarquezs]

I realised under the server chat, I cannot select a model in the dropdown unlike "New Chat". is that why I could not access the API?

That is normal, the model you select it when doing a request using the API, and then in that section of server chat it will show the conversations you did using the API, it's a little buggy tough in my case it only shows the replies by the api but not what I asked.

[ngkkv]

I would like to see how an end user would experience with GPT4All on the server, how do they do that? I'd thought hitting the ip address and the port would load GPT4all on the browser but nothing is showing.

[mmarquezs]

It works for me too, thanks!

how do I get someone on the local network to chat with GPT4all?

Regards, Victor

I don't think the app supports it, as far as I am aware, but I am also new to it.

The easiest way I guess, maybe not, if the app doesn't support it, would be to run some app that acts as http reverse proxy, so any request to it is send to 127.0.0.1. Just be carefull with it as the API doesn't have any security aka api key or similar, I guess that's why the default is only localhost/127.0.0.1 exposed.

I had this problem already trying to do requests from WSL2 which didn't work because the wsl2 vms are sort of in separate networks unless you activate certain configs.

[mmarquezs]

I would like to see how an end user would experience with GPT4All on the server, how do they do that? I'd thought hitting the ip address and the port would load GPT4all on the browser but nothing is showing.

Oh no no... you misunderstood what this is. This is just an API that emulates the API of ChatGPT, so if you have a third party tool (not this app) that works with OpenAI ChatGPT API and has a way to provide it the URL of the API, you can replace the original ChatGPT url with this one and setup the specific model and it will work without the tool having to be adapted to work with GPT4All.

[ngkkv]

thanks, unfortunately this does not fit my use casel.
I hope they work on in the later versions.

[cosmic-snow]

In a nutshell:

• The GPT4All chat application's API mimics an OpenAI API response. The implementation is limited, however.
• It's only available through http and only on localhost aka 127.0.0.1 on the machine that runs the chat application.
• If you want to see the conversation in the GUI, scroll the chat drawer to the bottom after you started talking to it (that is, through API requests. This is not a graphical web client).

To troubleshoot, the easiest way to see it responding is GET on http://localhost:4891/v1/models. As it's a GET request, you can just do that in a browser.

Currently, the only paths exposed are:

• GET to /v1/models
• GET to /v1/models/<name>
• POST to /v1/completions
• POST to /v1/chat/completions

[ngkkv]

Is my understanding correct, that when it connects to the OpenAI API, it's no longer a local server as it's sending information to the LLM at OpenAI?

[cosmic-snow]

I'm not entirely sure what your question is. There is an option to enable the OpenAI API as a model. In that case it of course will be sending the chat to OpenAI.

If you're only using the local web API server with a local model (and have data sharing disabled), then nothing will be sent anywhere.

[expositions]

Thanks @cosmic-snow for the clarification above.

Just sharing for others who might turn up here, wondering how to use the command line interface to request completions:

Using curl http://localhost:4891/v1/models in the terminal successfully returned the list of models indeed.

Using a model from the list for the POST request also worked out:

curl http://localhost:4891/v1/completions \
     -H "Content-Type: application/json" \
     -d '{
           "model": "Llama 3.1 8B Instruct 128k",
           "prompt": "Hello, how are you?",
           "max_tokens": 50
         }'

[zhouruicn]

The local server port is bound to 127.0.0.1:4891 instead of 0.0.0.0:4891, so it cannot be accessed from other machines. However, you can make it accessible from other machines by using port forwarding.

[ngkkv]

I would like to colleagues to chat with GPT4all based on our company's SOPs.

[cosmic-snow]

The local server only listens to requests from the current machine. You would have to either:

• Remove that restriction in the source and build it yourself.
• Work around that with for example a reverse proxy or as mentioned port forwarding. I haven't tried that myself, however.

[ngkkv]

I am on my current machine, should I not be expecting the browser to open the GPT4all-like chat app when I typed http://localhost:4891/v1/models?

[cosmic-snow]

No. That is only an easy way to see if you can access the server. It serves an API, it's not a chat client.

[ngkkv]

thanks, you have provided valuable knowledge to my learning. :)