-
Notifications
You must be signed in to change notification settings - Fork 171
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add no_proxy env for embedding-tei-server container #267
Conversation
After discussion with @huiyan2021 rerank container has the same issue, add no proxy for both embedding and rerank |
Consider expanding this PR to cover all applicable examples. That may include at least:
|
@dcmiddle @huiyan2021 Hello, Dan, Huiyan: Got it. |
At least for containers who need to get results from others, like megaservices also. @lvliang-intel please help give advices. |
Thanks @huiyan2021 and @lvliang-intel , my proposal is to add "no_proxy" along with all http_proxy/https_proxy envs. I will wait for the feedback and then make relevant updates. |
Could you please also add no_proxy for other microservices? Thanks. |
@lvliang-intel , this PR got 2 approvals, open a new PR to handle it or directly update in this PR? |
@moting9 , please update directly in this PR, thanks. |
@lvliang-intel @Spycsh Hello, I updated the PR: add no_proxy in docker_compose. yaml for ChatQnA, CodeGen, CodeTrans, DocSum. Please review and comment. Thanks! |
Signed-off-by: Zhou Ting <ting.z.zhou@intel.com>
Description
Add no_proxy env for embedding-tei-server container
Issues
In a proxy r&d or deployment environment ( need proxy to access external network, no proxy to access internal network)
we need to pass no_proxy setting to embedding-tei-server, otherwise by following https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/docker/xeon/README.md
"TEI embedding service" access is OK, but inside "Embedding Microservice", it failed to access "Embedding Microservice".
some discussion in #251
TEI Embedding Service
curl ${host_ip}:6006/embed
-X POST
-d '{"inputs":"What is Deep Learning?"}'
-H 'Content-Type: application/json'
Embedding Microservice
curl http://${host_ip}:6000/v1/embeddings
-X POST
-d '{"text":"hello"}'
-H 'Content-Type: application/json'
Type of change
List the type of change like below. Please delete options that are not relevant.
Dependencies
[List the newly introduced 3rd party dependency if exists.]
Tests
Follow (https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/docker/xeon/README.md) to set up ChatQnA in a proxy environment
without this patch and opea-project/GenAIComps#140
TEI Embedding Service ok
curl ${host_ip}:6006/embed
-X POST
-d '{"inputs":"What is Deep Learning?"}'
-H 'Content-Type: application/json'
Embedding Microservice ko
curl http://${host_ip}:6000/v1/embeddings
-X POST
-d '{"text":"hello"}'
-H 'Content-Type: application/json'