Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add no_proxy env for embedding-tei-server container #267

Merged
merged 1 commit into from
Jun 12, 2024

Conversation

moting9
Copy link
Contributor

@moting9 moting9 commented Jun 6, 2024

Description

Add no_proxy env for embedding-tei-server container

Issues

In a proxy r&d or deployment environment ( need proxy to access external network, no proxy to access internal network)
we need to pass no_proxy setting to embedding-tei-server, otherwise by following https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/docker/xeon/README.md

"TEI embedding service" access is OK, but inside "Embedding Microservice", it failed to access "Embedding Microservice".
some discussion in #251

TEI Embedding Service
curl ${host_ip}:6006/embed
-X POST
-d '{"inputs":"What is Deep Learning?"}'
-H 'Content-Type: application/json'
Embedding Microservice
curl http://${host_ip}:6000/v1/embeddings
-X POST
-d '{"text":"hello"}'
-H 'Content-Type: application/json'

Type of change

List the type of change like below. Please delete options that are not relevant.

  • [x ] Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)

Dependencies

[List the newly introduced 3rd party dependency if exists.]

Tests

Follow (https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/docker/xeon/README.md) to set up ChatQnA in a proxy environment

without this patch and opea-project/GenAIComps#140
TEI Embedding Service ok
curl ${host_ip}:6006/embed
-X POST
-d '{"inputs":"What is Deep Learning?"}'
-H 'Content-Type: application/json'
Embedding Microservice ko
curl http://${host_ip}:6000/v1/embeddings
-X POST
-d '{"text":"hello"}'
-H 'Content-Type: application/json'

@moting9
Copy link
Contributor Author

moting9 commented Jun 6, 2024

After discussion with @huiyan2021 rerank container has the same issue, add no proxy for both embedding and rerank

@dcmiddle
Copy link
Contributor

dcmiddle commented Jun 6, 2024

Consider expanding this PR to cover all applicable examples.

That may include at least:

  • ChatQnA
  • CodeGen
  • CodeTrans
  • DocSum

@moting9
Copy link
Contributor Author

moting9 commented Jun 7, 2024

@dcmiddle @huiyan2021 Hello, Dan, Huiyan:

Got it.
A question, Should I add no_proxy env for all containers along with http_proxy/https_proxy env, not limited to embedding and reranking contaienrs?

@huiyan2021
Copy link
Collaborator

At least for containers who need to get results from others, like megaservices also. @lvliang-intel please help give advices.

@moting9
Copy link
Contributor Author

moting9 commented Jun 7, 2024

Thanks @huiyan2021 and @lvliang-intel , my proposal is to add "no_proxy" along with all http_proxy/https_proxy envs. I will wait for the feedback and then make relevant updates.

@lvliang-intel
Copy link
Collaborator

@moting9,

Could you please also add no_proxy for other microservices? Thanks.

@moting9
Copy link
Contributor Author

moting9 commented Jun 12, 2024

@moting9,

Could you please also add no_proxy for other microservices? Thanks.

@lvliang-intel , this PR got 2 approvals, open a new PR to handle it or directly update in this PR?

@Spycsh
Copy link
Member

Spycsh commented Jun 12, 2024

@moting9 , please update directly in this PR, thanks.

@moting9
Copy link
Contributor Author

moting9 commented Jun 12, 2024

@lvliang-intel @Spycsh Hello, I updated the PR: add no_proxy in docker_compose. yaml for ChatQnA, CodeGen, CodeTrans, DocSum. Please review and comment. Thanks!

Signed-off-by: Zhou Ting <ting.z.zhou@intel.com>
@chensuyue chensuyue merged commit 2405879 into opea-project:main Jun 12, 2024
12 checks passed
Spycsh pushed a commit to Spycsh/GenAIExamples that referenced this pull request Jun 19, 2024
)

Signed-off-by: Zhou Ting <ting.z.zhou@intel.com>
yogeshmpandey pushed a commit to hteeyeoh/GenAIExamples that referenced this pull request Aug 12, 2024
)

Signed-off-by: Zhou Ting <ting.z.zhou@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants