Can't get langchain to work behind a corporate proxy.

[corrreia]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

none

Description

I'm deploying Langchain behind a corporate proxy and I can't get langchain to send all requests to the proxy. I'm using AzureOpenAI and already tried many of the supposed solutions, none of them worked.

Is there any way to make this the right way?

System Info

RHEL 8
Docker
Node version 20

[dosubot[bot]]

Based on the context provided, it seems you're trying to route all requests from LangChain JS through a corporate proxy. This can be achieved by using the httpAgent or httpsAgent property available in the OpenAICoreRequestOptions interface. This property allows you to specify custom agent settings for the HTTP or HTTPS requests, which can include proxy configuration.

Here's an example of how you can configure LangChain JS to use a corporate proxy when making requests to AzureOpenAI:

const { HttpsProxyAgent } = require('https-proxy-agent');
const { AzureOpenAI } = require('langchain-openai'); // Assuming this is how you import AzureOpenAI

// Configure your corporate proxy settings
const proxyUrl = 'http://your-corporate-proxy:port';
const proxyAgent = new HttpsProxyAgent(proxyUrl);

// Create AzureOpenAI instance with proxy configuration
const azureOpenAI = new AzureOpenAI({
  azureOpenAIApiKey: 'your-azure-openai-api-key',
  azureOpenAIApiInstanceName: 'your-instance-name',
  azureOpenAIApiDeploymentName: 'your-deployment-name',
  options: {
    httpAgent: proxyAgent, // Use httpAgent or httpsAgent based on your proxy protocol
  },
});

// Now you can use azureOpenAI instance to make requests that will be routed through the corporate proxy

In this example, the https-proxy-agent package is used to create an HTTPS proxy agent with the corporate proxy settings. This agent is then passed to the httpAgent property of the OpenAICoreRequestOptions when creating an instance of AzureOpenAI. You should replace the proxyUrl variable with your actual corporate proxy URL and port.

Also, remember to replace placeholders like 'your-azure-openai-api-key', 'your-instance-name', and 'your-deployment-name' with your actual Azure OpenAI configuration values.

This solution is based on the structure of the OpenAICoreRequestOptions interface in the LangChain JS codebase, which is as follows:

export type OpenAICoreRequestOptions<
  Req extends object = Record<string, unknown>
> = {
  path?: string;
  query?: Req | undefined;
  body?: Req | undefined;
  headers?: Record<string, string | null | undefined> | undefined;

  maxRetries?: number;
  stream?: boolean | undefined;
  timeout?: number;
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  httpAgent?: any;
  signal?: AbortSignal | undefined | null;
  idempotencyKey?: string;
};

You can find more details about this interface in the LangChain JS codebase.

Please try this solution and let me know if it works for you or if you need further assistance.

Sources

• Attempt to connect to Azure openai through @langchain/azure-openai failed
• ChatOpenAI This class does not provide proxy attributes
• Tunnel over proxy in Nodejs
• How do I set axios forward proxy?
• Streaming does not work if run behind a proxy server
• libs/langchain-openai/src/types.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[corrreia]

@dosu-bot options is not a property of AzureChatOpenAI

[dosubot[bot]]

Based on the context provided, it seems that the AzureOpenAI class in LangChain JS does not directly support proxy configuration. However, there are a few workarounds that you can consider.

One approach is to use the global-agent package to bootstrap a global HTTP/HTTPS proxy agent. Here's how you can do it:

1. Install the global-agent package:

npm install global-agent

2. Bootstrap the global-agent at the very beginning of your application:

require('global-agent/bootstrap');

3. Set the GLOBAL_AGENT_HTTP_PROXY environment variable to your corporate proxy URL before starting your application. This can be done in your .env file or directly in your environment:

GLOBAL_AGENT_HTTP_PROXY=http://your-corporate-proxy:port

4. Ensure your application reads the .env file using dotenv or a similar package if you're setting the proxy URL in the .env file:

require('dotenv').config();

This setup will route all HTTP/HTTPS requests made by your Node.js application, including those to AzureOpenAI, through the specified proxy.

Please note that this approach affects all HTTP/HTTPS requests globally within your application. If you need a more targeted approach (e.g., only proxying requests to AzureOpenAI), you might need to look into creating a custom HTTP/HTTPS agent specifically for your AzureOpenAI requests or explore other libraries that offer more granular proxy configurations.

Another approach is to set up a reverse proxy. You can modify the basePath in the ConfigurationParameters for openai when creating a new ChatOpenAI instance. This allows you to route all requests to the OpenAI APIs through your proxy server. Here's an example:

const chat = new ChatOpenAI(
	{ temperature: 0, openAIApiKey: env.OPENAI_KEY },
	{ basePath: 'https://proxy.your-server.com/v1' }
);

try {
  const response = await chat.call([
    new HumanChatMessage('Translate this sentence from English to French. I love programming.')
  ]);
  console.log(response);
} catch (e) {
  console.log(e);
}

In this example, https://proxy.your-server.com/v1 is the URL of your forward proxy server. You should replace it with the URL of your own server. This solution was suggested in a similar issue: How to set proxy for openai.

Please note that this solution assumes that your proxy server is already set up to forward requests to the OpenAI APIs. If you need help setting up a forward proxy server, I recommend referring to the documentation of the proxy server software you are using.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

• ChatOpenAI This class does not provide proxy attributes
• How to configure a forward proxy
• How to set proxy for openai
• examples/src/indexes/vector_stores/azure_aisearch/.env.example
• libs/langchain-azure-openai/src/tests/llms.int.test.ts
• libs/langchain-azure-openai/src/chat_models.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[tyadav7]

Has anyone found a solution to this? @corrreia