Support Azure OpenAI service

[LayZeeDK]

Please add support for Azure OpenAI service that now has General Availability (GA), including the GPT-3 model.

[zurawiki]

Thanks for the suggestion @LayZeeDK. I'm interested by your request. What's motivating it?

I certainly agree that this project should be generalized to support different LLM APIs in the future. If we want to implement Azure, I'd be curious to learn:

• Is there some feature Azure offers over OpenAI?
• Will the setup for the Azure API be any different, in terms of authentication or data format?

[LayZeeDK]

Azure OpenAI service supports enterprise-grade security, policies, compliance, and regional availability. Would you be interested in API access?

[LayZeeDK]

API reference for Azure OpenAI service:

• Completions and embeddings (Swagger)

[zurawiki]

Yup this is definitely something I'd be interested in making happen! I tagged the issue as help wanted

[LayZeeDK]

Completions

Request

Endpoint

In

gptcommit/src/openai.rs

Lines 66 to 71 in 6ee5bc5

	let request = self
	.client
	.post("https://api.openai.com/v1/completions")
	.header("Content-Type", "application/json")
	.header("Authorization", format!("Bearer {}", self.api_key))
	.json(&json_data);

Instead of https://api.openai.com/v1/completions, the endpoint URL would be in the format https://{resource_name}.openai.azure.com/openai/deployments/{deployment_id}/completions?api-version=<api_version> with 2022-12-01 being the latest API version so we must make the endpoint URL for completions configurable.

Headers

Instead of an Authorization header in the format Authorization: Bearer <api_key>, we would have the header api-key: <api_key>

Parameters

In the Azure OpenAI Service API, a completions request does not support a model parameter. Instead, this is represented by the {deployment_id} URL parameter.

Response

The OpenAI API response object has a usage property but the Azure OpenAI Service API does not.

The model property in the example response displays "ada". The models available for the Azure OpenAI Service are listed in Model Summary table and region availability. These are deployed then targeted in the {deployment_id} URL parameter for the completions endpoint. I'm not sure all the models supported by the Azure OpenAI Service are listed in

gptcommit/src/openai.rs

Lines 91 to 101 in 6ee5bc5

	pub(crate) fn get_prompt_token_limit_for_model(&self) -> usize {
	match self.model.as_str() {
	"text-davinci-003" => 4097,
	"text-curie-001" => 2048,
	"text-babbage-001" => 2048,
	"text-ada-001" => 2048,
	"code-davinci-002" => 8000,
	"code-cushman-001" => 2048,
	_ => 4097,
	}
	}

At least not if "ada" is a valid model response value.

[zurawiki]

Thanks for the context.

Currently, I'm blocked because I don't have access to the Azure OpenAI services. I applied but haven't heard back from Microsoft

[zurawiki]

Currently blocked on async-openai supporting Azure: 64bit/async-openai#32

[Laisky]

You can try this project that converts Azure to OpenAI API.

https://github.com/haibbo/cf-openai-azure-proxy/blob/main/README_en.md

then edit the gptchat's config file(~/.config/gptcommit/config.toml)

Note: api_key should be azure openai service's key, not openai's api key.