Skip to content

Commit

Permalink
feat:qiniu
Browse files Browse the repository at this point in the history
  • Loading branch information
@deanxv
deanxv committed May 31, 2024
1 parent 33ce360 commit e249a05
Show file tree
Hide file tree
Showing 10 changed files with 304 additions and 24 deletions.
46 changes: 28 additions & 18 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,13 @@ _聚合阿里、百度等开放平台，提供与`openai`请求格式对齐的
## 功能

### 支持内容审核渠道

- [x] 阿里云
- [x] 百度智能云
- [ ] 七牛云
- [x] 七牛云

### 支持`openai`对齐的接口 (支持的接口会先内容审核再转发，不支持的接口会直接转发)

- [x] `v1/chat/completions` 用户输入的内容审查。
- [x] `v1/images/generations`用户输入的内容审查。

Expand Down Expand Up @@ -116,38 +118,46 @@ Render 可以直接部署 docker 镜像,不需要 fork 仓库：[Render](https:/
</div>
</details>


## 配置

### 环境变量

#### 通用

| 变量参数 | 变量描述 | 是否必填 |
|:------------------:|:------------------------------------------------:|:----:|
| AUDIT_CHANNEL_TYPE | 审核渠道类型[ali:阿里、baidu:百度] | Y |
| 变量参数 | 变量描述 | 是否必填 |
|:------------------:|:---------------------------------------------------:|:----:|
| AUDIT_CHANNEL_TYPE | 审核渠道类型[ali:阿里、baidu:百度] | Y |
| BASE_URL | 审核通过后的转发接口请求地址域名或IP:端口（例如: https://api.openai.com ） | Y |
| AUTHORIZATION | 鉴权密钥，与转发接口的API-Key保持一致，多个以`,`分隔 | Y |
| ENABLE | 审核启用开关[0:关闭、1:打开]（默认:1） | N |
| AUTHORIZATION | 鉴权密钥，与转发接口的API-Key保持一致，多个以`,`分隔 | Y |
| ENABLE | 审核启用开关[0:关闭、1:打开]（默认:1） | N |

#### 审核渠道-阿里 [阿里云-内容审核](https://vision.console.aliyun.com/cn-shanghai/detail/imageaudit)
#### 审核渠道-阿里云 [阿里云-内容审核](https://vision.console.aliyun.com/cn-shanghai/detail/imageaudit)

| 变量参数 | 变量描述 | 是否必填 |
|:------------------------:|:--------------------------------------------------------------------------------------------------:|:----:|
| ALI_ACCESS_KEY_ID | 阿里云开放平台AccessKeyId | Y |
| ALI_ACCESS_KEY_SECRET | 阿里云开放平台AccessKeySecret | Y |
| ALI_ENDPOINT | 阿里云开放平台Endpoint | Y |
| ALI_ACCESS_KEY_ID | 阿里云AccessKeyId | Y |
| ALI_ACCESS_KEY_SECRET | 阿里云AccessKeySecret | Y |
| ALI_ENDPOINT | 阿里云Endpoint | Y |
| ALI_LABEL | 内容审核类型[spam:垃圾、politics:敏感、abuse:辱骂、terrorism:暴恐、porn:鉴黄、flood:灌水、contraband:违禁、ad:广告] （多个以`,`分隔 ） | Y |
| ALI_AUDIT_CONTENT_LENGTH | 审核文本切割字节长度[默认:4000] | N |

#### 审核渠道-百度 [百度智能云-内容审核平台](https://ai.baidu.com/censoring#/strategylist)
#### 审核渠道-百度智能云 [百度智能云-内容审核平台](https://ai.baidu.com/censoring#/strategylist)

| 变量参数 | 变量描述 | 是否必填 |
|:--------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:----:|
| BAIDU_API_KEY | 百度智能云APIKey | Y |
| BAIDU_SECRET_KEY | 百度智能云SecretKey | Y |
| BAIDU_LABEL | 内容审核类型[default:默认违禁词库、politics:政治敏感、abuse:低俗辱骂、terrorism:暴恐违禁、porn:文本色情、flood:低质灌水、ad:恶意推广、black:自定义黑名单]（多个以`,`分隔 ） | Y |
| BAIDU_AUDIT_CONTENT_LENGTH | 审核文本切割字节长度（默认:4000） | N |

#### 审核渠道-七牛云 [七牛云-内容审核](https://portal.qiniu.com/censor/main/overview)

| 变量参数 | 变量描述 | 是否必填 |
|:--------------------------:|:--------------------------------------------------------------------------------------------------------------------------------:|:----:|
| BAIDU_API_KEY | 百度开放平台APIKey | Y |
| BAIDU_SECRET_KEY | 百度开放平台SecretKey | Y |
| BAIDU_LABEL | 内容审核类型[default:默认违禁词库、politics:政治敏感、abuse:低俗辱骂、terrorism:暴恐违禁、porn:文本色情、flood:低质灌水、ad:恶意推广、black:自定义黑名单、white:自定义白名单]（多个以`,`分隔 ） | Y |
| BAIDU_AUDIT_CONTENT_LENGTH | 审核文本切割字节长度（默认:4000） | N |
| 变量参数 | 变量描述 | 是否必填 |
|:--------------------------:|:--------------------------------------------------------------------------------------------------------------------:|:----:|
| QINIU_ACCESS_KEY | 七牛云APIKey | Y |
| QINIU_SECRET_KEY | 七牛云SecretKey | Y |
| QINIU_LABEL | 内容审核类型[spam:含垃圾信息、politics:涉政、abuse:辱骂、terrorism:暴恐、porn:色情、flood:灌水、ad:广告、contraband:违禁、meaningless:无意义]（多个以`,`分隔 ） | Y |
| QINIU_AUDIT_CONTENT_LENGTH | 审核文本切割字节长度（默认:4000） | N |



Expand Down
12 changes: 11 additions & 1 deletion check/check.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,16 @@ func CheckEnvVariable() {
if config.AliLabel == "" {
logger.FatalLog("环境变量 ALI_LABEL 未设置")
}
} else if strings.ToLower(config.AuditChannelType) == "qiniu" {
if config.QiNiuAccessKey == "" {
logger.FatalLog("环境变量 QINIU_ACCESS_KEY 未设置")
}
if config.QiNiuSecretKey == "" {
logger.FatalLog("环境变量 QINIU_SECRET_KEY 未设置")
}
if config.QiNiuLabel == "" {
logger.FatalLog("环境变量 QINIU_LABEL 未设置")
}
} else if strings.ToLower(config.AuditChannelType) == "baidu" {
if config.BaiduApiKey == "" {
logger.FatalLog("环境变量 BAIDU_API_KEY 未设置")
Expand All @@ -43,7 +53,7 @@ func CheckEnvVariable() {
logger.FatalLog("环境变量 BASE_URL 未设置")
}

if config.Authorization == "" {
if config.ApiKey == "" {
logger.FatalLog("环境变量 AUTHORIZATION 未设置")
}

Expand Down
22 changes: 20 additions & 2 deletions common/config/config.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ import (
var AuditChannelType = os.Getenv("AUDIT_CHANNEL_TYPE")
var Enable = env.Int("ENABLE", 1)
var BaseUrl = os.Getenv("BASE_URL")
var Authorization = os.Getenv("AUTHORIZATION")
var Authorizations = strings.Split(os.Getenv("AUTHORIZATION"), ",")
var ApiKey = os.Getenv("API_KEY")
var ApiKeys = strings.Split(os.Getenv("API_KEY"), ",")

/*
spam：文字垃圾内容识别
Expand Down Expand Up @@ -48,6 +48,24 @@ var BaiduSecretKey = os.Getenv("BAIDU_SECRET_KEY")
var BaiduLabel = os.Getenv("BAIDU_LABEL")
var BaiduAuditContentLength = env.Int("BAIDU_AUDIT_CONTENT_LENGTH", 4000)

/*
normal：正常文本
spam：含垃圾信息
ad：广告
politics：涉政
terrorism：暴恐
abuse：辱骂
porn：色情
flood：灌水
contraband：违禁
meaningless：无意义
*/

var QiNiuAccessKey = os.Getenv("QINIU_ACCESS_KEY")
var QiNiuSecretKey = os.Getenv("QINIU_SECRET_KEY")
var QiNiuLabel = os.Getenv("QINIU_LABEL")
var QiNiuAuditContentLength = env.Int("QINIU_AUDIT_CONTENT_LENGTH", 4000)

var DebugEnabled = strings.ToLower(os.Getenv("DEBUG")) == "true"

var SessionSecret = uuid.New().String()
Expand Down
2 changes: 1 addition & 1 deletion common/constants.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ package common
import "time"

var StartTime = time.Now().Unix() // unit: second
var Version = "v0.0.0" // this hard coding will be replaced automatically when building, no need to manually change
var Version = "v0.1.0" // this hard coding will be replaced automatically when building, no need to manually change
24 changes: 24 additions & 0 deletions controller/chat.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,18 @@ func ChatForOpenAI(c *gin.Context) {
})
return
}
} else if strings.ToLower(config.AuditChannelType) == "qiniu" {
response, err = utils.QiNiuAudit(request)
if err != nil {
c.JSON(http.StatusInternalServerError, model.OpenAIErrorResponse{
OpenAIError: model.OpenAIError{
Message: "Unknown audit channel",
Type: "request_error",
Code: "AUDIT_CHANNEL_ERROR",
},
})
return
}
} else {
c.JSON(http.StatusInternalServerError, model.OpenAIErrorResponse{
OpenAIError: model.OpenAIError{
Expand Down Expand Up @@ -174,6 +186,18 @@ func ImagesForOpenAI(c *gin.Context) {
})
return
}
} else if strings.ToLower(config.AuditChannelType) == "qiniu" {
response, err = utils.QiNiuAudit(request)
if err != nil {
c.JSON(http.StatusInternalServerError, model.OpenAIErrorResponse{
OpenAIError: model.OpenAIError{
Message: "Unknown audit channel",
Type: "request_error",
Code: "AUDIT_CHANNEL_ERROR",
},
})
return
}
} else {
c.JSON(http.StatusInternalServerError, model.OpenAIErrorResponse{
OpenAIError: model.OpenAIError{
Expand Down
7 changes: 6 additions & 1 deletion go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,13 @@ require (
github.com/gin-contrib/sessions v1.0.1
github.com/gin-gonic/gin v1.10.0
github.com/google/uuid v1.6.0
github.com/qiniu/go-sdk/v7 v7.21.0
github.com/samber/lo v1.39.0
)

require (
github.com/BurntSushi/toml v1.3.2 // indirect
github.com/alex-ant/gomath v0.0.0-20160516115720-89013a210a82 // indirect
github.com/alibabacloud-go/alibabacloud-gateway-spi v0.0.4 // indirect
github.com/alibabacloud-go/darabonba-number v1.0.4 // indirect
github.com/alibabacloud-go/debug v1.0.0 // indirect
Expand All @@ -39,13 +42,14 @@ require (
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.20.0 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/gofrs/flock v0.8.1 // indirect
github.com/gorilla/context v1.1.2 // indirect
github.com/gorilla/securecookie v1.1.2 // indirect
github.com/gorilla/sessions v1.2.2 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.2.7 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/matishsiao/goInfo v0.0.0-20210923090445-da2e3fa8d45f // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
Expand All @@ -57,6 +61,7 @@ require (
golang.org/x/crypto v0.23.0 // indirect
golang.org/x/exp v0.0.0-20220303212507-bbda1eaf7a17 // indirect
golang.org/x/net v0.25.0 // indirect
golang.org/x/sync v0.6.0 // indirect
golang.org/x/sys v0.20.0 // indirect
golang.org/x/text v0.15.0 // indirect
golang.org/x/time v0.0.0-20200416051211-89c76fbcd5d1 // indirect
Expand Down
Loading

0 comments on commit e249a05

Please sign in to comment.