Skip to content

Commit

Permalink
Merge branch 'dev-2.0.0-rc' into dev-2.0.0-sshe-nn
Browse files Browse the repository at this point in the history
# Conflicts:
#	configs/default.yaml
#	python/fate/arch/federation/backends/standalone/_federation.py
  • Loading branch information
talkingwallace committed Dec 14, 2023
2 parents 2380613 + 85e3d5d commit fd6ce1f
Show file tree
Hide file tree
Showing 301 changed files with 6,459 additions and 3,284 deletions.
113 changes: 66 additions & 47 deletions configs/default.yaml
Original file line number Diff line number Diff line change
@@ -1,47 +1,66 @@
communicator:
verbose: False
debug:
debug_mode: False
validation_mode: False
encoder:
precision_bits: 24
functions:
max_method: "log_reduction"

# exponential function
exp_iterations: 8

# reciprocal configuration
reciprocal_method: "NR"
reciprocal_nr_iters: 10
reciprocal_log_iters: 1
reciprocal_all_pos: False
reciprocal_initial: null

# sqrt configuration
sqrt_nr_iters: 3
sqrt_nr_initial: null

# sigmoid / tanh configuration
sigmoid_tanh_method: "reciprocal"
sigmoid_tanh_terms: 32

# log configuration
log_iterations: 2
log_exp_iterations: 8
log_order: 8

# trigonometry configuration
trig_iterations: 10

# error function configuration:
erf_iterations: 8
mpc:
active_security: False
provider: "TFP"
protocol: "beaver"
nn:
dpsmpc:
protocol: "layer_estimation"
skip_loss_forward: True
cache_pred_size: True
safety:
serdes:
# supported types: unrestricted, restricted, restricted_catch_miss
restricted_type: "unrestricted"

phe:
paillier:
allow: True
minimum_key_size: 1024

ou:
allow: True
minimum_key_size: 1024

mock:
allow: False

psi:
ecdh:
allow: True
curve_type:
- curve25519

mpc:
active_security: False
provider: "TFP"
protocol: "beaver"
functions:
max_method: "log_reduction"

# exponential function
exp_iterations: 8

# reciprocal configuration
reciprocal_method: "NR"
reciprocal_nr_iters: 10
reciprocal_log_iters: 1
reciprocal_all_pos: False
reciprocal_initial: null

# sqrt configuration
sqrt_nr_iters: 3
sqrt_nr_initial: null

# sigmoid / tanh configuration
sigmoid_tanh_method: "reciprocal"
sigmoid_tanh_terms: 32

# log configuration
log_iterations: 2
log_exp_iterations: 8
log_order: 8

# trigonometry configuration
trig_iterations: 10

# error function configuration:
erf_iterations: 8

communicator:
verbose: False
debug:
debug_mode: False
validation_mode: False
encoder:
precision_bits: 16
1 change: 1 addition & 0 deletions configs/whitelist.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
fate: "*"
4 changes: 2 additions & 2 deletions doc/2.0/fate/ml/hetero_secureboost_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -71,8 +71,8 @@
"\n",
"def create_ctx(local, context_name):\n",
" from fate.arch import Context\n",
" from fate.arch.computing.standalone import CSession\n",
" from fate.arch.federation.standalone import StandaloneFederation\n",
" from fate.arch.computing.backends.standalone import CSession\n",
" from fate.arch.federation.backends.standalone import StandaloneFederation\n",
" import logging\n",
"\n",
" # prepare log\n",
Expand Down
10 changes: 5 additions & 5 deletions doc/2.0/fate/ml/homo_nn_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -351,8 +351,8 @@
"\n",
"def create_ctx(local):\n",
" from fate.arch import Context\n",
" from fate.arch.computing.standalone import CSession\n",
" from fate.arch.federation.standalone import StandaloneFederation\n",
" from fate.arch.computing.backends.standalone import CSession\n",
" from fate.arch.federation.backends.standalone import StandaloneFederation\n",
" import logging\n",
"\n",
" logger = logging.getLogger()\n",
Expand Down Expand Up @@ -450,7 +450,7 @@
"Guest Terminal Outputs:\n",
"\n",
"```\n",
"2023-09-13 20:36:13,319 - fate.arch.federation.standalone._federation - DEBUG - [federation.standalone]init federation: standalone_session=<fate.arch.computing.standalone._csession.CSession object at 0x7f90881e3a60>, federation_session_id=2023-09-13-20-36, party=('guest', 10000)\n",
"2023-09-13 20:36:13,319 - fate.arch.federation.standalone._federation - DEBUG - [federation.standalone]init federation: standalone_session=<fate.arch.computing.backends.standalone._csession.CSession object at 0x7f90881e3a60>, federation_session_id=2023-09-13-20-36, party=('guest', 10000)\n",
"2023-09-13 20:36:13,319 - fate.arch.federation.standalone._federation - DEBUG - [federation.standalone]init federation context done\n",
"2023-09-13 20:36:13,400 - fate.ml.nn.algo.homo.fedavg - INFO - Using secure_aggregate aggregator\n",
"2023-09-13 20:36:13,400 - fate.arch._standalone - DEBUG - [federation.standalone.remote.agg_type.default]remote data, type=<class 'bytes'>\n",
Expand Down Expand Up @@ -537,7 +537,7 @@
"Host Terminal Outputs:\n",
"\n",
"```\n",
"2023-09-13 20:36:12,803 - fate.arch.federation.standalone._federation - DEBUG - [federation.standalone]init federation: standalone_session=<fate.arch.computing.standalone._csession.CSession object at 0x7fcd12dc2ee0>, federation_session_id=2023-09-13-20-36, party=('host', 9999)\n",
"2023-09-13 20:36:12,803 - fate.arch.federation.standalone._federation - DEBUG - [federation.standalone]init federation: standalone_session=<fate.arch.computing.backends.standalone._csession.CSession object at 0x7fcd12dc2ee0>, federation_session_id=2023-09-13-20-36, party=('host', 9999)\n",
"2023-09-13 20:36:12,803 - fate.arch.federation.standalone._federation - DEBUG - [federation.standalone]init federation context done\n",
"2023-09-13 20:36:12,888 - fate.ml.nn.algo.homo.fedavg - INFO - Using secure_aggregate aggregator\n",
"2023-09-13 20:36:12,888 - fate.arch._standalone - DEBUG - [federation.standalone.remote.agg_type.default]remote data, type=<class 'bytes'>\n",
Expand Down Expand Up @@ -614,7 +614,7 @@
"Arbiter Terminal Outputs:\n",
"\n",
"```\n",
"2023-09-13 20:36:12,315 - fate.arch.federation.standalone._federation - DEBUG - [federation.standalone]init federation: standalone_session=<fate.arch.computing.standalone._csession.CSession object at 0x7fb2f65c1dc0>, federation_session_id=2023-09-13-20-36, party=('arbiter', 10000)\n",
"2023-09-13 20:36:12,315 - fate.arch.federation.standalone._federation - DEBUG - [federation.standalone]init federation: standalone_session=<fate.arch.computing.backends.standalone._csession.CSession object at 0x7fb2f65c1dc0>, federation_session_id=2023-09-13-20-36, party=('arbiter', 10000)\n",
"2023-09-13 20:36:12,316 - fate.arch.federation.standalone._federation - DEBUG - [federation.standalone]init federation context done\n",
"2023-09-13 20:36:12,316 - fate.arch._standalone - DEBUG - [federation.standalone.get.agg_type.default]\n",
"2023-09-13 20:36:13,418 - fate.arch._standalone - DEBUG - [GET] Got 2023-09-13-20-36-agg_type-default-guest-10000-arbiter-10000 type Object\n",
Expand Down
53 changes: 53 additions & 0 deletions doc/2.0/osx/bfia-x.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/*
* Copyright 2023 The BFIA Authors. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
syntax = "proto3";

package org.ppc.ptp;

// PTP Private transfer protocol
// 通用报头名称编码,4层无Header以二进制填充到报头,7层以Header传输
// x-ptp-version required 协议版本
// x-ptp-tech-provider-code required 厂商编码
// x-ptp-trace-id required 链路追踪ID
// x-ptp-token required 认证令牌
// x-ptp-source-node-id required 发送端节点编号
// x-ptp-target-node-id required 接收端节点编号
// x-ptp-source-inst-id required 发送端机构编号
// x-ptp-target-inst-id required 接收端机构编号
// x-ptp-session-id required 通信会话号,全网唯一

// 通信传输层输入报文编码
message Inbound {
map<string, string> metadata = 1; // 报头,可选,预留扩展,Dict,序列化协议由通信层统一实现
bytes payload = 2; // 报文,上层通信内容承载,序列化协议由上层基于SPI可插拔
}

// 通信传输层输出报文编码
message Outbound {
map<string, string> metadata = 1; // 报头,可选,预留扩展,Dict,序列化协议由通信层统一实现
bytes payload = 2; // 报文,上层通信内容承载,序列化协议由上层基于SPI可插拔
string code = 3; // 状态码
string message = 4; // 状态说明
}

// 互联互通如果使用异步传输协议作为标准参考,Header会复用metadata传输互联互通协议报头,且metadata中会传输异步场景下的消息相关属性
// 互联互通如果使用其他协议作为参考标准,Header会复用metadata传输互联互通协议报头
// 互联互通如果使用GRPC作为参考标准,Header会复用HTTP2的报头传输互联互通协议报头

service PrivateTransferProtocol {
rpc transport (stream Inbound) returns (stream Outbound);
rpc invoke (Inbound) returns (Outbound);
}
61 changes: 61 additions & 0 deletions doc/2.0/osx/bfia-y.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
/*
* Copyright 2023 The BFIA Authors. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
syntax = "proto3";

package org.ppc.ptp;

// PTP Private transfer protocol
// 通用报头名称编码,4层无Header以二进制填充到报头,7层以Header传输
// x-ptp-tech-provider-code: required 厂商编码
// x-ptp-trace-id: required 链路追踪ID
// x-ptp-token required 认证令牌
// x-ptp-session-id required 通信会话号,全网唯一
// x-ptp-target-node-id required 接收端节点编号,全网唯一
// x-ptp-target-inst-id optional 接收端机构编号,全网唯一

message PeekInbound {
string topic = 1; // optional 会话主题,相同信道具有唯一性,用于同一信道的传输隔离
}

message PopInbound {
string topic = 1; // optional 会话主题,相同信道具有唯一性,用于同一信道的传输隔离
int32 timeout = 2; // optional 阻塞超时时间,默认120s
}

message PushInbound{
string topic = 1; // optional 会话主题,相同信道具有唯一性,用于同一信道的传输隔离
bytes payload = 2; // 二进制报文
map<string, string> metadata = 3; // optional 保留参数,用于扩展性
}

message ReleaseInbound {
string topic = 1; // optional 会话主题,相同信道具有唯一性,用于同一信道的传输隔离
int32 timeout = 2; // optional 阻塞超时时间,默认120s
}

message TransportOutbound {
map<string, string> metadata = 1; // 可选,预留扩展,Dict,序列化协议由通信层统一实现
bytes payload = 2; // 二进制报文
string code = 3; // 状态码
string message = 4; // 状态说明
}

service PrivateTransferTransport {
rpc peek (PeekInbound) returns (TransportOutbound);
rpc pop (PopInbound) returns (TransportOutbound);
rpc push (PushInbound) returns (TransportOutbound);
rpc release (ReleaseInbound) returns (TransportOutbound);
}
84 changes: 81 additions & 3 deletions doc/2.0/osx/osx-tls.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,11 +130,89 @@ two-way TSL:
#### 完成以上步骤您将生成如下证书:
​ server文件夹包含: identity.jks 、server.cer、truststore.jks。
​ server文件夹包含: identity.jks 、server.cer、truststore.jks; 其中 identity.jks为私钥密码箱,truststore.jks 信任证书密码箱,server.cer为服务器证书(如果使用密码箱,无需配置此文件)
​ client文件夹包含: identity.jks 、client.cer、truststore.jks。
​ client文件夹包含: identity.jks 、client.cer、truststore.jks;其中 identity.jks为私钥密码箱,truststore.jks 信任证书密码箱,client.cer为客户端证书(如果使用密码箱,无需配置此文件)
## 2)方式二:单独文件存储私钥、证书、信任证书方式
#### 生成ca.key、ca.crt、client.crt、client.csr、client.key、client.pem、server.crt、server.csr、server.key、server.pem 命令如下:
```
ca.key
生成CA自己的私钥 root ca.key
# openssl genrsa -out ca.key 2048

ca.crt
根据CA自己的私钥生成自签发的数字证书,该证书里包含CA自己的公钥。
# openssl req -x509 -new -nodes -key ca.key -subj "/CN=osx" -days 5000 -out ca.crt

server.key
服务端的私钥和数字证书(由自CA签发)
生成服务端私钥
# openssl genrsa -out server.key 2048
将其转换成 pkcs8 格式,供java程序使用
#openssl pkcs8 -topk8 -inform PEM -outform PEM -in server.key -out server_pkcs8.key -nocrypt

server.csr
生成Certificate Sign Request,CSR,证书签名请求。
# openssl req -new -key server.key -subj "/CN=osx" -out server.csr

server.crt
自CA用自己的CA私钥对服务端提交的csr进行签名处理,得到服务端的数字证书server.crt
您的服务器还使用 IP 地址,请根据需要添加:
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 5000 \
-extfile <(printf "subjectAltName=DNS:grpcpro1.com,IP:your_server_ip")

要对客户端数字证书进行校验,首先客户端需要先有自己的证书。我们以上面的例子为基础,生成客户端的私钥与证书。
client.key
# openssl genrsa -out client.key 2048
将其转换成 pkcs8 格式,供java程序使用
# openssl pkcs8 -topk8 -inform PEM -outform PEM -in client.key -out client_pkcs8.key -nocrypt

client.csr
# openssl req -new -key client.key -subj "/CN=osx" -out client.csr

client.crt
# openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days 5000


```
产物:
CA:
私钥文件 ca.key
数字证书 ca.crt
Server:
私钥文件 server.key、server_pkcs8.key(实际配置此格式私钥)
数字证书 server.crt
client:
私钥文件 client.key、client_pkcs8.key(实际配置此格式私钥)
数字证书 client.crt
Loading

0 comments on commit fd6ce1f

Please sign in to comment.