Repo transferred to apache and renamed #174

dingelish · 2019-10-31T19:11:05Z

Hi all,

This project now belongs to apache, and has a new repo path at: https://github.com/apache/mesatee-sgx

Currently if one uses the forked crates, he/she still needs to use the old repo URL as git = "https://github.com/baidu/rust-sgx-sdk", rev = "v1.0.9", because all the forked crates are using the old repo URL.

Now, mixed usage of this SDK, like the following Cargo.toml would trigger collision between two sgx_tstd:

sgx_tstd = { rev = "v1.0.9", git = "https://github.com/apache/mesatee-sgx.git" }
wasmi = { git = "https://github.com/mesalock-linux/wasmi-sgx" }

You'll see

error: the `#[global_allocator]` in sgx_tstd conflicts with this global allocator in: sgx_tstd
error: duplicate lang item in crate `sgx_tstd`: `f32_runtime`.
  |
  = note: first defined in crate `sgx_tstd`.

error: duplicate lang item in crate `sgx_tstd`: `f64_runtime`.
  |
  = note: first defined in crate `sgx_tstd`.

error: duplicate lang item in crate `sgx_tstd`: `panic_impl`.
  |
  = note: first defined in crate `sgx_tstd`.

error: duplicate lang item in crate `sgx_tstd`: `begin_panic`.
  |
  = note: first defined in crate `sgx_tstd`.

error: duplicate lang item in crate `sgx_tstd`: `oom`.
  |
  = note: first defined in crate `sgx_tstd`.

error: aborting due to 6 previous errors

error: Could not compile `Helloworldsampleenclave`.

So I'm planning to have thorough update on all of the forked crates to use the new repo URL, and it would affect all of the projects which are depending on the old URL.

Any ideas?

Best,
Yu

The text was updated successfully, but these errors were encountered:

brenzi · 2019-11-01T07:06:00Z

Congratulations for the Apache tribute. Any chance you can update mesalock crates along with the next version bump? However, this is a delicate issue for us. We'd be blocked if this transition doesn't work smoothly. However, we did lock down our mesalock dependencies to specific git commits, so this might be less of an issue

dingelish · 2019-11-04T21:38:36Z

Thanks @brenzi !

I'm thinking of use a permanent (at least I think it's permanent) domain to host the redirect. I have a candidate rustsgx.com which triggers 301 redirect to https://github.com/apache/mesatee-sgx now. So the following git clone command works:

$ git clone https://rustsgx.com
Cloning into 'rustsgx.com'...
warning: redirecting to https://github.com/apache/mesatee-sgx/
remote: Enumerating objects: 363, done.
remote: Counting objects: 100% (363/363), done.
remote: Compressing objects: 100% (256/256), done.
remote: Total 15613 (delta 125), reused 212 (delta 71), pack-reused 15250
Receiving objects: 100% (15613/15613), 40.78 MiB | 4.87 MiB/s, done.
Resolving deltas: 100% (8118/8118), done.

Seems that git can work with HTTP 301 redirect. However, cargo does not allow me to do that. If I have sgx_types = { git = "http://rustsgx.com" } in Cargo.toml, error would be:

    Updating git repository `http://rustsgx.com/`
warning: spurious network error (2 tries remaining): cross host redirect not allowed; class=Net (12)
warning: spurious network error (1 tries remaining): cross host redirect not allowed; class=Net (12)
error: failed to load source for a dependency on `sgx_tstd`

Caused by:
  Unable to update http://rustsgx.com/

Caused by:
  failed to fetch into /home/ding/.cargo/git/db/_empty-f3f08613514edcc1

Caused by:
  cross host redirect not allowed; class=Net (12)

Any ideas?

dingelish · 2019-11-04T21:48:03Z

Just find a flag in .cargo/config which enables command line git instead of the built-in libgit-sys:

[net]
git-fetch-with-cli = true

If we enable this, we can directly use git = "https://rustsgx.com"

cargo build --release
    Updating git repository `https://rustsgx.com/`
    Updating crates.io index
   Compiling cc v1.0.46
   Compiling sgx_types v1.0.9 (https://rustsgx.com/#1d8a9f87)
   Compiling sgx_build_helper v0.1.0 (https://rustsgx.com/#1d8a9f87)
   Compiling sgx_tstd v1.0.9 (https://rustsgx.com/#1d8a9f87)
   Compiling sgx_demangle v1.0.9 (https://rustsgx.com/#1d8a9f87)
   Compiling sgx_libc v1.0.9 (https://rustsgx.com/#1d8a9f87)
   Compiling sgx_unwind v0.0.1 (https://rustsgx.com/#1d8a9f87)
   Compiling sgx_trts v1.0.9 (https://rustsgx.com/#1d8a9f87)
   Compiling sgx_alloc v1.0.9 (https://rustsgx.com/#1d8a9f87)
   Compiling sgx_tprotected_fs v1.0.9 (https://rustsgx.com/#1d8a9f87)
   Compiling sgx_backtrace_sys v1.0.9 (https://rustsgx.com/#1d8a9f87)
   Compiling Helloworldsampleenclave v1.0.0 (/home/ding/rust-sgx-sdk/samplecode/hello-rust/enclave)
    Finished release [optimized] target(s) in 15.09s

Is this a good choice?
Pros: no worry about any repo transfer in future. I'll pay for the domain registration and update the redirection if further repo transfer happens.
Cons: Extra config line in .cargo/config required.

@bradyjoestar @elichai @akhilles @davidp94 @cbeck88 @jasl

elichai · 2019-11-04T22:00:57Z

@moriaab @AvishaiW

jasl · 2019-11-04T22:04:43Z

I would prefer don't change global settings,

if I remember correctly, developers no need to do anything after you transferred the repo, GitHub will do the redirection automatically.

dingelish · 2019-11-04T22:06:09Z

@jasl You're right about git. But cargo is stupid. "github.com/baidu/rust-sgx-sdk" and "github.com/apache/mesatee-sgx" are considered as two different crates. It'll lead to panic handler collisions and mess up the dependency tree.

dingelish · 2019-11-05T06:35:10Z

This is a patch which you guys can apply to hello-rust sample. It shows some usage on "sdk.rustsgx.com" and "repo.rustsgx.com"

diff --git a/samplecode/hello-rust/enclave/.cargo/config b/samplecode/hello-rust/enclave/.cargo/config
new file mode 100644
index 0000000..c91c3f3
--- /dev/null
+++ b/samplecode/hello-rust/enclave/.cargo/config
@@ -0,0 +1,2 @@
+[net]
+git-fetch-with-cli = true
diff --git a/samplecode/hello-rust/enclave/Cargo.toml b/samplecode/hello-rust/enclave/Cargo.toml
index abd6c59..4c932cc 100644
--- a/samplecode/hello-rust/enclave/Cargo.toml
+++ b/samplecode/hello-rust/enclave/Cargo.toml
@@ -10,32 +10,36 @@ crate-type = ["staticlib"]
 [features]
 default = []

+[dependencies]
+log = { git = "https://repo.rustsgx.com/log-sgx" }
+
 [target.'cfg(not(target_env = "sgx"))'.dependencies]
 sgx_types = { git = "https://github.com/baidu/rust-sgx-sdk.git" }
-sgx_tstd = { git = "https://github.com/baidu/rust-sgx-sdk.git" }
+sgx_tstd  = { git = "https://github.com/baidu/rust-sgx-sdk.git" }
+
 [patch.'https://github.com/baidu/rust-sgx-sdk.git']
-sgx_alloc = { path = "../../../sgx_alloc" }
-sgx_build_helper = { path = "../../../sgx_build_helper" }
-sgx_cov = { path = "../../../sgx_cov" }
-sgx_crypto_helper = { path = "../../../sgx_crypto_helper" }
-sgx_libc = { path = "../../../sgx_libc" }
-sgx_rand = { path = "../../../sgx_rand" }
-sgx_rand_derive = { path = "../../../sgx_rand_derive" }
-sgx_serialize = { path = "../../../sgx_serialize" }
-sgx_serialize_derive = { path = "../../../sgx_serialize_derive" }
-sgx_serialize_derive_internals = { path = "../../../sgx_serialize_derive_internals" }
-sgx_tcrypto = { path = "../../../sgx_tcrypto" }
-sgx_tcrypto_helper = { path = "../../../sgx_tcrypto_helper" }
-sgx_tdh = { path = "../../../sgx_tdh" }
-sgx_tkey_exchange = { path = "../../../sgx_tkey_exchange" }
-sgx_tprotected_fs = { path = "../../../sgx_tprotected_fs" }
-sgx_trts = { path = "../../../sgx_trts" }
-sgx_tse = { path = "../../../sgx_tse" }
-sgx_tseal = { path = "../../../sgx_tseal" }
-sgx_tservice = { path = "../../../sgx_tservice" }
-sgx_tstd = { path = "../../../sgx_tstd" }
-sgx_tunittest = { path = "../../../sgx_tunittest" }
-sgx_types = { path = "../../../sgx_types" }
-sgx_ucrypto = { path = "../../../sgx_ucrypto" }
-sgx_unwind = { path = "../../../sgx_unwind" }
-sgx_urts = { path = "../../../sgx_urts" }
+sgx_alloc                       = { git = "https://sdk.rustsgx.com" }
+sgx_build_helper                = { git = "https://sdk.rustsgx.com" }
+sgx_cov                         = { git = "https://sdk.rustsgx.com" }
+sgx_crypto_helper               = { git = "https://sdk.rustsgx.com" }
+sgx_libc                        = { git = "https://sdk.rustsgx.com" }
+sgx_rand                        = { git = "https://sdk.rustsgx.com" }
+sgx_rand_derive                 = { git = "https://sdk.rustsgx.com" }
+sgx_serialize                   = { git = "https://sdk.rustsgx.com" }
+sgx_serialize_derive            = { git = "https://sdk.rustsgx.com" }
+sgx_serialize_derive_internals  = { git = "https://sdk.rustsgx.com" }
+sgx_tcrypto                     = { git = "https://sdk.rustsgx.com" }
+sgx_tcrypto_helper              = { git = "https://sdk.rustsgx.com" }
+sgx_tdh                         = { git = "https://sdk.rustsgx.com" }
+sgx_tkey_exchange               = { git = "https://sdk.rustsgx.com" }
+sgx_tprotected_fs               = { git = "https://sdk.rustsgx.com" }
+sgx_trts                        = { git = "https://sdk.rustsgx.com" }
+sgx_tse                         = { git = "https://sdk.rustsgx.com" }
+sgx_tseal                       = { git = "https://sdk.rustsgx.com" }
+sgx_tservice                    = { git = "https://sdk.rustsgx.com" }
+sgx_tstd                        = { git = "https://sdk.rustsgx.com" }
+sgx_tunittest                   = { git = "https://sdk.rustsgx.com" }
+sgx_types                       = { git = "https://sdk.rustsgx.com" }
+sgx_ucrypto                     = { git = "https://sdk.rustsgx.com" }
+sgx_unwind                      = { git = "https://sdk.rustsgx.com" }
+sgx_urts                        = { git = "https://sdk.rustsgx.com" }

dingelish · 2019-11-05T06:37:09Z

the .cargo/config only affects the enclave itself, because it locates in the "enclave" folder, along with enclave's Cargo.toml. with the help of "sdk.rustsgx.com" and "repo.rustsgx.com", I can make all of the forked crates depends on a permanent "repo" address "sdk.rustsgx.com", and xref each other using "repo.rustsgx.com", no matter where the git repos are.

dingelish · 2019-11-05T06:40:30Z

if this solution is acceptable, I can try if a shorter domain name is affordable. it'll be more ergonomic.

brenzi · 2019-11-05T17:11:23Z

Yes, that cargo issue crosses me all the time. I'd be happy for some endorsing comments (or critique) on my suggestion to resolve this properly:
rust-lang/cargo#7497

I'm not sure a custom domain will resolve the underlying problem. It's just a workaround. I could live with changing the global setting but I don't think this is elegant because it might change other behaviour as well as a side effect

dingelish · 2019-11-05T18:51:59Z

@brenzi Yeah I've seen your comments and I agree with you.

btw, the .cargo/config can be local and no need to be global. it can only affect the enclave itself by just putting it together with the enclave's Cargo.toml

dingelish · 2019-11-05T21:54:49Z

the other choice is sgx.rs, sdk.sgx.rs and repo.sgx.rs

nabilschear · 2019-11-06T17:36:59Z

given that this project is now under the apache org, i don't see any reason why you'd want to transfer repos again. apache is pretty prestigious. So, why not just go with https://github.com/apache/mesatee-sgx in the forked crates?

dingelish · 2019-11-06T18:42:37Z

@nabilschear The reason is that "mesatee" is a brand name owned by Baidu, and Baidu does not donate it to Apache. So we have to change it soon.

besides, we may have option to make this sdk an isolated project and get rid of the prefix in a couple of years. in this case, we have to change the repo name again.

nabilschear · 2019-11-13T13:42:21Z

understood. Whatever the solution to the path problem, when will it roll out to all the forked crates?

dingelish · 2019-11-13T16:12:05Z

@nabilschear I will do this next week after I get back in town.

A new problem is that MesaTEE is being renamed... which means that the repo has to change its repo name again. So should we use the "sgx.rs" jumper, or keep up with every renaming? How do you think?

elichai · 2019-11-13T16:14:16Z

What about creating a registery?
https://doc.rust-lang.org/cargo/reference/registries.html

dingelish · 2019-11-13T16:19:39Z

@elichai Yes I'm thinking about it seriously!

I have done a lot of experiment on that. I think it should be do-able, but with less transparency of porting. Currently, the forked crates can be "compared" directly with the upstream, and the diff reveals the difference clearly just in one click.

Another though is that some crates are forked while some others are not. Does Cargo supports "partially overrided"?

elichai · 2019-11-13T16:21:45Z

Well you can still keep them on github.
Just publish them afterwards.
What do you mean partially ovrrided?

(btw I would first spend time reaserching the --sysroot new feature I linked before, I still have hopes for that)

dingelish · 2019-11-13T16:24:45Z

In the previous crates.io design, a create published on crates.io can only depends on crates.io crates.

So can I publish a crate on sgx's crates.io, but make it depends on crates hosted on official crates.io?

dingelish · 2019-11-13T16:25:05Z

regarding to the --sysroot feature, I'm trying it as well :-)

elichai · 2019-11-13T16:32:15Z

@dingelish that's a great question. Can we use 2 separate crates.io together? Idk.

tomtau mentioned this issue Dec 20, 2019

Problem: repo uses 1.1.0-beta SGX SDK (CRO-675) crypto-com/thaler#712

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repo transferred to apache and renamed #174

Repo transferred to apache and renamed #174

dingelish commented Oct 31, 2019

brenzi commented Nov 1, 2019

dingelish commented Nov 4, 2019

dingelish commented Nov 4, 2019 •

edited

Loading

elichai commented Nov 4, 2019 •

edited

Loading

jasl commented Nov 4, 2019

dingelish commented Nov 4, 2019

dingelish commented Nov 5, 2019

dingelish commented Nov 5, 2019 •

edited

Loading

dingelish commented Nov 5, 2019

brenzi commented Nov 5, 2019

dingelish commented Nov 5, 2019

dingelish commented Nov 5, 2019

nabilschear commented Nov 6, 2019

dingelish commented Nov 6, 2019 •

edited

Loading

nabilschear commented Nov 13, 2019

dingelish commented Nov 13, 2019

elichai commented Nov 13, 2019

dingelish commented Nov 13, 2019 •

edited

Loading

elichai commented Nov 13, 2019

dingelish commented Nov 13, 2019

dingelish commented Nov 13, 2019

elichai commented Nov 13, 2019

Repo transferred to apache and renamed #174

Repo transferred to apache and renamed #174

Comments

dingelish commented Oct 31, 2019

brenzi commented Nov 1, 2019

dingelish commented Nov 4, 2019

dingelish commented Nov 4, 2019 • edited Loading

elichai commented Nov 4, 2019 • edited Loading

jasl commented Nov 4, 2019

dingelish commented Nov 4, 2019

dingelish commented Nov 5, 2019

dingelish commented Nov 5, 2019 • edited Loading

dingelish commented Nov 5, 2019

brenzi commented Nov 5, 2019

dingelish commented Nov 5, 2019

dingelish commented Nov 5, 2019

nabilschear commented Nov 6, 2019

dingelish commented Nov 6, 2019 • edited Loading

nabilschear commented Nov 13, 2019

dingelish commented Nov 13, 2019

elichai commented Nov 13, 2019

dingelish commented Nov 13, 2019 • edited Loading

elichai commented Nov 13, 2019

dingelish commented Nov 13, 2019

dingelish commented Nov 13, 2019

elichai commented Nov 13, 2019

dingelish commented Nov 4, 2019 •

edited

Loading

elichai commented Nov 4, 2019 •

edited

Loading

dingelish commented Nov 5, 2019 •

edited

Loading

dingelish commented Nov 6, 2019 •

edited

Loading

dingelish commented Nov 13, 2019 •

edited

Loading